Assessing the Effect of Five Gasoline
Properties on Exhaust Emissions from
Light-Duty Vehicles Certified to Tier 2
Standards:
Analysis of Data from EPAct Phase 3
(EPAct/V2/E'89)
Final Report
&EPA
United States
Environmental Protection
Agency
-------
Assessing the Effect of Five Gasoline
Properties on Exhaust Emissions from
Light-Duty Vehicles Certified to Tier 2
Standards:
Analysis of Data from EPAct Phase 3
(EPAct/V2/E'89)
Final Report
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
NOTICE
This technical report does not necessarily represent final EPA decisions or
positions. It is intended to present technical analysis of issues using data
that are currently available. The purpose in the release of such reports is to
facilitate the exchange of technical information and to inform the public of
technical developments.
United States
Environmental Protection
Agency
EPA-420-R-13-002
April 2013
-------
Contents
Executive Summary 1
Analysis 2
Results for Regulated Emissions, Total Hydrocarbons and Methane 3
Results for Toxic Compounds 8
1 Introduction 13
1.1 Mandate and Scope 13
1.2 Development of the Fuel Matrix 14
1.3 Selection of the Vehicle Sample 15
1.4 Study Execution 16
1.5 Overview of the Report 17
2 Study Design 19
2.1 Design Optimization 19
2.2 Measurements 23
2.2.1 Regulated Emissions, Total Hydrocarbons and Methane 23
2.2.2 Hydrocarbon speciation 24
2.3 Correlations Among Fuel Parameters 24
2.3.1 Standardization of Fuel Parameters 28
3 Dataset Construction 32
3.1 Selection of Aggregate (Bag) vs. Continuous Data 32
3.2 Imputation of Speciated Hydrocarbons (NMOG, NMHC) 35
3.2.1 Imputation of NMOG 36
3.2.2 Imputation of NMHC 43
4 Data Review 49
4.1 NOX (Bagl) 49
4.1.1 Linear Effects 50
4.1.2 Interactions 51
4.2 NO* (Bag 2) 63
4.2.1 Linear Effects 63
-------
4.2.2 Interactions 63
4.3 Paniculate Matter (PM, Bag 1) 75
4.3.1 Linear Effects 75
4.3.2 Interactions 76
5 Preliminary Modeling 89
5.1 Assumptions 89
5.2 Identification of Influential Observations 92
5.3 Reduced Models 96
5.3.1 Minimal Censoring (Mixed Models) 97
5.3.2 Severe Censoring (Tobit Regression) 98
5.4 Initial Modeling: Summary 100
5.5 Initial Modeling: Influence Analysis 100
6 Measurement Issues 114
6.1 Data Quality at Very Low Emission Levels 114
6.1.1 NOX 114
6.1.2 Particulate Matter 118
6.1.3 NMOGandNMHC 122
6.2 Analyzer Drift 130
7 Final Modeling 134
7.1 Design Efficiency for Extended Models 134
7.2 Fitting Reduced Models 137
7.2.1 Guiding Assumptions 137
7.2.2 Methods 138
7.2.3 Coefficients for Reduced Models 164
7.3 Detailed Review and Interpretation 174
7.3.1 Example 1: NOx(Bag 1) 174
7.3.2 Example 2: CO (Bag 1) 181
8 Analyses for Speciated Hydrocarbon Compounds and Air Toxics 187
8.1 Measurements 187
8.2 Parameters and Design Efficiency 187
8.3 Additional Terms for Modeling 190
ii
-------
8.4 Correlations Among fuel parameters 190
8.5 Measurement Issues 190
8.6 Review of Data 193
8.6.1 Linear Effects 193
8.7 Model Fitting 201
8.7.1 The Estimated Dependent Variable Model (EDV) 202
8.7.2 Fitting by Backwards Elimination 203
8.7.3 Mixed models 203
8.7.4 Tobit regression 207
9 Summary and Conclusions 213
9.1 Regulated Emissions, Total Hydrocarbons and Methane 213
9.1.1 Modeling Results 213
9.1.2 Models Selected for Application 229
9.1.3 Comparison to Previous Results 231
9.2 Speciated Hydrocarbons and Air Toxics 234
9.2.1 Model Fitting under the Reduced Design 237
9.2.2 Models selected for Application 243
10 References 246
Appendices
Appendix A. The Fuel Matrix
Appendix B. Imputation of NMOG
B.I Imputing NMOG Measurements (Bag 1)
B.2 Imputing NMOG Measurements (Bag 2)
B.3 Imputing NMOG Measurements (Bag 3)
Appendix C. Imputation of NMHC
C.I Imputing NMOG Measurements (Bag 1)
C.2 Imputing NMOG Measurements (Bag 2)
in
-------
C.3 Imputing NMOG Measurements (Bag 3)
Appendix D. Influential Measurements
Appendix E. Preliminary Model Results
Appendix F. NO* Analyzer Drift
Appendix G. Oxides of Nitrogen (NOX)
G.I Effects Plots (Bag 1): All Vehicles
G. la Effects Plots (Bag 1): without Focus
G.2 Effects Plots (Bag 2): All Vehicles
G.2a Effects Plots (Bag 2): without Cobalt
G.4 Final Model Fitting (Bag 2)
G.5 Final Model Fitting (Bag 3)
Appendix H. Carbon Monoxide (CO)
H.I Effects Plots (Bag 1)
H.2 Effects Plots (Bag 2)
H.3 Effects Plots (Bag 3)
H.4 Final Model Fitting (Bag 2)
H.5 Final Model Fitting (Bag 3)
Appendix I. Hydrocarbons
I.I Total Hydrocarbons (THC)
I. la Effects Plots (Bag 1)
Lib Effects Plots (Bag 2)
Lie Effects Plots (Bag 3)
Lid Final Model Fitting (Bag 1)
Lie Final Model Fitting (Bag 2)
iv
-------
I. If Final Model Fitting (Bag 3)
1.2 Non-methane Organic Gases (NMOG)
I.2a Effects Plots (Bag 1)
I.2b Effects Plots (Bag 2)
I.2c Effects Plots (Bag 3)
I.2d Final Model Fitting (Bag 1)
I.2e Final Model Fitting (Bag 2)
I.2f Final Model Fitting (Bag 3)
1.3 Non-methane Hydrocarbons (NMHC)
I.3a Effects Plots (Bag 1)
I.3b Effects Plots (Bag 2)
I.3c Effects Plots (Bag 3)
1.3d Final Model Fitting (Bag 1)
I.3e Final Model Fitting (Bag 2)
I.3f Final Model Fitting (Bag 3)
1.4 Methane (CH4)
I.4a Effects Plots (Bag 1)
I.4b Effects Plots (Bag 2)
I.4c Effects Plots (Bag 3)
I.4d Final Model Fitting (Bag 1)
I.4e Final Model Fitting (Bag 2)
I.4f Final Model Fitting (Bag 3)
Appendix J. Particulate Matter (PM)
J.I a Effects Plots (Bag 1): including outliers
-------
J.lb Effects Plots (Bag 1): excluding outliers and censored values
J.2a Effects Plots (Bag 2): including extreme observations
J.2b Effects Plots (Bag 2): excluding extreme observations
J.3a Effects Plots (Bag 3): including extreme observations
J.3b Effects Plots (Bag 3): excluding extreme observations
J.4 Final Model Fitting (Bag 1)
J.5 Final Model Fitting (Bag 2)
J.6 Final Model Fitting (Bag 3)
Appendix K. Acetaldehyde
K.I Effects Plots (Bag 1)
K.2 Effects Plots (Bag 2)
K.3 Model Fitting (Bag 2)
Appendix L. Formaldehyde
L.I Effects Plots (Bag 1)
L.2 Effects Plots (Bag 2)
L.3 Model Fitting (Bag 1)
L.4 Model Fitting (Bag 2)
Appendix M. Acrolein
M.I Effects Plots (Bag 1)
M.2 Effects Plots (Bag 2)
Appendix N. Ethanol
N.I Effects Plots (Bag 1)
N.2 Effects Plots (Bag 2)
N.3 Model Fitting (Bag 1)
N.4 Model Fitting (Bag 2)
vi
-------
Appendix O. Benzene
O.I Effects Plots (Bag 1)
O.2 Effects Plots (Bag 2)
O.3 Model Fitting (Bag 1)
Appendix P. 1,3-Butadiene
P.I Effects Plots (Bag 1)
P.2 Effects Plots (Bag 2)
P.3 Model Fitting (Bag 1)
Appendix Q. Ethane
Q.I Effects Plots (Bag 1)
Q.2 Effects Plots (Bag 2)
Q.3 Model Fitting (Bag 1)
Q.4 Model Fitting (Bag 2)
vn
-------
Executive Summary
Since the early 1990's, a large body of data has demonstrated that the properties of gasoline fuels
have measurable effects on exhaust emissions from cars and trucks. Since that time, vehicle
technologies have changed substantially and increasingly stringent emissions standards have
been implemented, leading to marked reductions in exhaust emissions from motor vehicles. In
model year 2004, cars and light trucks certified to Federal Tier 2 emissions standards entered the
market. By 2017, we project that 70 percent of the car and light truck fleet will be comprised of
Tier 2 vehicles, accounting for 80 percent of total vehicle miles travelled (VMT).
Existing fuel-effects models, such as the EPA Predictive Model and the Complex Model1, were
developed using data representing 1990s-technology vehicles meeting the Tier 0 and Tier 1
emission standards, levels an order of magnitude higher than current (Tier 2-compliant)
r\
vehicles . With the fleet turning over to much lower-emitting vehicles, the Agency and
stakeholders were interested in generating a coherent body of updated fuel-effects data, to
provide the basis for generation of updated fuel effects models representing the gasoline vehicle
fleet at the time of the study. In addition, in the Energy Policy Act of 2005 (EPAct), Congress
required EPA to conduct the necessary research and develop updated models.
To carry out this effort, EPA entered a partnership with the Department of Energy (DOE) and the
Coordinating Research Council (CRC) to undertake the largest fuels research program conducted
since the Auto/Oil program in the early 1990s3. The program is aimed specifically at
understanding the effects of fuel property changes on regulated and selected unregulated exhaust
emissions from later technology vehicles certified to Tier 2 standards.
To allow estimation of selected fuel effects across their respective ranges, a statistically optimal
study design was developed to represent variation in five fuel properties: ethanol volume,
aromatic content, RVP, T50 and T90. These five parameters were selected based on previous
studies as having potential to affect exhaust emissions51. Ethanol, T50 and T90 were included in
the design at four, five and three levels, respectively, to allow assessment of potential nonlinear
effects on emissions. The remaining two fuel properties, aromatic content and RVP, were
measured at two levels each. A critical feature of the study design is that the properties of the
test fuels are assigned to span the ranges of in-use fuel properties, with the intent of providing a
basis for the development of statistical models capable of predicting emissions for the majority
of in-use fuels.
An initial sample of 19 vehicles was chosen with the intent of representing the latest-technology
light-duty vehicles sold at the time the program was launched (model year 2008). In terms of
regulatory standards, the sample was to conform on average to Tier-2 Bin-5 exhaust levels and
a Sulfur also affects exhaust emissions, but due to its impact on vehicles' catalysts, it is necessary to
assess the effects of sulfur separately from those of other fuel properties.
1
-------
employ a variety of emission control technologies, realized by including a range of vehicle sizes
and manufacturers. No additional criteria were used to select the individual test vehicles. Due to
budget constraints, the sample was reduced from 19 to 15 vehicles for the Phase-3 program. A
power analysis was performed using data from 15 vehicles retained from Phase 1, and results
suggested a power in the range of 0.7-0.8 for detecting a 25 percent relative difference at a 95%
confidence level. (During analysis, the confidence level was relaxed to 90%), increasing power
to detect smaller effects. After considering sales levels, vehicle and engine sizes, representation
of manufacturers and emissions sensitivity to ethanol, a set of 15 vehicles were used to generate
the full dataset over the 27 test fuels.
Phase 3 data collection was completed in June 2010. Emissions measured include carbon
dioxide (62), carbon monoxide (CO), total hydrocarbons (THC), methane (CH4), non-methane
hydrocarbons (NMHC), oxides of nitrogen (NOX), and particulate matter (PM2.5). Emissions
were measured on the LA92 test cycle at a nominal temperature of 75°F. In addition,
hydrocarbons were speciated for subsets of vehicles, allowing calculation of derived parameters
such as non-methane organic gases, as well as independent analyses of specific compounds
including acetaldehyde, formaldehyde, acrolein, benzene and 1,3-butadiene.
Analysis
Following the completion of data collection, construction of the dataset involved intensive
evaluation and quality assurance. Successive rounds of statistical modeling were applied to the
data, to achieve several goals, including identification of potential candidate models,
identification and review of outlying observations, identification and review of subsets of data
from influential vehicles, and identification of models including subsets of terms that best
explain the results obtained.
The analysis process involved ongoing consultation among EPA, DOE and CRC staff and
contractors. However, it should be noted that this report describes analyses performed and
conclusions reached by EPA independently of its partners, except where noted.
The models reported in this section are as parsimonious as the data and subject-matter
knowledge allow. That is to say, they do not include all possible terms, but rather subsets of
terms considered to give the best fit to the dataset. This approach was followed for several
reasons: (1) the candidate fuel effects identified for inclusion were selected because we
anticipated that they could be important for one or more emissions. However, we did not
anticipate that all fuel properties would be meaningful for all the compounds selected for
measurement. (2) Insofar as possible, it is highly desirable that the models selected be intelligible
and interpretable in terms of physical and chemical processes, and that the models include effects
describing important processes affecting emissions. (3) It is important to avoid "overfitting" of
models by including terms that may prove to represent study artifacts or random variation.
-------
Results for Regulated Emissions, Total Hydrocarbons and Methane
The Phase 3 study was conducted to assess the effects of fuel properties on the emissions of
vehicles certified to Tier-2 standards, primarily the Bin-5 standards. Reviewing the results of
statistical modeling the measured emissions, it is clear that such effects exist and are measurable.
It is important to note that the effects of different fuel properties are not cleanly separable. It is
difficult to modify one property in an actual fuel without affecting one or more of the others.
The study design and analysis of the data are structured so as to allow assessment of fuel effects
as though they were independent of each other. However, in interpreting or applying the models,
it is critical to consider the effects of all five fuel properties in conjunction with each other.
Consideration of single coefficients in isolation can easily result in misleading conclusions.
Tables ES-1 and ES-2 summarize model coefficients for the regulated pollutants, total
hydrocarbons and methane. The values in Table ES-1 represent results for "Cold-start"
emissions, based on results for Bag 1 of the LA92 cycle. Similarly, the values in Table ES-2
represent results for "hot-running" emissions, based on results from Bag 2 of the LA92. Results
for Bag 3 emissions are not presented, as review of results suggests that the models for Bag 3
may be less reliable than those in Bags 1 and 2, especially for PM and NOX. In addition Figures
ES-1 and ES-2 give qualitative summaries of the direction and size of the coefficients presented
in the tables.
It is important to note that the coefficients represent abstract quantities that cannot be directly
interpreted in terms of fuel properties themselves (% ethanol, psi, °F, etc.)b. However, the
coefficients for different fuel properties can be directly compared, allowing assessment of the
relative importance of the effects of the fuel properties on the emissions constituent modeled. A
positive coefficient indicates an increase in emissions with an increase in the fuel property, or a
decline in emissions with a decrease in the fuel property. Similarly, a negative coefficient
indicates a decrease in emissions with an increase in the fuel property, or an increase in
emissions with a decrease in the fuel property.
b The values presented are "standardized coefficients," representing the change in the natural logarithm of
emissions due to a change in the fuel property of one standard deviation, calculated with reference to the
fuel matrix used in the project.
-------
Table ES-1. Models representing "Cold-start" Emissions for the Regulated Pollutants1.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
Ze
7-a
7-r
Z5
Z9
zzee
2Z55
77
ZjZj QQ
77
Z-,Z-,er
Z%e5
ZZep
THC
-0.8664
0.0548
0.0676
-0.0445
0.1288
0.0183
0.0436
0.0736
0.0179
0.0445
0.0214
CH4
-3.0074
0.06994
-0.1053
-0.03275
0.07554
0.02844
0.05170
0.02088
0.01082
0.03048
NMOG
-0.95209
0.08019
0.08782
-0.04224
0.1345
0.04432
0.07579
0.01693
0.04653
NMHC
-1.0315
0.03094
0.09461
-0.04568
0.13689
0.02160
0.04612
0.07534
0.02045
0.04729
0.02441
CO
1.3466
-0.1049
-0.01242
-0.00762
-0.03273
-0.1571
0.07304
0.05358
0.02086
0.01596
0.1064
NO,1
-2.8594
0.06750
0.1339
0.04783
-0.02369
PM
0.6559
0.1582
0.3833
0.0550
0.2923
0.0935
Vehicle variance
Residual error
0 veh
G\
0.1325
0.06872
0.2855
0.03014
0.1224
0.07538
0.1266
0.07624
0.3920
0.07214
0.5925
0.1458
0.4251
1.0359
Models fit on basis of 11-term design model, representing results for Bag 1 on LA92 cycle.
2 Fit excluding the Ford Focus. See 6.1.1.
Table ES-2. Reduced Models representing "Hot-running" Emissions for the Regulated Pollutants1.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
Ze
Za
Zr
zs
Z9
zzee
ZZ55
77
ZjZjea
77
f-if-ier
ZZeJ
ZjZjQf)
THC'
-4.6533
0.0327
-0.0195
-0.0355
0.0501
0.0514
0.0337
CH4
-5.7075
0.05860
-0.09836
-0.02049
0.04394
0.02575
0.01227
0.008769
NMOG'
-5.2360
0.02673
0.03634
-0.04786
0.04915
0.07252
0.05349
0.02171
0.02586
NMHC'
-5.3253
0.03987
-0.05881
0.04548
0.08202
0.04774
CO
-1.3893
0.0913
0.0299
0.0261
0.0440
NO/
-4.5692
0.06299
0.04407
PM
-1.3107
0.1126
0.1662
0.1072
Vehicle variance
Residual error
2
0 veh
02e
0.8384
0.06717
1.1108
0.02518
0.8502
0.1310
0.9691
0.1708
1.9187
0.1256
0.4720
0.1836
0.7827
1.1337
Models fit on basis of 11-term design model, representing results for Bag 2 on LA92 cycle.
2 Fit excluding the Honda Odyssey and Toyota Sienna. See 6.1.3.
3 Fit excluding the Chevrolet Cobalt.
-------
Figure ES-1. Qualitative Summary of the Sign and Magnitude of Linear-Effects Coefficients for "Cold-
Start" (Bag 1) Reduced Models, based on the 11-term Design Model (NOTE: This figure does not
attempt to represent interaction terms).
Fuel Property
THC
NMOG
NMHC
CH
NO
PM
CO
Ethanol
o
Aromatics
O
RVP
T50
o
T90
= positive coefficient
= negative coefficient
= no effect
-------
Figure ES-2. Qualitative Summary of the Sign and Magnitude of Linear-Effects Coefficients for "Hot-
running" (Bag 2) Reduced Models, based on the 11-term Design Model (Note: This figure does not
attempt to represent interaction terms).
Fuel Property
THC
NMOG
NMHC
CH
NO
PM
CO
Ethanol
Aromatics
RVP
T50
"H r^ = positive coefficient
= negative coefficient
= no effect
In reviewing the tables and figures, we can make some generalizations with respect to the
individual fuel properties:
Ethanol: In most models, the linear-effect coefficients for ethanol are positive for both running
and start emissions, implying that increases in ethanol content would be associated with
increases in emissions (if the remaining fuel properties could be kept constant while increasing
the ethanol level). A conspicuous exception to the pattern is CO, which has a negative coefficient
for start emissions and no ethanol term for running emissions. Another exception is NMHC,
which no ethanol term for running emissions. For start emissions, the etOHxetOH quadratic term
is positive for all HC species and CO, imparting some curvature to ethanol trends for these
species.
Aromatics: The patterns for aromatics are less consistent. Coefficients are positive for most
models, with several exceptions for both start and running emissions. One exception is CO,
which has a small negative coefficient for start emissions and a larger positive coefficient for
running emissions. A second exception is THC, for which the start coefficient is positive and the
-------
running coefficient negative. Thirdly, coefficients for CH4 are large, negative and similar in size
for both start and running emissions, which is unique in implying that changes in ethanol have
similar relative effects on both start and running emissions. For start emissions, the interaction
between aromatics and ethanol appears in all models except PM. The start interaction terms are
consistent in size and positive in sign for all emissions except NOX.
RVP: A linear term is included for all pollutants except NOX and PM. The sign of the term is
consistently negative with a single exception for running CO, which has a positive term. For the
hydrocarbons, the size of the term is relatively consistent, although the coefficients for running
models tend to be somewhat smaller than those for start emissions. The interaction with ethanol
appears in two models for start emissions, but in no models for running emissions. In both start
models, the terms are small and positive.
T50: Coefficients for this property consistently positive, with the single exception of start CO,
and appears in all models except running NOX and PM. For start emissions, the effects are largest
for THC, NMOG and NMHC, and smaller for CH4, NOX and PM; for CO, the term is negative
and relatively small. For the hydrocarbons except CH4, T50 is the largest single term. For
running emissions, coefficients are positive but smaller than for start emissions. For the
hydrocarbon species, T50 shows a consistent reinforcement interaction with ethanol for start
emissions, for running emissions, the interaction applies only to NMOG. For start CO, the
interaction is present but acts as an interference, in that both linear terms are negative and the
interaction is positive.
T90: This term is unique in that it appears more frequently in models for running than for start
emissions, and in that it is sometimes larger in running models than start models. In the start
models, the term is large and positive for PM, small and positive for THC and NMHC, large and
negative for CO, and absent for the remaining models. In the running models, the term is large
and positive for the hydrocarbons except methane, small and positive for PM and CO, and absent
for NOX. The interaction between T90 and ethanol is retained in only two models for start
emissions, THC and NMHC, in which it is positive and similar in size. In both models, the linear
and interaction terms are all positive, qualifying this effect as a reinforcement interaction. The
T90 coefficient is largest for start PM, where it has a reinforcement interaction with the even
stronger aromatics effect.
In addition, it is possible to make some general points about the responses of exhaust emissions
to changing fuel properties that apply across the measured compounds and species, and for both
start and running emissions.
Other factors being equal, increasing ethanol is associated with an increase in
emissions, as indicated by the positive ethanol coefficients in most models, both for
running and start emissions.
-------
Other factors being equal, increasing volatility is associated with reductions in
(exhaust) emissions, as indicated by generally negative coefficients for RVP (and
generally positive coefficients for T50).
In relative terms, fuel effects are generally more pronounced for start than for running
emissions, as indicated by the fact that in most cases, the coefficients for Bag 1 models
are larger than their counterparts for Bag 2 models, with exceptions for the ethanol
coefficients for NOX and the aromatics coefficients for CH/i. If we assume that we can
validly make direct comparisons between coefficients between Bag-1 and Bag-2
models, this result may suggest that the effects of fuel properties are more pronounced
during engine starts than during running operation. One interpretation might be that
fuel effects could be damped by efficient operation of the catalyst after the engine
comes up to temperature.
It is important to consider the applicability and representativeness of the results. As noted above
the vehicle sample comprises a judgment sample of high-sales models from major manufacturers
in model year 2008. In terms of standards, the vehicles represent the emissions standards that are
most prevalent for light-duty vehicles, including Bins 3 and 5 (or equivalent LEV and ULEV
standards under LEV-II), as well as a single Bin 8. The selection of makes and models does not
qualify as a random sample, as limitations in the size of the study precluded drawing a
reasonably sized random sample of makes and models. Nonetheless, given the size of the
sample, it is likely that a well-designed judgment sample can perform as well as a random
sample.
Results for Toxic Compounds
Summary results for all compounds representing "cold-start" (Bag 1) and "hot-running" (Bag 2)
emissions are presented below in Tables ES-3 and ES-4. Qualitative summaries of the direction
and size of the coefficients are also shown in Figures ES-3 and ES-4.
-------
Table ES-3. Models representing "Cold-start" Emissions for Selected Air Toxics.
Model term
Intercept
etOH
Arom
RW
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RW
etOH x T50
etOH x T90
Notation
Intercept
zs
Za
Z-r
zs
Z9
zzee
ZZ55
77
^^ea
Z2er
22e5
ZZe9
Compound
Acetaldehyde1
-5.2323
0.81449
0.03483
-0.04170
0.08670
0.03801
-0.1669
0.06665
0.01840
0.02194
Formaldehyde1
-5.9771
0.2299
0.02822
-0.04718
0.1672
0.1302
0.05262
0.01651
-0.01627
0.02004
Acrolein1
-7.9338
0.2476
0.1122
-0.06450
0.1880
0.2489
-0.08310
-0.1186
0.04617
Ethanol1
-4.9080
1.4627
-0.06054
0.07029
-0.09923
-0.4970
0.1108
Benzene
-4.1029
-0.00468
0.4056
1,3-
Butadiene2
-5.8371
-0.01729
0.02673
Ethane
-4.3079
0.1204
-0.1728
0.04242
0.01133
0.1247
0.1004
0.2169
0.09531
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
22ar
22a,
22ag
22$g
22rg
0.03959
0.03489
0.05986
Vehicle
residual
02veh
C/s
0.1149
0.0885
0.3358
0.1407
0.1032
0.3629
0.1283
0.5730
0.2741
0.1873
0.2192
0.1089
0.1407
0.04970
Reduced models fit under the full design, including 15 vehicles measured on 27 fuels.
2 Full models fit under the reduced design, including 15 vehicles measured on 1 1 fuels. Note that these models do not include a
linear term for RVP, and do not include any 2nd -order terms.
Table ES-4. Models representing "Hot-Running" Emissions for Selected Air Toxics1.
Model term
Intercept
etOH
Arom
T50
T90
Notation
Intercept
Ze
Za
Z5
Z9
Com]
Acetaldehyde
-9.4189
0.1520
0.07991
-0.02997
-0.07836
Formaldehyde
-8.6574
0.08456
0.01575
0.01863
-0.08138
Acrolein
NO
MODEL
sound
Ethanol
-9.3072
0.9233
-0.3772
-0.01910
-0.3017
Benzene
NO
MODEL
1,3-
Butadiene
NO
MODEL
Ethane
-7.7241
0.07345
-0.1260
0.1815
0.1322
Vehicle
residual
0 veh
v\
0.05654
0.3814
0.08205
0.3762
0.3707
1.0889
2.6785
0.1458
1 Full models fit under the "reduced design," including 5 vehicles measured on 1 1 fuels.
-------
Figure ES-3. Qualitative Summary of the Sign and Magnitude of Linear-Effects Coefficients for "Cold-
Start" Emissions (Bag 1) (Note: This figure does not attempt to represent interaction terms).
Fuel Property
Acet.
Form.
Aero.
Ethanol
Benz.
1,3-buta.
Ethanol
£
-o
Aromatics
RVP
T50
T90
= positive coefficient
= negative coefficient
= no effect
10
-------
Figure ES-4. Qualitative Summary of the Sign and Magnitude of Linear-Effects Coefficients for "Hot-
Running" Emissions (Bag 2) (Note: No models were fit for acrolein, benzene or 1,3-butadiene. This
figure does not attempt to represent interaction terms).
Fuel Property
Ethanol
Aromatics
RVP
T50
T90
Acet.
Form.
Aero.
Ethanol
Benz.
i,3-buta
"Hi
= positive coefficient
- negative coefficient
= no effect
The model results reflect the study design applied to each compound as well as the underlying
physico-chemical processes. The reduced model structures are more complex for those
compounds fit with the full design, specifically start emissions for the aldehydes, acrolein and
ethanol (Table ES-3). These models are discussed in more detail below.
Ethanol. The ethanol coefficients are positive and large for the aldehydes, acrolein and ethanol.
For acetaldehyde and ethanol, the ethanol effects are clearly dominant. These results are not
surprising, given the structural affinity between acetaldehyde and ethanol, and that the strongest
indicator of ethanol in the exhaust is ethanol in the fuel. For formaldehyde and acrolein, the
ethanol coefficients are important but not as dominant. Neither benzene nor 1,3-butadiene retain
ethanol coefficients in their reduced models. All compounds except formaldehyde retain large
and negative etOHxetOH quadratic terms, which are clearly required to fit the downward
curvature in the logarithmic trends.
Aromatics. In contrast to ethanol, the aromatics coefficients are small for the aldehydes, although
several times stronger for acrolein. Ethanol does not retain an aromatics term in its reduced
11
-------
model. Not surprisingly, the aromatics coefficient for benzene is large (Note that fuel benzene is
also a strong predictor of exhaust benzene, but was not a target study parameter). The two
aldehydes retain small but significant reinforcement interactions between aromatics and ethanol.
RVP. The sign and size of RVP coefficients are similar for all four compounds fit under the full
design (but absent for those fit under the reduced design). As with the RVP terms in the models
for aggregated hydrocarbons (THC, NMHC and NMOG), the signs of the RVP linear effects are
negative and similar in size to those for the aggregate HC (-0.04 to -0.06). The interaction
between ethanol and RVP is retained only in the acetaldehyde model, in which it is positive and
small.
T50. For the four compounds fit under the full design, linear-effect coefficients for T50 are
positive. However, the pattern in the size of the coefficients mirrors that for ethanol, in that the
two compounds with largest ethanol coefficients (acetaldehyde and ethanol) have smaller T50
coefficients than formaldehyde and acrolein, which have T50 coefficients about twice as large.
These results may reflect similarities in structure between the two pairs of compounds, or
similarities in formation processes during combustion. In addition to large linear coefficients,
formaldehyde and acrolein have small interference interactions between T50 and ethanol.
T90. More so than for the other properties, linear coefficients for T90 differ among the
compounds fit under the full design. The coefficients for acetaldehyde, formaldehyde and
acrolein are positive, but increasing, respectively, with the values for formaldehyde and acrolein
approximately 3 and 8 times larger than that for acetaldehyde. In contrast, the coefficient for
ethanol is negative, suggesting reduced ethanol emissions for less volatile fuels. In addition to
large linear effects for ethanol and T90, formaldehyde and acrolein have small reinforcement
interactions between these properties.
The structures for reduced models are much simpler for benzene, 1,3-butadiene and ethane,
reflecting the limits imposed by the reduced design. It is clear that in model fitting for these
compounds that only strong effects appear significant and are hence retained in the reduced
models.
Corresponding sets of coefficients for the hot-running models are shown in Table ES-4. As with
three of the cold-start models, these models are simpler, having been fit to a smaller data set
(fewer vehicles and fuels). For these emissions, it was not possible to fit RVP effects for any
compound, nor did we attempt to fit quadratic or interaction terms. For acrolein, benzene and
1,3-butadiene, no model fitting was attempted, given that large numbers of measurements for
these compounds were lower than background levels. Intercepts are much lower than for cold
start, showing the much lower emission levels during hot-running operation. The relative sizes
of the ethanol effects on the aldehydes and ethanol emissions for running are similar to those for
starts, i.e., ethanol emissions are affected most, followed by acetaldehyde and formaldehyde.
12
-------
1 Introduction
1.1 Mandate and Scope
Since the early 1990's, a wealth of data has been collected, demonstrating that the properties of
gasoline fuels, including aromatics, olefms, oxygenates, vapor pressure and distillation
parameters had measurable effects on exhaust emissions from cars and trucks. Since that time,
vehicle technologies have changed substantially and increasingly stringent emissions standards
have been implemented. The net result of these two factors has been marked reductions in
exhaust emissions from motor vehicles.
In model year 2004, cars and light trucks certified to Federal Tier 2 emissions standards (or their
equivalents under California LEV-II) entered the market, with phase-in of the standards slated
for completion by MY 2010. By 2017, we project that 70 percent of the car and light truck fleet
will be comprised of Tier 2 vehicles, accounting for 80 percent of total vehicle miles travelled
(VMT). Uncertainty remained as to whether vehicles employing improved technologies and
certified to the new standards would respond to fuel property changes in ways similar to vehicles
employing older technologies and certified to less stringent standards.
Prior fuel-effects models, such as the EPA Predictive Model and the Complex Model1, were
developed using data representing 1990s-technology vehicles meeting the Tier 0 and Tier 1
emission standards, levels an order of magnitude higher than current (Tier 2-compliant)
r\
vehicles . With the fleet turning over to much lower-emitting vehicles, the Agency and
stakeholders were interested in generating a coherent body of updated fuel-effects data, to
provide the basis for generation of updated fuel effects models representing the gasoline vehicle
fleet at the time of the study. In addition, in the Energy Policy Act of 2005 (EPAct), Congress
required EPA to conduct the necessary research and develop updated models.
To carry out this effort, EPA entered a partnership with the Department of Energy (DOE) and the
Coordinating Research Council (CRC) to undertake the largest fuels research program conducted
since the Auto/Oil program in the early 1990s3. The program is aimed specifically at
understanding the effects of fuel property changes on regulated and selected unregulated exhaust
emissions from later technology vehicles certified to Tier 2 standards. The resulting study was
dubbed the "EPAct/V2/E-89" program, with the three components of the label representing the
designations of the study by the three partners, EPA, DOE and CRC, respectively.
The program was conducted in three phases. Phases 1 and 2 were pilot efforts involving
measurements on 19 light-duty cars and trucks on three fuels, at two temperatures4. This work
was completed at Southwest Research Institute between September 2007 and January 2009. The
preliminary efforts in Phases 1 and 2 laid the groundwork for design of a full-scale research
program. The full-scale program, involving the incorporation of experimental design, is
designated as Phase 3.
13
-------
This report describes the analysis of the dataset collected in Phase 3 of the EPAct/V2/E-89
program, conducted at Southwest Research Institute in San Antonio, Texas. A separate report
describing the program design and data collection activities is available5, but an overview is
provided below.
The analysis process involved ongoing consultation and collaboration among EPA, DOE and
CRC staff and contractors. However, it should be noted that this report describes analyses
performed and conclusions reached by EPA independently of its partners, except where noted.
1.2 Development of the Fuel Matrix
To allow estimation of selected fuel effects across their respective ranges, a fuel matrix was
developed to represent variation in five fuel properties: ethanol volume, aromatic content, RVP,
T50 and T90. These five parameters were selected based on previous studies as having potential
to affect exhaust emissions0. Prior studies also showed olefm content as playing an important
role. However, funding limitations precluded expanding the fuel matrix to include a sixth
parameter.
Some fuel parameters have nonlinear impacts on some emissions. To capture this behavior,
three or more treatment levels of a given parameter must be included in the study design.
Statistical models of data from prior studies suggested that T50, T90, and ethanol content may
have nonlinear impacts on emissions. With support from DOE, the fuel matrix included four
levels of ethanol (0, 10, 15, and 20 percent by volume). In addition to the potential for nonlinear
emission impacts of T50 on emissions, five levels of T50 were also chosen to allow detailed
characterization of its relations with ethanol. Finally, due to concerns over potential nonlinear
effects of T90, CRC contributed additional funding to add fuels representing a third level of this
parameter. The remaining two fuel properties, aromatic content and RVP, were measured at
two levels each.
A critical point about the design of the program is that the properties of the test fuel are assigned
so as to span the boundaries of in-use fuel properties. This approach is designed specifically to
provide a basis for the development of statistical models capable of predicting emissions for the
majority of in-use fuels. The parameter ranges to be covered for T50, T90, aromatic content, and
RVP were selected to represent the range of in-use fuels based on a review of the Alliance of
Automobile Manufacturers' 2006 North American Fuel Survey. As the emissions tests were to
be performed at a nominal temperature of 75°F, summer survey data was used. Test fuel
parameter ranges were originally drafted to span roughly the 5th to 95th percentiles of survey
0 Sulfur also affects exhaust emissions, but due to its impact on vehicles' catalysts, it is necessary to
assess the effects of sulfur separately from those of other fuel properties.
d The intermediate level of T90 occurs along one edge of the fuel domain in Phase 3. Statistical analysis
of nonlinear T90 effects was intended to include a fuel used in Phase 1 of the program as an additional
source of data for the intermediate T90 level.
14
-------
results for U.S. gasoline, though some test fuel parameters were adjusted after the actual fuel-
blending process began. An intermediate level of T50 in EO fuels was selected to coincide with
the high level of T50 in E10 fuels. Similarly, an intermediate level of T50 in E10 fuels was
selected to coincide with the low level of T50 in EO fuels.
For E15 and E20 fuels, aromatics, RVP and T90 ranges selected for EO and E10 fuels were
applied. A single level of T50 was selected for E20 blends based on the information obtained
from a report compiled by the Coordinating Research Council6, as well as petroleum industry
sources which indicated that it was largely independent of the hydrocarbon fraction of the fuel
and would not deviate more than several degrees from 160°F due to the presence of a large
fraction of ethanol.6 At the time this fuel matrix was designed, no information was available on
distillation properties of E15 fuels. Two levels of T50 were selected for the E15 fuels, the low
level equal to the lowest T50 assumed for E10 fuels and the high level being a linear
interpolation between the highest T50 of E10 fuels and the sole T50 level of E20 fuels.
1.3 Selection of the Vehicle Sample
An initial sample of 19 vehicles was chosen with the intent of representing the latest-technology
light-duty vehicles sold at the time the program was launched (model year 2008). In terms of
regulatory standards, the sample was to conform on average to Tier-2 Bin-5 exhaust levels and
employ a variety of emission control technologies, realized by including a range of vehicle sizes
and manufacturers.
Engine family sales data obtained from EPA certification and Wards databases was analyzed to
generate a list of high-sales vehicles as candidates for inclusion. Grouping sales data by engine
family allowed additional transparency and flexibility in choosing test vehicles that represent a
wider group than one specific make and model. No additional criteria were used to select the
individual test vehicles for lease.
Due to budget constraints, the sample was reduced from 19 vehicles for the Phase-3 program. A
power analysis was performed using data from 15 vehicles retained from Phase 1, and results
suggested a power in the range of 0.7-0.8 for detecting a 25 percent relative difference at a
confidence level of 0.05. (During analysis, relaxing the confidence level to 0.10 effectively
increases power for the effect of the same size, and increases power for smaller effects.
e As ethanol blend level moves beyond 10 vol%, T50 becomes increasingly correlated (inversely) with
ethanol content. At El5, the two can be manipulated independently with some effort within a relatively
limited range. By E20, the behavior of the center of the distillation curve (where T50 lies) is dominated
by ethanol's boiling point, and thus T50 cannot be moved outside a narrow range around 165°F. Thus,
T50 and ethanol should only be understood to be independently blended parameters at E10 and below.
f Engine family (or "test group") is a term used in manufacturing and certification to describe a
combination of a base engine and after-treatment system that may be used in several vehicle makes and
models offered by a manufacturer.
15
-------
Statistical analysis of Phase 1 data found significant fuel effects smaller than 25 percent, so
assigned relative difference shouldn't be understood as a lower limit of detectable effects, but
rather as a screen for the largest effect that is unlikely to be missed at the assigned power level.)
Reduction of the sample required choosing four vehicles to eliminate. Primary considerations in
this process included retaining high-sales engine families, a balance of vehicle and engine sizes,
and maintaining representation of all manufacturers originally included in order to represent a
range of technologies and emission control strategies. There was also consideration of the fact
that changes in the sample could shift the average program results. To explore this issue, all
nineteen vehicles were ranked according to their NOx and NMHC sensitivity to fuel ethanol
level based on the Phase 1 data, with the intent of avoiding removal of several vehicles with
similar emissions behavior (though the two pollutants could provide conflicting direction). In
the end, a set of 15 vehicles were used to generate the full dataset over the 27 test fuels.
Table 1. Candidate Vehicles for the Phase-3 EPAct Program; all vehicles in MY2008 (the four
highlighted Vehicles not included in the Phase 3 Vehicle Sample).
Make
GM
GM
GM
GM
Toyota
Toyota
Toyota
Toyota
Ford
Ford
Ford
Ford
Chrysler
Chrysler
Chrysler
Honda
Honda
Honda
Nissan
Brand
Chevrolet
Chevrolet
Saturn
Chevrolet
Toyota
Toyota
Toyota
Toyota
Ford
Ford
Ford
Ford
Dodge
Dodge
Jeep
Honda
Honda
Honda
Nissan
Model
Cobalt
Impala FFV
Outlook
Silverado
FFV
Corolla
Camry
Sienna
Tundra
Focus
Taurus
Explorer
F150 FFV
Caliber
Caravan FFV
Liberty
Civic
Accord
Odyssey
Altima
Program ID
CCOB
CIMP
SOUT
CSIL
TCOR
TCAM
TSIE
TTUN
FFOC
FTAU
FEXP
F150
DCAL
DCAR
JLIB
HCIV
HACC
HODY
NALT
Engine
Size
2.2L 14
3.5LV6
3.6LV6
5.3LV8
1.8LI4
2.4L 14
3.5LV6
4.0L V6
2.0L 14
3.5LV6
4.0L V6
5.4L V8
2.4L 14
3.3LV6
3.7LV6
1.8LI4
2.4L 14
3.5LV6
2.5L 14
Engine Family
8GMXV02.4025
8GMXV03.9052
8GMXT03.6151
8GMXT05.3373
8TYXV01.8BEA
8TYXV02.4BEA
8TYXT03.5BEM
8TYXT04.0AES
8FMXV02.0VD4
8FMXV03.5VEP
8FMXT04.03DB
8FMXT05.44HF
8CRXB02.4MEO
8CRXT03.3NEP
8CRXT03.7NEO
8HNXV01.8LKR
8HNXV02.4TKR
8HNXT03.54KR
8NSXV02.5G5A
Tier 2
Bin
5
5
5
5
5
5
5
5
4
5
4
8
5
8
5
5
5
5
5
LEVII
Std
NA
L2
L2
NA
U2
U2
U2
U2
NA
NA
NA
NA
U2
U2
L2
Phase 3
Starting
odometer
4,841
5,048'
5,212'
5,3472
5,019'
4,9742
4,997
5,150''2
6,7993
5,523'
4,959
5,2824
4,785
4,765
4,850
5,2112
These vehicles added to the Phase 3 sample at a later date. Prior to inclusion, they received on-road mileage accumulation every other week.
2 These vehicles included in an FTP interim program conducted between Phases 1 and 2.
3 During Phase 1, the initial 4,000 miles of vehicle break-in was conducted with crankcase lubricant of the wrong viscosity grade. An additional
2,000-mile break-in was conducted with the correct lubricant.
4 This vehicle was measured only on E85 fuel.
1.4 Study Execution
In September 2007, Southwest Research Institute (SwRI) began work at their facilities in San
Antonio, Texas. By January 2009, SwRI had completed the pilot phases of the program (referred
16
-------
to as Phases 1 and 2). These phases involved testing of the 19 light duty cars and trucks from
Table 2-3 on three fuels, at two temperatures.
In March 2009, SwRI began Phase 3 of the program (also referred to as the full program) This
report covers work conducted for Phase 3, also known as EPAct/V2/E-89, which involved the
testing of the 15 vehicles listed in Table 1 (not shaded) on the set of 27 test fuels.g Phase 3 data
collection was completed in June 2010.
Emissions measured include carbon dioxide (CO2), carbon monoxide (CO), total hydrocarbons
(THC), methane (CH/j), non-methane hydrocarbons (NMHC), oxides of nitrogen (NO*), and
particulate matter (PIVb.s). Emissions were measured on the LA92 test cycle at a nominal
temperature of 75°F. In addition, hydrocarbons were speciated for subsets of vehicles, allowing
calculation of derived parameters such as non-methane organic gases, as well as independent
analyses of specific compounds including acetaldehyde, formaldehyde, acrolein, benzene and
1,3-butadiene.
1.5 Overview of the Report
This report describes the study design, dataset construction and subsequent analyses of data
obtained in the Phase-3 dataset.
Chapter 2 describes the design of the Phase 3 program. In addition to the considerations
discussed in 1.2 above, development of the fuel matrix involved the application of optimal
experimental design to develop a matrix allowing for estimation of targeted fuel effects with the
maximum attainable precision within budgetary and technical constraints. The experimental
design entailed an iterative process that involved balancing the study goals of the three partners,
(EPA, DOE and CRC) with technical limitations of fuel blending, in the context of experimental
design. This process is summarized in 2.1.
Section 2.2 identifies the specific sets of vehicles and fuels for which speciation of hydrocarbon
measurements was performed.
Section 2.3 describes an assessment of the degree of correlation among the fuel parameters in the
dataset. The study design effectively neutralized correlations among the fuel parameters
themselves, as intended. However, when quadratic or interactive terms are constructed from the
linear terms, some strong correlations result. As strong correlation among predictors can
adversely affect statistical models, we applied analytical techniques to neutralize the additional
correlations, allowing the analysis of fuel effects to proceed.
Chapter 3 describes specific topics involved in the construction of the dataset. One important
question, discussed in 3.1, was whether to base analyses of fuel effects on aggregate "Bag"
8 Phase 3 also included testing of four flexible-fuel vehicles (FFVs) from the 15-vehicle sample on an E85
fuel. Analysis of results obtained on the E85 fuel is not discussed in this document.
17
-------
measurements, on summed continuous measurements, or on some combination of the two. The
underlying issue concerned whether aggregate measurements became unreliable at the low end
of the measurement range, and whether the use or substitution of summed continuous
measurements would represent an improvement in data quality. After some consideration, study
participants elected to rely on the aggregate data, while applying appropriate techniques to
address the resulting "censoring" of the data at low end of the range of values.
A second issue was raised by the fact that only subsets of vehicles and fuels were speciated in
Bags 2 and 3. An implication of the situation was that the advantages of the full study design
would not be available in analysis of speciation-dependent parameters such as NMOG and
NMHC for the hot-running portions of the test. Given the importance of these parameters in
analysis and interpretation of the results, we elected to statistically impute missing measurements
from a related measure, "NMHC as measured by FID," (NMHCpio), to which both NMOG and
NMHC are very strongly correlated. These analyses are described in 3.2.
Prior to analysis and modeling, it is important to gain familiarity with the datasets. To achieve
this aim, the data was plotted in raw and aggregate forms. To assess the existence of "linear" or
"main effects," we averaged and plotted the data by each fuel property and by fuel. These plots
indicated whether specific fuel effects seemed evident when the data was averaged across the
remaining fuel properties. As an initial indication of the possibility of interactions between fuel
properties, we constructed "conditional effects" plots by averaging the data by two selected fuel
properties, repeating this step for multiple pairs of properties. Plotting and review for three
subsets of data: NO* (Bag 1), NO* (Bag 2) and PM (Bag 1) were selected for purposes of
illustration in this report. Similar plots for the remaining compounds are presented in
Appendices G-Q.
Chapter 5 describes initial modeling performed for the purposes of data exploration and
influence analysis. Influence analysis was used to identify influential observations and vehicles
for detailed review. This chapter also describes methods and criteria adopted for model fitting,
including issues such as treatment of outlying observations and the existence of "censoring" in
the data, which addresses loss of observations at the low end of the range due to limitations in
emissions measurement techniques.
Chapter 6 discusses measurement issues at the low end of the range of emissions highlighted by
the influence analysis for vehicles. Specific issues included the validity of sample measurements
falling into the range of background measurements, and the apparent drift of some measurements
during the course of such a lengthy project.
Chapter 7 covers a second round of model fitting, incorporating the findings of the analyses in
Chapters 5 and 6. This chapter applies the methods described in Chapter 5 to the development of
a set of "reduced models" which include subsets of terms contributing to the fit of the models to
the data, but excluding terms not found to significantly improve fit. This chapter also illustrates
18
-------
a more detailed review of models fit to the data, illustrated using the examples of NOX (Bag 1)
and CO (Bag 1).
Chapter 8 describes analyses for selected speciated hydrocarbons. These compounds included
aldehydes (acetaldehyde, formaldehyde and acrolein), ethanol, benzene, 1,3-butadiene and
ethane. The methods used were very similar to those used for modeling the other emissions,
with modifications to address issues of study design and measurement limitations specific to
these compounds.
Chapter 9 summarizes results of the model fitting and analysis based on reduced models
presented in Chapters 7 and 8.
2 Study Design
The design and implementation of the study, including the aspects of fuel blending, measurement
methods and logistics are described in a separate report5. This section focuses on the
experimental design of the fuel matrix.
2.1 Design Optimization
The EPAct Phase-3 program was conducted as a controlled experiment, for which the plan was
to analyze results in terms of fuel properties, which are abstracted from actual fuels and treated
as continuous numeric variables. Five fuel parameters were selected as experimental factors,
specifically: Ethanol content (%), Aromatics content (%), Reid Vapor Pressure (RVP, psi) , and
two distillation parameters, T50 (°F) and T90 (°F). The first two parameters represent the
chemical composition of the fuels, and the remaining three represent commonly measured bulk
physical parameters.
It is well known for fuel properties to be moderately to strongly correlated. This tendency stems
from the fact that it is impossible to modify one factor without also affecting one or more of the
others. As the goal is to enable analysis of fuel effects as though independent, and as statistical
models assume independence of factors and can be adversely affected by strong correlations
among factors, it is necessary to address these correlations in design and analysis. An important
implication for experimental design is that orthogonal factorial designs (full or partial) cannot be
constructed.
In such cases, which are not unusual in real applications, it is common practice to construct
"optimal" designs, generated by iterative computer algorithms and based on specific criteria7. It
is important to recognize that optimized designs are not unique - algorithms produce multiple
solutions to specific problems. It is also important to remember that in optimized designs the
h This parameter was measured as Dry Vapor Pressure Equivalent (DVPE), but for simplicity and
consistency, we will refer to it as RVP, which is numerically equivalent.
19
-------
parameters are approximately rather than fully orthogonal and that effects can be somewhat
correlated rather than uncorrelated.
In optimizing designs, it is necessary to specify a model to be analyzed using the study results, as
well as a criterion for evaluating the "efficiency" of the optimized design, relative to an
orthogonal fractional factorial. The criterion selected for design development for this study is "G
efficiency". This parameter attempts to minimize the maximum standard error for prediction
over the design points for the specified model. Efficiency is expressed as a percent, under the
assumption that a fractional factorial represents 100%.
For the five fuel parameters selected, design points were constructed based on the levels shown
in Table 2. A full factorial, based on this design, would include 4x22x5><3 = 240 fuels. Given
the level of effort and expense involved in vehicle emissions measurement, and limitations in
resources, it was not practical to perform measurements on this number of fuels. In addition, due
to relationships among fuel parameters, many fuels in the full factorial design either do not exist
or are not feasible to blend.
Table 2. Levels assigned to Experimental Factors (Fuel parameters) for the Phase-3 EPAct program.
Factor
Ethanol (%)
Aromatics
RVP (psi)
T50 (°F)
T90 (°F)
No. Levels
4
2
2
5
3
Levels
Low
0
15
7
150
300
Middle
10, 15
165, 190, 220
325
High
20
35
10
240
340
The final design is the result of an iterative process involving balancing among research goals,
fuel blending feasibility, and experimental design. The design was developed through a series of
steps, described below:
Step 1'. An initial design was optimized for a domain covering an ethanol range from 0-10%.
Out of a possible total of 64 possible fuels (in a full factorial), 20 were dropped due to issues
with blending feasibility. Of the 44 remaining fuels, the optimization gave a reduced set of 16
fuels, with G-efficiency estimated at 73%. In this step, the response model input to the
optimization process includes 10 terms and is shown in Equation 1.
7 =
Equation 1
20
-------
Step 2: The initial design was augmented to include a second domain covering an ethanol range
from 10-20%, while keeping the original 16-fuel design intact, and adding a quadratic term for
ethanol. At this point, we imposed additional design constraints: (1) that nine additional fuels be
added, to give a total of 25 fuels, and (2) that three of the nine fuels have 15% ethanol, and that
an additional three have 20% ethanol. To meet these constraints, it was not possible to assess
this modification through an optimization algorithm, as in the previous step. Rather, all sets of
fuels meeting these criteria were added to the design, and the efficiency evaluated for each whole
25-fuel design. The design selected had a G-efficiency of 69%.
7 = pQ
/?6T502+/?netOH2
Equation 2
/?7etOHx Arom + /?8etOHxRVP + /?9etOHx T50 + /?10etOHxT90 +
£
Step 3: a third quadratic term was added to the design model (T90xT90). Three fuels with pre-
specified parameters were added, plus two additional fuels to maximize efficiency, to give a total
of 30 fuels.
Step 4: As a final step, three fuels were removed from the 30-fuel matrix, to give a final total of
27 fuels. In addition, the exact fuel parameter levels assigned to certain fuels were modified to
improve blending feasibility. The G-efficiency of this modified design was evaluated at 51.6%.
This final fuel set was adopted for the Phase-3 EPAct program and is shown in Table 3. Note
that the values of the fuel parameters in the table are nominal target values, rather than actual
measured values, which varied slightly from the targets. Actual measured values, which were
used in the analysis, are shown in Table 4.
21
-------
Table 3. Nominal Target parameters for fuels in the Phase-3 EPAct program.
Fuel1
1
2
o
J
4
5
6
7
8
9
10
11
12
13
14
15
16
20
21
22
23
24
25
26
27
28
30
31
12-fuel subset2
etOH (%)
10
0
10
10
0
10
0
0
0
10
10
10
0
0
0
10
20
20
20
20
20
20
15
15
15
10
20
Aromatics (%)
15
15
15
15
35
15
15
15
35
35
35
35
35
15
35
35
15
35
15
15
15
35
35
15
35
35
35
RVP (psi)3
10
10
7
10
7
7
7
10
10
7
10
10
7
7
10
7
7
7
10
7
10
10
10
7
7
10
7
T50 (°F)
150
240
220
220
240
190
190
220
190
220
190
150
220
190
190
220
165
165
165
165
165
165
165
220
220
150
165
T90 (°F)
300
340
300
340
300
340
300
300
340
340
300
340
340
340
300
300
300
300
300
340
340
340
340
340
300
325
325
Note that numbering of fuels is not entirely sequential throughout.
Speciation performed for these fuels differs from that performed for the fuels not in the subset. See Table 5.
This parameter was measured as "DVPE," but for simplicity, will be referred to as "RVP" in this document.
22
-------
Table 4. Measured Parameters for Fuels in the Phase-3 EPAct Program.
Fuel1
1
2
o
J
4
5
6
7
8
9
10
11
12
13
14
15
16
20
21
22
23
24
25
26
27
28
30
31
etOH (%)
10.03
0
10.36
9.94
0
10.56
0
0
0
9.82
10.30
9.83
0
0
0
10.76
20.31
21.14
20.51
20.32
20.51
20.03
15.24
14.91
14.98
9.81
20.11
Aromatics (%)
15.4
14.1
15.0
15.5
34.7
15.0
17.0
15.7
35.8
34.0
35.0
34.8
34.1
16.9
35.3
35.6
15.2
35.5
15.0
15.9
15.3
35.2
35.6
14.9
34.5
35.5
35.5
RVP (psi)2
10.07
10.2
6.93
10.01
6.95
7.24
7.15
10.2
10.30
7.11
9.93
10.13
6.92
7.14
10.23
7.12
6.70
7.06
10.21
6.84
10.12
10.16
10.21
6.97
6.87
10.23
6.98
T50 (°F)
148.9
236.7
217.5
221.9
237.0
188.5
193.1
221.1
192.8
217.1
189.3
152.2
222.5
192.8
189.7
218.8
162.7
167.6
163.2
162.5
165.1
166.9
160.3
221.5
216.6
152.9
167.3
T90 (°F)
300.2
340.1
295.9
337.5
300.0
340.4
298.4
303.1
341.8
340.2
298.6
339.8
337.9
338.5
299.4
300.6
298.7
305.0
297.3
338.2
338.1
337.9
338.7
340.3
298.8
323.8
325.2
Note that numbering of fuels is not entirely sequential throughout.
2
This parameter was measured as "DVPE," but for simplicity, will be referred to as "RVP" in this document.
2.2 Measurements
2.2.1 Regulated Emissions, Total Hydrocarbons and Methane
The emissions measured on all vehicles and test fuels included carbon dioxide (CO2), carbon
monoxide (CO), total hydrocarbons (THC), methane (CH/i), oxides of nitrogen (NOX), and
particulate matter (PM^.s). Measurements were conducted on the LA92 cycle at a nominal
temperature of 75%. Measurement methods are discussed in detail in the testing report5. To
allow calculation of derived measurements such as non-methane hydrocarbons (NMHC) and
non-methane organic gases (NMOG), hydrocarbons were speciated for subsets of vehicles and
fuels, as described below.
23
-------
2.2.2 Hydrocarbon speciation
Due to the additional time and expense required, speciation was applied to results from a subset
of fuels. For the selected fuels, measurements from a subset of five vehicles were speciated.
Alcohols and carbonyls were speciated during bag 1 for all tests (all vehicles, fuels and
replicates). For the first replicate test, Ci-Ci2 hydrocarbons were speciated for all vehicles while
operated on a subset of twelve fuels which included Fuels 3, 4, 6, 7, 10, 13, 14, 21, 23, 27, 28,
and 31. This subset was selected to provide, as nearly as possible, useful comparisons between
differing levels of ethanol, aromatics, T50 and T90. In addition, all types of speciation were
conducted for Bags 2 and 3 on a subset of five vehicles over the 12-fuel subset. The vehicle
subset included the Honda Civic, Toyota Corolla, Chevrolet Impala, Ford F150, and Chevrolet
Silverado. These vehicles were selected to represent the range of sizes and technologies present
in the full vehicle sample. Table 5 summarizes the speciation schedule by vehicle. For additional
detail on speciation, see the Testing Report5.
Table 5. Speciation Schedule by Vehicle, Fuel, Bag and Replicate.
Vehicle Set
Speciation
Type
Replicate 1
Bagl
Bags 2-3
Replicate 2+
Bagl
Bags 2-3
Impala,
Silverado,
F150, Civic,
Corolla
Alcohols,
Carbonyls
Hydrocarbons
All fuels
12-fuel subset1
12-fuel subset1
12-fuel subset1
All fuels
-
-
-
Remaining
10 vehicles
Alcohols,
Carbonyls
Hydrocarbons
All fuels
12-fuel subset1
-
-
All fuels
-
-
-
'See Table 3.
2.3 Correlations Among Fuel Parameters
As mentioned, the purpose of this study is to attempt to relate emissions to changes in fuel
parameters, treated as continuous variables, using multiple regression as the analysis technique.
Thus, the "fuel-parameter matrix" is a set of properties abstracted from the set of fuels on which
measurements were performed. However, it is commonly observed that fuel properties tend to
be correlated. In addition, interaction terms often show strong correlations with the linear-effects
terms from which they were constructed. Such correlations can be an issue in analysis of the
24
-------
data as they result in collinearities among predictors, with potentially adverse effects on models
fit with collinear terms, such as reduced precision in estimation of model coefficients. In
extreme cases, coefficients can change substantially as additional terms are added to or removed
from models, or can be of the wrong magnitude or sign.
The first step taken to neutralize correlations among fuel parameters was the design and
optimization of the fuel-parameter matrix itself. A goal of the design process is to approximate,
as closely as possible, the efficiency of a full factorial design in which all model terms would be
independent and orthogonal. Taking the optimized full matrix as a starting point, we assessed
the potential for issues related to collinearity by constructing all candidate 2nd order terms and
then compiling a correlation matrix (R) of model terms. Results showed that a number of strong
correlations persist among model terms in the optimized parameter matrix, in which "strong" is
defined as | R > 0.50, as shown in Table 6. In addition to correlations of moderate strength
(0.50 < R < 0.75), a number of very strong correlations are apparent (R > 0.90). Not
unexpectedly, these results appear in quadratic terms, e.g., etOHxetOH and T5QxT50 are highly
correlated to etOH and T50, with,/? = 0.95 and 1.00, respectively. Additionally, strong
correlations appear among the several 2n order interaction terms. For example, etOH is strongly
correlated with all four of its interactions with the other fuel parameters.
In addition to correlations among the linear effects and interactions, and correlations among
interactions, we can see one fairly strong correlation among the linear effects, specifically,
between etOH and T50 (R = -0.57). This residual correlation reflects the physical relationship
between ethanol content, reflecting the hydrocarbon content of the fuel, and the T50, which as a
bulk property, is strongly influenced by ethanol, which tends to increase the volatility of the fuel.
Specifically, it is not possible to fully "orthogonalize" a fuel matrix between etOH and T50 for
etOH levels much above 10%, as fuels with high ethanol and high T50 cannot be blended. In the
full parameter-matrix, all fuels with 20% ethanol have T50 in the neighborhood of 165 °F. The
relationship between etOH and T50 is shown in Figure 1. For comparison, "rectangular"
relationships between ethanol and aromatics and RVP are shown in Figure 2 and Figure 3.
These plots show the greater level of success the optimization process achieved in
"orthogonalizing" relationships between etOH, aromatics and RVP, as opposed to T50.
Additional two-way plots of the fuel properties are presented in Appendix A.
25
-------
Table 6. Correlation matrix for linear-effect and interaction terms in the design fuel-parameter
matrix.
Ethanol
Arom
RV?
T5C
~5Z
at OH* St OH
T5C:-:T=C
7SC:-.73C
5tOH:-:T5C
atOH^TSC
5tOH:-:Arom
etOH:-.RV?
Ethane 1
LCC
Arcra
-;:.:-
l.CC
RV?
-C.I;
c.c;
1.00
Z. Q
-0.57
-C.1C
-C.26
l.CC
sc
-C.C1
-c.c:
0.13
-c.c:
l.CC
st OH* si OH
0.95
-C.C7
-C.15
-0.57
-C.C1
l.CC
-^.-."'C
-0.56
-C.1C
-C.25
1.00
-c.c:
-0.57
l.CC
-SC.-. -90
-C.C1
-c.c:
C.13
-c.c:
1,00
-c.c:
-c.c:
l.CC
stOH.-.T5C
0.9S
-C.C4
022
-C.43
-C.C1
O.SO
-c.4:
-c.c:
l.CC
stOH.-.T90
1,00
-C.C4
-C.13
-0,57
C.C6
0,95
-0,56
C.C6
J.9S
l.CC
stOHf.Arcm
O.B4
C.4C
-c.::
-C.5C
-c.c:
0.7B
-o,;o
-c.c:
0,B^
O.BJ
1.00
stOH^RI1?
0.96
-C.C3
C.C3
-0.63
C.C4
0.91
-0.61
C.C4
0.9i
0.96
0,30
L.CC
26
-------
Figure 1. Values of T50 vs. Ethanol for design points in the full fuel-parameter matrix.
Bhind
Figure 2. Values of Aromatics vs. Ethanol for design points in the full parameter-matrix.
27
-------
Figure 3. Values of RVP vs. Ethanol for design points in the full parameter-matrix.
Since it does not fit the goals of the study to resolve collinearities by removing terms from the
models, it was necessary to find an additional means to neutralize the correlations.
2.3.1 Standardizatio n of Fuel Parameters
As a preliminary step to modeling, an analytic step to neutralize remaining correlations in the
fuel-parameter matrix is to center and scale the fuel parameters, a process known as
"standardization." Standardization is commonly used for this purpose in analysis of
experiments8. Standardization simply involves first "centering" the measured fuel properties by
subtracting their means, and then "scaling" by then dividing the centered values by their
respective standard deviations, as shown in Equation 3, with statistics for the fuel properties
presented in Table 7. The result is a "Z score," representing a "standard normal distribution"
with a mean of 0.0 and a standard deviation of 1.0.
Z, =
X: X
Equation 3
28
-------
2.3.1.1 One-Stage Standardization
In "one-stage" standardization, the linear effects terms are centered and scaled, as shown above
in Equation 3. Interaction terms are constructed directly from the linear effects and then
standardized. Using ethanol as an example, the standardization of the linear-effect term was
performed as shown in Equation 4.
_ "^etOH "^etOH
~ Equation 4
Using the etOHx Aromatics interaction term as an example, the standardized values were
constructed as shown in Equation 5, where xetoHxArom and SetoHxArom are the mean and standard
deviation of Xeton-^Arom, respectively.
<7 _ XetOHXAmm ~ ^etOHxArom . _
ZetOHxArom ~ Equation 5
etOHxArom
After performing the "one-stage" standardization for linear and interaction terms, we reevaluated
correlations among the standardized fuel-parameter matrix (see Table 8). Results show that this
step did not neutralize strong correlations, particularly between linear effects and interactions, or
among interactions. In fact, the pattern of correlations is identical to that in the unstandardized
matrix (Table 6).
2.3.1.2 Two-stage standardization
As one-stage standardization did not neutralize correlations among model terms, we applied a
second stage of standardization to the 2n order terms9 . This step was conducted by constructing
2n order terms, not from the measurements themselves, but rather from the standardized values
for the linear effects. Using the etOHxetOH term as an example, the two-stage standardized
value, denoted by ZZetoH2 , was calculated as
ZZetoH2 = -^ ^L Equation 6
where the square of the standardized etOH term (Z2etoH) is centered by subtracting its mean m
and standard deviation s (Equation 6). Similarly, the two-stage value for the etOHx Aromatics
interaction, ZZetoHxArom was calculated as shown in Equation 1.
29
-------
ZZ
-7H7
etOHxArom
Equation 7
Following calculation of the two-stage values, we evaluated correlations in the standardized fuel-
parameter matrix when one-stage standardized values (Z) were used for linear-effect terms and
two-stage standardized values (ZZ) were used for quadratic and interaction terms. Table 9 shows
that the combination of one- and two-stage standardization neutralizes the remaining
correlations, with the exception of remaining correlation between the linear effects for ethanol
and T50, as previously described. Several moderate correlations remain among 2n order terms,
but no correlations with R > 0.70 remain.
Table 7. Means and Standard deviations for Fuel Properties, based on Fuel Matrices for the Full and
Reduced Designs.
Model Term
Ethanol (%)
Aromatics (%)
RVP (psi)
T50 (°F)
T90 (°F)
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Arom x RVP
Arom x T50
Arom x T90
T90 x T90
T50 x T90
RVP x T90
Full Design1
Mean
10.3137
25.6296
8.5178
190.611
320.533
0.962963
0.962963
-0.03674
-0.0992352
-0.541342
0.0163277
0.043792
-0.068030
-0.0062526
0.962963
-0.036304
0.126761
Standard
deviation
7.87956
10.0154
1.61137
28.5791
19.4801
0.802769
0.739766
0.978461
0.999615
0.769153
0.972825
0.984096
0.991737
0.983536
0.346951
0.960011
0.972829
Reduced Design2
Mean Standard
Deviation
11.0182 8.05925
24.3909 9.92426
197.000 23.4536
323.527 19.6015
1 Applies to models fit with data for 15 vehicles measured on 27 fuels.
2 Applies to models fit with data for 5 or 15 vehicles measured on 1 1 fuels. See Chapter 9.
30
-------
Table 8. Correlation matrix for one-stage standardized linear-effect and interaction terms in the
optimal-design fuel-parameter matrix.
Ethanol
Arom
RVP
7;c
is c
stOH.-.etOH
75C;-.T5C
75C.-.7SC
5tOH:-:75C
5tOH:-:7SC
it OH:-: Aram
5tOH.-.RV?
Ethancl
1.00
Arem
-C.C4
l.CC
Rrv7>
-C.15
C.C5
l.CC
T5C
-0.57
-0.10
-C.26
1 CO
79C
-C.CI
-c.c:
0.13
-c.c:
L.CC
it OH:-. =1 OH
0.9s
-C.C7
-0.15
-O.S7
-c.c:
1.00
"^O.-.T^C
-0.56
-0.10
-0.25
1.00
-c.c:
-&S7
l.CC
-SC.-..9C
-0.01
-c.c:
0.13
-C C"*
l.CC
-C C^
-C.c:
l.CC
siOH.-.75C
0.9S
-0.04
-c.::
-0.13
-0.01
0.90
-0.4:
-c.c:
l.CC
itCK.-.79c
1,00
-0.04
-C.13
-0, = T
C C6
0,9;
-11, = 6
C.C6
[1,93
,^,-v
itOH.-.Arcra
0,34
C.4C
-c.i:
-0,50
_!-.!-.'
0,73
-n,;n
.r r "
o,s:
0,33
l.CC
itCH.-.RV?
0,96
-C.C3
0.03
-0,63
C.C4
0,91
-n.e:
0.04
0,9:
0.96
0,80
l.CC
Table 9. Correlation matrix for Standardized linear-effect (one-stage) and interaction (two-stage)
terms in the full-design fuel-parameter matrix.
Ethanol
Arom
RV?
T5C
T3C
5tOH:-:5tOH
75C:-.75C
TSC.-.TSC
-tOH:-:T5C
etOHxT9C
»tOH».Afo«i
atOHx.R\-?
Ethanol
l.OC
Arcm
-0.04
l.CC
R'v-?
-0.15
C.C5
l.CC
T5C
-0.57
-0.10
-c.:s
l.OC
7SC
-0.01
-c.c:
C.13
-c.c:
l.CC
5tOH:-.StOH
-: -.-
-C.ll
-C.CI
-0.03
-C.C3
l.OC
"Ci-.T'C
-0.01
-C.1C
C.16
c.i:
C.C5
-0.12
l.CC
7SC.-.79C
-C.13
-c.:3
Off)
0.26
-c.5:
-C.C7
-C.1S
l.CC
stOH.-~;C
.;.:;
0,15
-0.06
-0.01
-0.03
-0.6B
-C 44
C.1S
1.00
5tOH:-.79C
-c.c:
r. < -i
C.C9
-c.c:
-0.07
-c.c:
-00"^
-c.i:
C.C4
l.CC
at OH.-; Arc-in
-;.;f
C.C3
-c.c:
c.i:
c.c:
-0.09
C.C6
-C.31
-C.CI
C.CI
i.CC
5tOHxR\l>
-;.;:
-c.c:
c.c:
-0.05
C.C3
-C.C7
C.C5
C.C6
-0.04
0.14
-0.05
l.CC
-------
3 Dataset Construction
3.1 Selection of Aggregate (Bag) vs. Continuous Data
Both dilute-bag and raw continuous (second-by-second) emission results were generated for
THC, CH4, CO, NOX and CC>2 for all tests in this program. The dilute-bag method has been
optimized over decades of use and is considered the "gold standard" for light-duty vehicle
emission measurements. In this constant-volume sampling system, the vehicle exhaust is mixed
with a large amount of filtered dilution air, and a small portion of this stream is continuously
withdrawn to fill a sealed bag over the course of a test cycle. The total flowrate of exhaust plus
dilution air is held constant by a critical flow venturi, and the bag fill rate is held constant by a
pump and flowmeter, such that the concentration of emissions in the bag is a time-weighted
average for the cycle. At the end of the cycle, the contents of the bag are flowed through an
analyzer to determine the pollutant concentration. The primary advantage of this method is that
it relies on well-understood physical phenomena for controlling flowrates and dilution of exhaust
gases during collection of the sample, which obviates the need for continuous monitoring and/or
adjustment of these parameters. Its primary disadvantage is that the overall dilution ratio of
background air to exhaust must be fixed for an entire test, and is set relatively high to avoid
condensation of water vapor within the system during periods of high exhaust flow. Emission
rates of species of interest continue to decline with more stringent regulations and better control
technology, however, the emission rate of water vapor, and thus sampling dilution ratio, is a
function of fuel economy, which hasn't changed much over time. As a result, for very low-
emitting vehicles the concentration of pollutants of interest that must be quantified in the dilute
bag may be indistinguishable from the level in the background (dilution) air, given the
limitations of the instruments and test procedures. When assessing the effects of fuel properties
on very low emissions concentrations, the challenge is even greater.
In the raw continuous system, a small portion of the vehicle exhaust is continuously withdrawn
and flowed through an analyzer, which generates a continuously-varying concentration
measurement with a short lag that is a function of the system's internal dimensions and sample
flowrate. This method does not perform any dilution, alleviating the issue of needing to quantify
very low concentrations (and there is no background measurement to subtract). However, it does
require continuous (>10 Hz) measurement of exhaust flow rate and precise coordination of the
time series with the emission concentration in order to produce an accurate time-integrated
emission result for a test cycle. These latter processes introduce varying amounts of error into an
integrated continuous measurement. While there is very good agreement in general, it is not
uncommon for the overall result to vary from the dilute bag measurement made on the same test
especially at low emission levels.
Plots and regressions run between the dilute bag and integrated continuous datasets generally
showed very good agreement for the vast majority of results. Plotting the lowest 10% of data
sorted by magnitude, however, showed notable differences in trend as measurements approached
32
-------
zero. The integrated continuous results were suggestive of a "floor" while the dilute bag results
maintained a more linear trend toward zero (with increasing scatter) until censoring occurred at a
very low level. Figure 4 and Figure 5 depict the behavior of NOX in Bags 1 and 2, respectively.
Figure 4. Bag 1 NOX: Integrated-Continuous vs. Dilute-Bag Measurements.
D
£
oi
0.01
Ol
o
0.001
0.0001
Bog 1 A/Ox results, integrated modal vs. dilute bag
0.0001
» * *»:
0.001
0.01
log dilute bag (g/mi)
0.1
33
-------
Figure 5. Bag 2 NO*: Integrated-Continuous vs. Dilute-Bag Measurements.
Bog 2 A/Ox results, integrated modal vs. dilute bag
E
u>
15
D
D
&
Ol
o
0.0001
0.01
0.001
0.0001
0.001
0.01
log dilute bag (g/mi)
0.1
As discussed further in Section 5, most datasets generated in this program contain results
quantified as zero, due to the sample concentration being either below the detection limit of the
analyzer, or quantified as being less than or equal to the measured background concentration.
Zero values resulting from the first case represent low-end censoring, a statistical term meaning
that there are one or more nonzero levels below which no emissions are captured due to
limitations in the measurement process. If there are many censored values, they can introduce a
bias into the analysis because they are unlikely to be truly zero; it is known that some emissions
were produced at some point during the test cycle, but they were simply too small to measure
with the given method. In the second case, where the sample concentration appears to be less
than or equal to the background, it is likely in most cases due to the sample concentration being
indistinguishable from the background due to measurement variability or error (see Section 6 for
further discussion of measurement error).1 This situation is evidenced by looking at the time
1 This assessment of the situation neglects the possibility that the vehicle actually consumes or destroys a
given pollutant species during parts of the test cycle, resulting in periods of "negative emissions", such
that the average emission level over a test is truly zero. While such situations may occur over a limited
period for some emissions in a highly polluted environment, e.g., PM or NMHC in congested traffic, it is
highly unlikely in an emission test cell.
34
-------
series for raw continuous measurements, which show emissions being produced at various points
during all tests, even those having zero measurements as the net dilute bag result.
Given this fact, it is possible to use an integrated continuous result as an estimate to replace a
zero bag result. The decision to do so would presume the continuous measurement has lower
relative measurement error than the dilute bag measurement at very low emission levels;
however, a nonzero result doesn't necessarily ensure that this condition holds. All measurement
methods have error associated with them, and given the history and widespread use of the dilute
bag method, its error levels and limitations are relatively well understood. Conversely, there is
less history with continuous measurement methods to give confidence that those results are an
improvement over dilute bag measurements (even when results were zero). For these reasons,
the decision was made not to replace missing in the dilute bag dataset with integrated continuous
measurements. Rather, these measurements were treated as "censored." Modeling approaches
adopted to address the presence of "censoring" are described in 5.3 (page 96).
3.2 Imputation of Speciated Hydrocarbons (NMOG, NMHC)
Due to the speciation schedule described in Section 2.2, most tests in the dataset do not have
alcohol and carbonyl measurements for bags 2-3 .J As NMOG and NMHC are calculated
emission results that use speciation data, they could not be computed for the portions of the
dataset without speciation. A conservative approach to modeling these emissions might only use
the small subset with speciation (71 tests). As a result, models fit for NMOG and NMHC would
not have the advantage of the full optimized study design, as do the models for the other
emissions (THC, NO*, etc.). This outcome would impose a severe limitation, and is
unsatisfactory, considering the importance of NMOG and NMHC.
However, it is possible to compensate for the limited level of speciation by drawing on an
alternate measure of hydrocarbon emissions obtained from the flame ionization detector (FID).
The alternate measure "NMHC as measured by FID" (NMHCpio), was collected for the entire
dataset, and is very tightly correlated with both NMOG and "true" NMHC. It is thus possible to
estimate NMOG and NMHC results for tests without speciation by using correlations generated
from those with speciation. This technique essentially estimates the offset between the response
of the FID and the fully characterized emission stream, due to the incomplete measurement of
oxygenates by the FID. For NMOG, this estimated value is typically between 2-20% higher than
the NMHCpio measurement, depending on emission bag and fuel ethanol level.
To accomplish this step, we investigated the relationship between NMOG and NMHC with
NMHCpio, by bag. Based on strong correlations between these species, we developed statistical
1 Additionally, a small number of tests (thirty), which had originally been voided for having incomplete
speciation data for bag 1, were included in the main dataset as "salvaged" or "makeup" tests because they
could provide valid results for other emissions such as NOx and CO. It was thus useful to impute NMOG
and NMHC results for bag 1 from these tests as well.
35
-------
models to impute NMOG and NMHC from corresponding NMHCpio measurements. The results
of these analyses are presented below. The results of the current project are very similar to
those obtained in work performed at Oak Ridge National Laboratory that related NMOG to
NMHC for selected ethanol blends on the FTP cycle10.
In application of the models described in 3.2.1 and 3.2.2 were applied deterministically, i.e, the
imputed values represent means at different levels of the predictors, rather than individual
measurements. In taking this step, the random error, or "scatter" around the predicted means was
neglected. Due to the very tight model fits, we expect that the degree of associated error in
imputation is small, and did not substantially affect the results.
3.2.1 Imputation of NMOG
Scatterplots of NMOG vs. NMHCpio for Bag 1 are shown in Figure 6. A tight linear relationship
is apparent between the two sets of measurements. Moreover, distinct trends are visible for each
ethanol level, from lowest (EO) to highest (E20), which are even more conspicuous on the
logarithmic plot. On this basis, we fit linear models for NMOG and NMHC in terms of
NMHCpio. The models were fit as least-squares regressions, with the ethanol levels set as
indicator, or "dummy" variables.
Model results for Bag 1 NMOG are shown in Table 10. The intercept for the reference category
(EO) is not significantly different from 0.0, suggesting that NMOG is 0 when NMHCpiD is 0.
However, the intercepts for the E10, E15 and E20 levels are positive and significantly different
from 0.0, suggesting that at higher ethanol levels, some fraction of NMOG exists independently
of NMHCpio. This outcome is most likely a result of a combination of the presence of
undetected formaldehyde (no FID response) and the increasing levels of unaccounted FID
response due to ethanol and acetaldehyde emissions (which are partially detected by the FID)
with increasing ethanol blends. The slope term for EO indicates that NMOG is numerically very
close to NMHCpio, or approximately 0.9% higher. At ethanol levels of 10% or higher, the
differences in the slope terms increase relative to EO, with the exception that the slope increment
for E15 is slightly lower than that for E10. However, this difference is small, and not statistically
significant at the 95% confidence level (two-tailed/? = 0.92). All terms in this model were
retained, because they gave an increase in overall fit, along with a more symmetric distribution
of residuals around 0.0, than an alternative model with only intercepts for each ethanol level, but
uniform slopes.
k The individual data files produced by SwRI do not contain any imputed values; both NMOG and
NMHC values reported there for Bags 2-3 were simply set equal to NMHCFiD when there were no
speciation data. The summary database file produced by EPA contains a separate sheet with estimated
NMOG results, showing imputed values inserted where they could not be rigorously computed. This
sheet also contains another emission item called NM_FIDHC (equal to NMHCFiD) computed by EPA for
each test during post-processing of the data. This value served as the input to the models shown in
Equation 8 through Equation 11, and can also be used to produce plots or other analyses of the
correlations and modeling.
36
-------
Figure 7 shows scatterplots of NMOG vs. NMHCpio for Bag 2. The plot shows a tight linear
trend, but no apparent sub-trends for ethanol levels. Model results for Bag 2 NMOG are shown
in Table 11. Unlike Bag 1, the reference intercept is statistically different from 0.0, although
very small. The model structure is simpler than that for Bag 1, in that intercepts were fit for each
ethanol level, but a single slope was used for all ethanol levels. A more complex structure with
individual slopes by ethanol level was fit, but the additional complexity did not improve the fit,
nor were the additional slope effects significant. The slope term is again positive and highly
significant, but very small (NMOG is 0.3 1% higher than NMHCpio)- In fitting this model, a
single influential and outlying observation was deleted (studentized-deleted residual > 5.0).
Scatterplots of NMOG vs. NMHCpio for Bag 3 are presented in Figure 8. The plot shows a tight
linear trend, but no apparent subtrends for ethanol levels. The picture is very similar to that for
Bag 2, except that several points fall well off the main trend. Model results for Bag 3 are shown
in Table 12. The model structure is identical to that in Bag 2, with intercepts by ethanol level
with a uniform slope. The reference intercept for EO is not significantly different from 0.0, the
intercept for E10 is marginally significant, and that for E20 is significant, while that for El 5 is
insignificant. The separate intercepts were retained to retain the significant effect for E20. One
outlier with a large studentized-deleted residual was removed before fitting this model.
The model for Bag 1 NMOG was applied using Equation 8
= A + Ao *W + As *15 + Ao *20 + (ft) + YlO *10 + Yl5 *15 + ^20 *20 )*NMHC Equation 8
where XNMHC is a measured value of NMHCpio, >^JMOG is the predicted mean NMOG level for a
given measurement of XNMHC, $) is the reference intercept for EO fuels, /'io, /'is and /2o are
indicator or dummy variables for E10, E15 and E20 fuels, (e.g., /'io = 1 where ethanol level =
10%, 0 elsewhere), /?io, fiis and ^20 are offsets to the reference intercept for E10, E15 and E20
fuels, YO is the reference intercept for EO fuels, and yio, yis and 720 are offsets to the reference
slope for E10, El 5 and E20 fuels, respectively. Thus, for EO fuels, where /'io, /'is and /2o all equal
0, Equation 8 reduces to Equation 9.
>WoG = A + 7o*NMHc Equation 9
Similarly, for E10 fuels, where /'io = 1 and /'is and /2o both = 0, Equation 8 reduces to
= A + Ao + (YO + Yw KMHC Equation 10
For the bag 2 and 3 models, the counterpart to Equation 8 is Equation 11, which simplifies for
blends other than EO similarly to Equation 8, except that the slope term is always as in Equation
9.
= A + Ao *io + As iis + Ao ho + ^NMHC Equation 11
37
-------
During examination of measurements below 0.001 g/mi in scatterplots and regression fits, it was
noted that there was higher variability of NMOG measurements at very low emission levels.
This behavior seemed to be especially prominent below 0.0001 g/mi NMHCpio, and is likely due
to the fact that the FID and wet-chemistry methods have different sources and magnitudes of
measurement variability, an issue not resolvable through statistical methods. Thus, a decision
was made to substitute a zero result for NMOG in cases where NMHCpio values were <0.0001
g/mi, rather than perform the imputation. This outcome occurred in 44 bag-2 tests and 119 bag-3
tests.
Additional detailed information on these analyses is presented in Appendices B.I - B.3.
38
-------
Figure 6. Scatterplots of NMOG and In(NMOG) vs NMHCFiD and ln(NMHCFiD), for BAG 1, by ethanol level,
showing results for five Vehicles with Speciated Measurements.
i*
1C
m
13
12
tl
10
, oa
' OB
0.7
08
03
OS
0.1
'.
0.0 0.1 03 OS O* OS O* Q.T DA OB -Q V '.1 \3 V 15
-------
Figure 7. Scatterplots of NMOG and In(NMOG) vs NMHCFiD and ln(NMHCFiD), for BAG 2, by ethanol level,
showing results for five Vehicles with Speciated Measurements.
CUMf
0.0*}
otm
0030
MM
(MBO
naa
sr «*»
0.023
aon
iino
o.oc
am
0000
0,004
O.W2
0.000
01X0
53D& 0.0 T?
".OJ1? J O3t-- C 0^0
NVHC[RD]
- «!
-S
-s
-3
40
-------
Figure 8. Scatterplots of NMOG and In(NMOG) vs NMHCFiD and ln(NMHCFiD), for BAG 3, by ethanol level,
showing results for five Vehicles with Speciated Measurements.
o.w
0.09
0.08'
0.07
o.oe
0.05-
0.04
0.03-
002
0.01
0.00
0.00 0.01 0.02 CSX3 OM 0.05 O.OB 0.07 0.08 009 0.10 0.11
NMHC[FID] (Bag 3)
-7
-11
8 8 -7 -8 5 -4 3 -2
In(NMHC)
If«v> ODO o D n o jo i. i. i 15 * m * 20
41
-------
Table 10. Model Coefficients, Tests of Effect, and Goodness-of-fit Parameters for BAG 1 NMOG
(dependent variable = NMOG (g/mi), independent variable = NMHCFm (g/mi) ).
Parameter
Intercept (EO)
Slope (EO)
Intercept Increment (E10)
Slope Increment (E10)
Intercept Increment (El 5)
Slope Increment (E15)
Intercept Increment (E20)
Slope Increment (E20)
Estimate
0.000438657
0.009240268
0.002882016
0.030289819
0.010042668
0.030289819
0.010298206
0.048048225
Standard Error
0.00077499
0.00166026
0.00107999
0.00236797
0.00167492
0.00336111
0.00120701
0.00285922
lvalue
0.57
608
2.67
13.0
6.00
9.01
8.53
16.8
Pr>|r|
0.5715
0.0001
0.0078
O.0001
O.0001
O.0001
O.0001
0.0001
Fit Information:
d.f.[model] = 7, d.f. [error] = 911, d.f.[total] = 918.
F value =157,3 16, Pr>F = O.0001.
R2 = 0.999173, Root MSB = 0.005771.
Table 11. Model Coefficients, Tests of Effect, and Goodness-of-fit Parameters for BAG 2 NMOG
(dependent variable = NMOG (g/mi), independent variable = NMHCpm (g/mi) ).
Parameter
Intercept (EO)
Slope
Intercept Increment (E10)
Intercept Increment (El 5)
Intercept Increment (E20)
Estimate
0.000156975
1.003147342
0.000057909
0.000123063
0.000087647
Standard Error
0.00002878
0.00111376
0.00003388
0.00003767
0.00003699
/-value
5.45
900.68
1.71
3.27
2.37
Pr>|r|
0.0001
0.0001
0.092
0.0018
0.021
Fit Information:
d.f. [model] = 4, d.f. [error] = 64, d.f. [total] = 68.
F value = 204 ,200, Pr>F = O.0001.
R2 = 0.999922, Root MSE = 0.000107.
Table 12. Model Coefficients, Tests of Effect, and Goodness-of-fit Parameters for BAG 3 NMOG
(dependent variable = NMOG (g/mi), independent variable = NMHCpm (g/mi) ).
Parameter
Intercept (EO)
Slope
Intercept Increment (E10)
Intercept Increment (El 5)
Intercept Increment (E20)
Estimate
0.000169016
1.002069357
0.000153740
0.00285066
0.000492911
Standard Error
0.00013708
0.00290768
0.0016850
0.00018829
0.00018850
lvalue
1.23
345
0.91
1.51
2.61
Pr>|r|
0.22
O.0001
0.37
0.14
0.011
Fit Information:
d.f. [model] = 4, d.f. [error] = 64, d.f. [total] = 68.
F value = 30,136, Pr>F= 0.0001.
R2 = 0.999469, Root MSE = 0.000534.
42
-------
3.2.2 Imputation of NMHC
As with NMOG, we first look to the plots of the data to investigate the feasibility of imputation
and inform the process of model development. Inspecting the plots for Bag 1 (Figure 9), we see a
pattern similar to that for NMOG, but in reverse. In the case of NMHC, distinct trends are
visible for each ethanol level, but trends for higher ethanol levels are situated lower than for
lower ethanol levels. In the top plot (linear scale) we see that the slopes also differ by ethanol
level, with the sub-trends arrayed in a fan. In the logarithmic plot, however, the slopes appear
parallel.
Following the visual review, we fit regression models, as described above for NMOG. Model
fitting results for Bag 1 NMHC are shown in Table 13. As Figure 9 suggests, the best fit model
includes distinct intercepts and slopes by ethanol level. The main intercept (representing 0%
ethanol) is not significantly different from zero, and the main slope (again representing 0%
ethanol) is very close to 1.0, although significantly different, displaying the very strong
correlation between NMHC and NMHCpio The intercept and slope increments for 10%, 15%
and 20% ethanol levels are all negative, as suggested in the plot, and increase in size with
increasing ethanol, also as suggested in the plot. All these terms are significant, but with the
degree of significance increasing with ethanol level. A likely explanation is that the more
ethanol there is in the fuel, the more of the exhaust stream is comprised of oxygenated species.
This means that the FID measurement of HCs deviates further and further from the true mass of
emissions as determined by chemical speciation. As this difference increases, it is easier for the
statistical tests to resolve in the midst of measurement variability present. In developing the best
fit model, seven outlying measurements with high studentized-deleted residuals were removed (|
r.i | > 3.5). The model shown in Table 13 was applied using Equation 8. However, for Bag 1 it
was only necessary to impute a relatively small number of NMHC measurements (< 50). For the
vast majority of cases, actual measurements were available.
In the scatterplot for Bag-2 results (Figure 10), it is more difficult to visually assess the form of
the best fitting model. Viewing the linear-scale plot (top), the measurements appear to sit on a
single trend, very close to but not coinciding with a one-to-one trend. The set of available data is
smaller (five vehicles), and a distinction of trends by ethanol level is not obvious. The same is
true of the logarithmic plot (bottom), although is it apparent that the degree of relative scatter
around the trend is greater at the low end than at the high end.
The model structure fit for Bag 2 is identical to that for Bag-2 NMOG; distinct intercepts for
each ethanol level, but with a uniform slope across all levels (Table 14). The main intercept is
negative and very small, but it is statistically significant, suggesting the possibility of a small but
real offset between running NMHC and NMHCpio for the EO blends measured. Based on these
results and others, it appears there is a certain level of production of oxygenated species
(primarily formaldehyde, around a few percent by mass), which is a baseline resulting from
43
-------
combustion of any gasoline, regardless of oxygenate content. This appears as an offset for EO,
which then grows as ethanol is added and emissions of ethanol and acetaldehyde increase. The
intercept increments for the ethanol blends are negative and generally increasing (in absolute
value) with ethanol level. One apparent anomaly is that the offset for 15% ethanol is slightly
larger (more negative) than that for 20% ethanol. Both offsets are significantly different from
zero, but almost certainly not significantly different from each other. If we discount the results at
15% ethanol due to the small number of fuels at that level and their combined properties, the
pattern appears consistent, if not necessarily statistically significant. The model for Bag 2 was
applied to impute NMHC measurements using Equation 11.
The visual impression for Bag 3 (Figure 11) is very similar to Bag 2, except that a small number
of measurements sitting well off the main trend are more conspicuous, particularly in the
logarithmic plot. The characteristics of the model fit are generally similar to Bag 2, with two
exceptions (Table 15). A first exception is that the main intercept is not significantly different
from zero, making the Bag-3 model more similar to the Bag-1 model in this respect. The main
slope is again significant and close to 1.0, but not as close as the slope for Bag 2, although closer
than that for Bag 1. The intercept increments are all negative; those for 10 and 15% ethanol are
not significant, although the increment for 20% ethanol is highly significant. It is also an order
of magnitude larger than the increments for the other two ethanol levels, ostensibly reflecting the
greater variability of the measurements for 20% ethanol, even after removing two outlying
measurements. As with Bag 2, the Bag 3 model was applied for imputation using Equation 11.
Additional information on these analyses is presented in Appendices C.1-C.3.
44
-------
Figure 9. Scatterplots of NMHC and In(NMHC) vs NMHCFiD and ln(NMHCFiD), for BAG 1, by ethanol level.
showing results for fifteen Vehicles with Speciated Measurements.
1.6
1,5
1.4
1.3
1.2
1.1
1.0
0.9
0.8
0.7
0,6
0,6
0,4
0,3
0,2
0.1
1556Monfcy MniuvH. 2011 1
0,0
0.0
0.1 0.2 0.3 0.4 OS C.8 0.7 OJB OS 10 1.1 1.2 13 1.4 1.5 1.8
NM_FIDHC (Be« 1)
Eibaaol te*d (X) ODOO ana 10
15 54Ho«la>-, FetxuirvH. 2011 2
-1
oS"
-1
ln{NM_RDHC)
i (%) O 0 O 0 ODD 10 -^.-
45
-------
Figure 10. Scatterplots of NMHC and In(NMHC) vs NMHCFiD and ln(NMHCFiD), for BAG 2, by ethanol
level, showing results for five Vehicles with Speciated Measurements.
0.0*4
0.042
O.CMO
0.038
O.Q3B
0.034
0.032
0.030
0023
0.058
0.024
0.022
o.oao
O.OB
O.OB
O.OVJ
0.012
O.OC
0.008
aooe
0.004
0.002
0.000
IV 56 ModUy, Febrary 14,2011 1
0.003 0.005 OJ010
OJ01S 0.02C OXKS 0.030
NM_RDHC (Bag 2)
0.0% 0.040 0.045
14,2011 2
-5
-7
-7 -6 -5
ln(NM_F10Hq
ErluiMl Ipvd (%} OOOO ODD 10 2 £ £ 15 ***20
-3
46
-------
Figure 11. Scatterplots of NMHC and In(NMHC) vs NMHCFiD and ln(NMHCFiD), for BAG 3, by ethanol
level, showing results for five Vehicles with Speciated Measurements.
0.044
0042
0.04O
0.033
ooae
0034
0.092
0.033
0029
0.02B
0.024
0022
0.033
O.OB
O.OB
0.0t4
0.012
Q.OO
0.008
OOOB
0.004
0.002
ooco
0.000 0006 0X110 0X115 0.020 3.026 0000 0035
NM_RDHC (3ag 2}
Blm»II«m(%) D0° 0 oon 10 a^i B «»« 20
-s
-7
-7 -6 -6
ln(NM_RDHC)
Elharal tevd (S) ODOO ODD 10 .: .- ^ i; *ic*20
-3
47
-------
Table 13. Model Coefficients, Tests of Effect, and Goodness-of-fit Parameters for BAG 1 NMHC
(dependent variable = NMHC (g/mi), independent variable = NMHCFm (g/mi) ).
Parameter
Intercept (EO)
Slope (EO)
Intercept Increment (E10)
Slope Increment (E10)
Intercept Increment (E15)
Slope Increment (El 5)
Intercept Increment (E20)
Slope Increment (E20)
Estimate
-0.0002943088
0.9987517260
-0.0019509760
-0.0209384772
-0.0051528497
-0.0246110233
-0.0064657095
-0.0333647556
Standard Error
0.00055230
0.00118319
0.00078811
0.00175580
0.00118670
0.00236599
0.00085103
0.00199484
lvalue
-0.53
844.12
-2.48
-11.93
-4.34
-10.40
-7.60
-16.73
Pr>|r|
0.59
0.0001
0.014
O.OOOl
O.OOOl
O.OOOl
O.OOOl
0.0001
Fit Information:
d.f.[model] = 7, d.f. [error] = 911, d.f. [total] = 918.
F value = 276,173, Pr>F = O.OOOl.
R2 = 0.999529, Root MSE = 0.004112.
Table 14. Model Coefficients, Tests of Effect, and Goodness-of-fit Parameters for BAG 2 NMHC
(dependent variable = NMHC (g/mi), independent variable = NMHCpm (g/mi) ).
Parameter
Intercept (EO)
Slope
Intercept Increment (E10)
Intercept Increment (El 5)
Intercept Increment (E20)
Estimate
-0.000031742
1.000457255
-0.000026796
-0.000039842
-0.000036134
Standard Error
0.00001232
0.00047680
0.00001450
0.00001613
0.00001584
lvalue
-2.58
2098.27
-1.85
-2.47
-2.28
Pr> t
0.012
0.0001
0.069
0.016
0.026
Fit Information:
d.f. [model] = 4, d.f. [error] = 64, d.f. [total] = 68.
F value =1,108,270, Pr>F= O.OOOl.
R2 = 0.999986, Root MSE = 0.000046.
Table 15. Model Coefficients, Tests of Effect, and Goodness-of-fit Parameters for BAG 3 NMHC
(dependent variable = NMHC (g/mi), independent variable = NMHCpm (g/mi) ).
Parameter
Intercept (EO)
Slope
Intercept Increment (E10)
Intercept Increment (El 5)
Intercept Increment (E20)
Estimate
-0.000051391
1.000231724
-0.000038854
-0.000021166
-0.000292149
Standard Error
0.00006850
0.00144955
0.00008424
0.00009618
0.00009253
lvalue
-0.75
690.03
-0.46
-0.22
-3.16
Pr>|r|
0.46
O.OOOl
0.65
0.83
0.0024
Fit Information:
d.f.[model] = 4, d.f[error] = 64, d.f. [total] = 68.
F value =121,075, Pr>F= O.OOOl.
R2 = 0.999868, Root MSE = 0.000267.
48
-------
4 Data Review
The data collected in this study are difficult to visualize in that they encompass variation of
emissions in the five-dimensional fuel-parameter space. Due to human limitations, it is practical
to view the data in only two dimensions at a time, in some cases including multiple series to
represent levels in a third dimension. Despite the risk of misinterpreting visual portrayals that
may oversimplify the actual emissions behavior in all dimensions, it is valuable to review the
data visually before model development.
At the outset, it is helpful to get an overview of the raw results, sorted by vehicle and fuel, which
gives an initial impression of variability among vehicles and fuels, as well as within vehicles.
This view also gives an initial impression of vehicles or observations that may prove influential.
In addition, we averaged and plotted the data to check for evidence of "main effects," or "linear
effects," i.e., trends in emissions across all levels of a single fuel parameter. We constructed
these views by averaging the data by the levels of one fuel parameter and by vehicle, across all
levels of the remaining four parameters, repeating the process for each fuel parameter in turn.
We took this step for the emission results themselves (i.e., in "linear space"), as well as for
natural-log transforms of the data (i.e., "log space"). We made a point of examining the log-
transformed results, as the statistical models were developed using the transforms, rather than the
raw results.
The study design anticipates the possibility that the response of emissions to changes in multiple
fuel parameters may involve several 2-way interactions, which suggests that limiting our
examination to "linear effects" may be simplistic. To examine 2-way emissions responses, we
also averaged and plotted the data by two fuel parameters simultaneously to examine potential
"conditional" or interaction effects, or how the effect of each fuel parameter varied with the
levels of the other parameters.
Below, we illustrate these concepts for three sets of results: Bag 1 NOX, Bag 2 NOX, and Bag 1
PM. For these compounds, as for others, the ethanol *T50 interaction gives an example of two
interrelated variables and the importance of supplementing "linear-effects" plots with
"interaction" plots.
The plots presented and described below, as well as additional plots not shown in this document
are presented in Appendices G - J.
4.1 NO* (Bag 1)
Figure 12 shows the set of observations for Bag 1 NOX, with the data portrayed as the common
logarithm of the measurements (base 10). Across all fuels, the range of variability differs by
vehicle. For several vehicles, the range of variability over all fuels spans about one order of
magnitude (e.g., Civic, Corolla, Odyssey). The two cleanest vehicles (Focus, Sienna) are also
49
-------
the most variable, spanning over an order of magnitude. Variability for the remaining vehicles
spans half an order of magnitude or less.
4.1.1 Linear Effects
A linear-effects plot for ethanol is shown in Figure 13, which suggests that an ethanol effect is
visible when the data are averaged across the other four fuel properties. Trends for individual
vehicles show a general increase in NO* with increasing ethanol, with some exceptions. For
example, the Altima and Odyssey show generally declining trends, and several vehicles are
lower at 20% ethanol than at 10%, including the Impala, the Outlook and Caliber. The view of
ln(NOx ) is similar, except that it may be suggestive of a shift in trends by vehicle, with vehicles
having lower ln(NOx ) showing stronger ethanol increases than those with higher levels.
In a similar plot for aromatics (Figure 14), an increase in NO* with increasing aromatics is
clearly evident for all vehicles. These results display the common "fan" effect, with trends for
vehicles with higher emissions generally steeper than for vehicles with lower emissions. This
pattern is consistent with the plot for ln(NOx), which shows generally uniform trends among
vehicles.
When viewing data averaged by RVP and vehicle (Figure 15), the results suggest no net effect,
although individual vehicles show gentle positive or negative trends. This view leaves the
impression that RVP may not prove to be an important predictor for Bag 1 NO*, although it is
possible that RVP effects may be masked by other fuel effects.
The linear-effects plot for T50 (Figure 16) is not clearly suggestive of an effect, whether positive
or negative. Some patterns within the data raise questions. Some vehicles show a general
positive trend, others a general negative trend. A curious feature is that the NO* level for some
vehicles is higher at 165 than at 150 or 190. When viewed as a whole, the view for ln(NOx)
seems suggestive of a slight negative effect. However, this conclusion would be spurious. The
observed pattern stems from the relationship between ethanol and T50, both in fact as well as in
the fuel-parameter matrix (See Figure 1). What the linear-effects view obscures is that the data
points with lower T50 have higher ethanol and vice versa, with some overlap in the center of the
range. Thus, what purports to give a picture of the linear effect of T50 is confounded by the
somewhat stronger effect of ethanol. Thus, the case of T50 is an example of case where the
linear effects view is misleading.
The corresponding view for T90 (Figure 17) shows mixed results, possibly suggesting positive
trends for vehicles with lower NO* and vice versa. A conspicuous feature is that NOX at the 325°
level tends to be higher than at 340°. A superficial conclusion could be that a quadratic fit for
T90 might be appropriate. However, it is important to remember that the points at 325° may not
deserve as much weight as those at the other two levels, as only two fuels were assigned T90
levels of 325°. The other properties of these fuels may also be important. One had ethanol at
10% and the other at 20%. T50 for both fuels was correspondingly low (<170°). However, both
50
-------
had aromatics at the higher level (35%), which may best account for the apparent curvature in
the trend.
4.1.2 Interactions
It is necessary to go a step further, and look at "interaction" or "conditional effects" plots,
starting with the interaction of ethanol and T50, which deserves special attention due to the
remaining correlation between these two properties, which cannot be neutralized by the fuel-
matrix design. To construct the plot, we average the data by levels of etOH and T50, across the
levels of the other properties, as well as across vehicles. Note that in taking this step, we average
by the target fuel properties for each fuel, not the actual measured properties, which have some
variability. In plotting the set of means, we can construct two views. We can plot the averages
vs. ethanol, with a separate series for each T50 level (etOHxTSO), and we can plot averages vs.
T50, with a separate series for each ethanol level (TSOxetOH).
In the plot for etOHxTSO (Figure 18), the view seems to indicate an upward trend from EO
through E20, but with some downward curvature above E10. At first glance, the picture seems
to suggest the existence of negative quadratic trend for ethanol. However, in interpreting this
plot, we may need to discount what we think we see. Specifically, we need to consider that not
all points on the plot represent equal numbers of fuels, or are equally balanced in terms of the
remaining three fuel properties, for which reason it is not certain that all points make equal
contributions to the overall trend. For example, the leftmost point in the green trend (etOH=15%,
T50=165°), represents a single fuel, which also has the higher aromatics level (35%). Similarly,
the rightmost point on the red trend (etOH=10%, T50=190°), represents two fuels with one point
having high aromatics and low T90, and the other low aromatics and high T90. Taking these
factors into account, recognizing the probable positive effects of aromatics and T90 on NOX,
leads us to the tentative conclusion that it is possible, but not certain, that models may fit a
quadratic trend to these data.
Plotting the data averaged by T50, with series for different ethanol levels (TSOxetOH), gives
another view (Figure 19). At first glance, the trend appears to "zig-zag" from low to high. As
with the previous view, however, several points creating this apparent visual effect probably do
not make contributions to the main trend as strong as other sets of points that are not as far off
the main trend. For example, the left-most point on the red trend (T50=165°, etOH=15%),
represents a single fuel with high aromatics, and appears conspicuous as in the previous view
(leftmost green). The center point in the green trend (T50=190, etOH=10%) represents the same
two fuels as the rightmost red point in the previous plot. The center point in the black trend also
remains conspicuous in this view (T50=220, etOH=0%). This point represents two fuels (8,13)
which are, in concept, balanced with respect to aromatics and T90, in that fuel 8 has low
aromatics and T90, and fuel 13 has high aromatics and T90. Accounting for these points leads to
tentative conclusion that the effect for T50 may be positive, but not large.
51
-------
The interaction between ethanol and aromatics is more straightforward. First, the fuel-parameter
designs had better success in balancing the numbers of fuels with respect to these two
parameters. Averaging four ethanol levels and two aromatics levels gives eight means. In the
etOHx aromatics plot (Figure 20), the eight means are arranged in two series of four points. The
impression is that the trend for the lower aromatics level (black) has a somewhat steeper slope
than the trend for the higher aromatics level. In the aromaticsxetOH view (Figure 21) the eight
means give four series with 2 points in each. In this view, the trend for the lowest ethanol level
appears to have a steeper slope than those for the remaining three ethanol levels, which have
approximately equal slopes. Taken together, these pictures are suggestive of a negative, or
"interference" interaction between ethanol and aromatics.
52
-------
Figure 12. Common logarithm of Bag 1 NOX, by Vehicle and Fuel. The second view highlights
"censored" measurements.
-1
-2
-3
4* «*
?R
P 4%f
,7&,
WO 200
+ ALT1MA
COROLUCE
300 400 500 600 700
Obs. (sorted by vehicle, fuel)
C.AUBEP.
EXPLOREMLT
i ODYSSE\'
^ - C.AJ-1RV * * * CIVIC
FOO » * « FOCUS
;- OUT10OKXE * ' = SttNKA
am soo tx»
I COBALT
-1 MPALALS
SILVEUDO
-2
-3
-4
l*&$fe
.AT°8c^=!4jSf oo*
«Q
s°
a
a'fco
o°8
0 »_«
B
0 o
£Sa£8
0/o
*°. V
.^8
°°%°
0 100 330
SCO 400 500 600 703
Obs., (sorted by vehicle, fueD
900 BOO
cfiBst of c.o eddad to nog. or
53
-------
Figure 13. Linear-effects plot for Ethanol on Bag-1 NOX: NO* and ln(NOx) vs. Ethanol Level, with data
averaged by four Ethanol levels and by Vehicle.
vehicle 'ALTlIWi
- COROLLACE
~-* LIBERTY
vehicle
COROLLACE
LIBERTY
COBALT
IMFALALB
SILVERADO
54
-------
Figure 14. Linear-effects plot for Aromatics on Bag-1 NOX: NO* and ln(NOx) vs. Aromatics Level, with
data averaged by two Aromatics levels and by Vehicle.
0.18
0.17
0.16
0.15
0.14
O.B
0.12
0.11
O.t)
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02-
0.01
0.00
40
vehicle ALTIMA
-*- COROLLACE
~~ LIBERTY
CALIBER
EXPLORERXLT
'ODYSSEY
F150
CMC
FOCUS
SIENNA
'COBALT
IMBMALS
SILS/ERADO
-1-
-4-
-6
40
Aromatics (%)
vehicle *-" ALT1MA
CORdLACE
UBERTY
' CAUBER "« CAMRY
EXPLCSERXLT F150
! OWSSEY - OOTLDOKXE
CMC
FOCUS
SIENNA
-COBALT
MMRMALS
SILVERADO
55
-------
Figure 15. Linear-effects plot for RVP on Bag-1 NOX: NO* and ln(NOx) vs. RVP Level, with data averaged
by two RVP levels and by Vehicle.
0.171
0.16
0.15
0.14
0.13
0.12
0.11
0.10
0.08
O.OB-
0.07
0.06
0.05
0.04
0.03
0.02
O.D1
0.00
10
vehicle
*ALT1WA
- COROLLACE
LIBERTY
CAUBER
EXPLORERXLT
'ODYSSEY
RVPflb)
CAMRY
F150
'OUTLQOKXE
CMC
FOCUS
'OOBKJ
MMFALALS
SILVERADO
-1
-2
-3
-5
-6
vehicle
10
COROLLACE
' LIBERTY
RVP (Ib)
CALIBER «CAMRY CMC
EXPLORERXLT F150 * FOCUS
'ODTSSEY ""OUTLOOKXE SIENr4A
^COBALT
IMFALALS
SILVERADO
56
-------
Figure 16. Linear-effects plot for T50 on Bag-1 NOX: NO* and ln(NOx) vs. T50 Level, with data averaged
by five T50 levels and by Vehicle.
150
vehicle "-'ALTIMA
" COROLLACE
LIBERTY
170
190
'CALIBER
EXPLCRERXLT
5 ODYSSEY
190 200
T50
-'"CAMRY
F150
"^OUTLDOKXE
220
230
24O
CMC
FOCUS
SIENhJA
'COBALT
»IMF*LA1S
SILVERADO
57
-------
Figure 17. Linear-effects plot for T90 on Bag-1 NOX: NO* and ln(NOx) vs. T90 Level, with data averaged
by three T90 levels and by Vehicle.
300
vehicle 'ALT1WA
' COROLLACE
LIBERTY
340
CAUBER
EXPLOHERXLT
'ODYSSEY
COBALT
IMhA! AIH
SILVERftDO
-1
-2
-3
-6
300
3-D
vehicle
-AO1WA
- COROLLACE
-LIBERTY
CALIBER
EXPLORERXLT
ODYSSEY
320
T90(deg,F)
-"CAMPY
F150
OUTLDOKXE
330
340
CMC
FOCUS
SIENNA
'COBALT
MMFALALS
SILVERftDO
58
-------
Figure 18. Interaction plot for EtOHxTSO: Data are averaged by Ethanol and T50 levels, and plotted
vs. Ethanol level, with a separate series for each T50 level. Note that not all points on the plot
represent the same number of fuels.
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0.00
150 (deg.F)
150
10
Ethand (%)
165 »* 190
220
240
-1
-2
-3
-4
TCO
150
10
Ethanol p&)
165 190
20
220
240
59
-------
Figure 19. Interaction plot for TSOxetOH: Data are averaged by T50 and etOH levels, and plotted vs.
T50 level, with a separate series for each ethanol level. Note that not all points on the plot represent
the same number of fuels.
0.06
0.07
0.06
0.06
0.04
0.03
0.02
0.01
0.00
160
160
170
180
190 200
T50
-------
Figure 20. Interaction plot for etOHxAromatics: Data are averaged by etOH and Aromatics levels,
and plotted vs. Ethanol level, with a separate series for each Aromatics level. Note that not all points
on the plot represent the same number of fuels.
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0.00
10
Ethand (%)
Aromatios (%)*" e -
35
-2
-3
-4
10
Bhanol fX>)
Aromatics (K> ~ 15 ~ 35
61
-------
Figure 21. Interaction plot for etOHxAromatics: Data are averaged by etOH and Aromatics levels, and
plotted vs. Ethanol level, with a separate series for each Aromatics level. Note that not all points on
the plot represent the same number of fuels.
0.09
0.08
0.07
0.06
0.05-
0.04
0.03
0.02
0.01
0.00
20
30
Aromatics (%)
0 s~f** 10 B"e** 15
-1
-2
-3
-4
20 30
Aromatics (%)
Ethanol (%} *~~ 0 B"""" t) "^° 15 *"*" 20
62
-------
4.2 NOx(Bag2)
Overall, the patterns observed for NOX in Bag 2 are quite similar to those for Bag 1. The view of
all data, sorted by vehicle and fuel, looks similar to Bag 1, except that, as expected, Bag-2 values
are roughly one order of magnitude lower than values in Bag 1 (Figure 22). Overall, the data in
both Bags span about the same range, 2.5 orders of magnitude when including "low" vehicles,
i.e., Focus and Sienna for Bag 1, and Cobalt for Bag 2, or 1.5 orders of magnitude, excluding the
"low" vehicles. As in Bag 1, variability for most individual vehicles spans one third to half an
order of magnitude. Also as in Bag 1, there are several censored measurements for the vehicle
with the lowest measurements (Cobalt).
4.2.1 Linear Effects
The linear-effects plot for ethanol shows a general increasing trend with increasing ethanol
(Figure 23). An apparent anomaly is that several vehicles show higher NO* at the 15% level
than at 10% or 20% levels, ostensibly due to the high aromatics or high T90 levels for these
fuels. Several vehicles show no apparent trend. Two vehicles, Civic and Odyssey, track very
closely, and show generally steeper trends than the other vehicles. In the plot of ln(NOx), the
general NO* increase is apparent, with three exceptions, the Corolla, the Focus and the Cobalt.
The plot for Bag-2 aromatics is quite similar to its counterpart for Bag 1 (Figure 24). Most
vehicles show an increase with aromatics, but the trends for the vehicles with higher emissions
tend to be steeper. In the logarithmic plot, the increase is apparent, although small, except for
the Cobalt, which shows a noticeable decrease.
As in Bag 1, the trends across the two RVP levels vary by vehicle, with no apparent overall
pattern across vehicles (Figure 25). The effects for T50 and T90 are similar in that the trends
across the five T50 levels and three T90 levels are similar, with no overall trend apparent, for the
same reasons noted above for Bag 1 (Figure 26, Figure 27). With respect to T50 in particular,
we avoid drawing conclusions from the Linear Effects plot, pending review of the interaction
plots.
4.2.2 Interactions
With respect to ethanol and T50, the plot of etOH by T50 (Figure 28) shows a pattern quite
similar to that in Bag 1. Overall the pattern suggests a small positive ethanol effect. The view of
TSQxetOH for Bag 2 is also similar, but not identical to its counterpart for Bag 1 (Figure 29). On
the whole, it is not clear that this plot (particularly the logarithmic view), suggests an evident
effect for T50.
The interaction plots for ethanol and aromatics are suggestive of a negative or "interference"
interaction between these two properties. In the etOHx Arom plot (Figure 30), the slope for the
low aromatics level appears steeper than for the high aromatics level. Similarly, in the Arom x
etOH plot (Figure 31), the slope for the lowest ethanol level appears to be steeper than those for
63
-------
the higher levels. It is not clear, though, that the apparent effect will be considered significant
when models are fit.
64
-------
Figure 22. Common logarithm of Bag-2 NOX, by Vehicle and Fuel. The second view highlights
"censored" measurements.
16,2011 52
100 aoo
+ » + A1HMA 0 E C
^ ^ C010L1ACE
+ * + ^iifTT1^ o o o
40C SCO
70Q SCO DDO
Obs.. (sorted by vehicle, fuel)
nan ooav * * * avc
nse ** * FOCUS
odda
DO 3DO 300 100 500 ECO 700
Ota, (Borted by vehicte, fuel)
000 BOO 1000
I * atptnt tn> 010 i
offelt of OO ncttd » rwg, or ZMO
65
-------
Figure 23. Linear-effects plot for Ethanol on Bag-2 NOX: NO* and ln(NOx) vs. Ethanol Level, with data
averaged by four Ethanol levels and by Vehicle.
CCBULT
IMRHLAL3
SIL^RADO
lante 6,2011 2
0
-1
-2
vehicle --"AU1MA
OCnCLL
UaSTTY
CALIBEB
EXPLOfleFXLT
PBO
OUTIOCKXF
FOCU3
SIEWJA
'IMFHLAL3
SIL^RMXJ
66
-------
Figure 24. Linear-effects plot for Aromatics on Bag-2 NOX: NO* and ln(NOx) vs. Aromatics Level, with
data averaged by two Aromatics levels and by Vehicle.
oca>
oaa*
0022
0050
IXO-B
ooc-
00*
0004
Aramatrcs (
-A01MA
CCBQLLACE
CWJBEFl
EXFLCflERXLT
PEO
* QUTLOQOtE
owe
FOCJS
COBALT
'IMFKLM.3
16,2011 4
0
-1
-4
-5
-e
-7
CCnCLLME
-uaanv
OJBER
EXPtOBEBXLT
CORSET
Artmalics (Kj
"-'CUMFV
PBO
67
-------
Figure 25. Linear-effects plot for RVP on Bag-2 NOX: NO* and ln(NOx) vs. RVP Level, with data averaged
by two RVP levels and by Vehicle.
I6.2D11 5
vahids
*ALJ1MA
COBOLLACE
CALIBER
EXFLOBEJWLT
cer/sser
FVP (b)
"CAMFIV
FtO
OIJT1-OCKXE
CMC
FOCUS
SIENNA
'COBWT
IMFHLALS
SILWRADO
16,2011 i
-1
-6
-7
vahidB
CCBOLLACE
'UBStTY
CALIBER
EXPLOflEBXLT
RWP(lb)
CAMFT/
FBI
* OUTLOCKXE
CMC
FOCU3
SIENNA
CCSALT
'IMPALAL3
SILSERADO
68
-------
Figure 26. Linear-effects plot for T50 on Bag-2 NOX: NO* and ln(NOx) vs. T50 Level, with data averaged
by five T50 levels and by Vehicle.
16,2011 7
ISO (cteg.F)
vehde
'AU1MA
-CCHOLLACE
-LJBBVTY
CA1JBER
EXFLCflSWLT
-OCK-SSEY'
ffO
OUIU3CKXE
OUIC
FOCUS
3IEMJA
"OCB*LT
'IMRALAL3
0
-1
-2
-5
-e
-7
-6
tso
TBD
-"AOIMA
OCROUACE
CMJBCR
EXPIDREFW1T
CCTOSEY
PEO
~* OUTLOCKXE
owe
FOCU3
CC6MT
IMFALAL3
SILVERftDO
69
-------
Figure 27. Linear-effects plot for T90 on Bag-2 NOX: NO* and ln(NOx) vs. T90 Level, with data averaged
by three T90 levels and by Vehicle.
CC6ALT
IMPMLAL3
SILVERADO
30C
"AO1MA
CCBCUACE
-uaanv
CWJKFI
EXFIDHEFKLT
P60
DUILOOKXE
OVIC
FOCU3
'IMFWLALS
SILVEFWDO
70
-------
Figure 28. Bag-2 NOX: Interaction plot for EtOHxTSO. Data are averaged by Ethanol and T50 levels,
and plotted vs. Ethanol level, with a separate series for each T50 level. Note that not all points on the
plot represent the same number of fuels.
O.OB
3.015
SOU
000
0.012
son
DOC
OA09
GOOB
0007
GO06
OOO5-
ooos
acw
DODO
BO *** lEB "***"* BO
-1
Bhand (%)
71
-------
Figure 29. Bag-2 NOX: Interaction plot for TSOxetOH. Data are averaged by T50 and ethanol levels,
and plotted vs.TSO level, with a separate series for each ethanol level. Note that not all points on the
plot represent the same number of fuels.
16,2011 15
I
0,05
0014:
ootJ
0.012:
Ojtm
0.00
0009
DOCK
0007
0.006
0005
OO04
0003
0002
0.001
coco
-tro
Bhand
KO 2QC ^t 220 £30 940
0 * -jo »«« -5 *** 2Q
16,2011 it
BO
Shnnd (*)
BO
TH)
0 **" 10
210
330 MO
72
-------
Figure 30. Bag-2 NOX: Interaction plot for EtOHxAromatics. Data are averaged by Ethanol and
Aromatics levels, and plotted vs. Ethanol level, with a separate series for each Aromatics level. Note
that not all points on the plot represent the same number of fuels.
16,2011 u
0.0 «
O.C-Q:
0012:
0,
OOt):
0000
C.CCB
0007:
0008
0005
0004:
0003:
0002
0001
ooco
t>
BlTBTOl (*)
An; rurtira ffc)
16,2011 12
-4
n
Bhand (%)
Arcmatcs p6) «-f B **» 36
73
-------
Figure 31. Bag-2 NOX: Interaction plot for AromaticsxetOH. Data are averaged by Aromatics and
ethanol levels, and plotted vs. Aromatics level, with a separate series for each ethanol level. Note that
not all points on the plot represent the same number of fuels.
c.c-0
0.012
OJOH
CUOO:
00°*
o«»
0007
ocos
0005
OC04:
0003
ooos
cotn
oooo
Bhnnd (It)
Aromatics pt)
0 *** 10 ***
Arcfnadcs (*)
Bhnnd (K)
74
-------
4.3 Particulate Matter (PM, Bag 1)
As with NO*, we begin with a view of all results, by vehicle and fuel, shown in Figure 32. As
before, this view presents the data as common logarithms, and includes censored values and two
apparent outlying measurements. The variability within vehicles is about 1-1.25 orders of
magnitude. Interestingly, the variability between vehicles is not large. A striking exception to
this pattern is the Liberty, which is considerably higher than the other vehicles. This vehicle also
has a very high, apparently outlying observation. While not as extreme, the Corolla also has a
single measurement that appears high relative to the remaining measurements on the same
vehicle. About a third of the vehicles have measurements that appear quite low in comparison to
the remaining measurements on their respective vehicles. Also, for this set of results, 45 out of
913 measurements are censored, shown in red at the bottom of the figure.
Note that the linear-effect and interaction plots presented below were generated after excluding
the two outlying observations and all censored measurements. Both outliers and censored values
can affect the apparent patterns in the resulting means, and in any case, it is not possible to
reflect censored values in logarithmic plots.
4.3.1 Linear Effects
The Linear Effects plot for ethanol shows some mixed results (Figure 33), but with an apparent
increase from 0% to 10% ethanol, followed by a leveling or decline at higher ethanol levels. It
appears possible that models may fit a quadratic as well as a linear term for ethanol.
The plot for aromatics shows a pronounced aromatics effect with the results for different vehicles
arrayed in a "fan" (Figure 34). The vehicle with highest PM (Liberty) has steeper trend than the
other vehicles in the linear plot (top). In the logarithmic plot, most vehicles have similar slopes
(accounting for some degree of variability). It is interesting to note that the logarithmic trend for
the Liberty is similar to those for the other vehicles, although its emissions are a full order of
magnitude higher. The results lend to some support to the assumption that the effect of
aromatics (and other properties) on PM (and other emissions) can be expressed multiplicatively
and is similar across vehicles, even across widely ranging emission levels.
As with NOX, the linear-effects plot for RVP shows mixed results, with increases for some
vehicles but decreases for others (Figure 35). This plot is suggestive of no overall effect but it is
difficult to draw a conclusion, without having viewed interaction plots.
The Linear Effects plot for T50 also appears suggestive of no overall effect, or perhaps a slight
negative effect (Figure 36). But this impression is almost certainly spurious, for the same
reasons as described for Bag 1 NO*, above. It is necessary to defer conclusions pending review
of interaction plots model results.
75
-------
The Linear Effects plot for T90 is clearly suggestive of an overall positive effect, when
considering all vehicles (Figure 37). An apparent anomaly is that the data points at 325° are
higher than at the other two levels. This outcome can be ostensibly attributed to both fuels at this
T90 level having high aromatics, with one of the fuels having 20% ethanol and the other 10%.
4.3.2 Interactions
The view of etOH x T50 is not clearly suggestive of an interaction (Figure 38), although the
trend for 165° (green) appears anomalous. However, we can probably discount the results at this
temperature. The leftmost green point represents a single fuel, with both high aromatics and
high T90, which may be expected to give elevated PM, as we have seen. The point on the right
represents an additional five fuels, but appears more in line with the main trend. The right-hand
point of blue trend (220°) represents two fuels, balanced in aromatics and T90, but with low
RVP. In the complementary plot of T50 x etOH (Figure 39), the trend for 15% ethanol (red)
looks anomalous. The left-hand point represents the same fuel as the left-hand point in the green
trend in the previous plot. Aside from this one point, the behavior of the trends for 0% and 10%
ethanol (green and black) may suggest a positive quadratic curvature with respect to T50, but
does not obviously suggest an interaction between ethanol and T50.
In contrast, the plots for ethanol and aromatics are suggestive of a positive or "reinforcement"
interaction. In the etOH x arom view (Figure 40), the trend for the higher aromatics level
(green) appears steeper than for the lower aromatics level (black). Similarly, in the arom x etOH
view (Figure 41) the trends with respect to aromatics increase in steepness with ethanol level,
with (typical) exception of the trend for 15%. As before, we can discount this result; the left-
hand point in the 15% trend represents a single fuel, with low aromatics but high T90, and the
right-hand point end only 2 fuels. Of course, any conclusions based on the visual review must
remain tentative pending the generation of modeling results.
The plot of aromatics xT90 may suggest a reinforcement interaction, as the trend for 340 is
steeper than that at 300 (Figure 42). Both fuels with T90 at 325 have 35% aromatics and so do
not help confirm or rule out an interaction. The view of T90 x Aromatics gives a similar picture
(Figure 43), with the trend with respect to T90 steeper for high aromatics (green) than for low
aromatics (black).
76
-------
Figure 32. Common logarithms of Particulate Measurements for Bag 1. The bottom view highlights
censored measurements.
i
* *
«
i
'^
5v-
TO 200 300 400 sco BOO TOO AGO BOO 1000
Cbs.. (eorted by vehicle, fuel)
,
& 3 C CKJTIOOKXZ
* *
2Q0300-4005006007000C08001000
Obs,. (coded by vehicle. tjeJ)
cft*t of 00 acMM to mg, or ZM> mM*mmMi>
77
-------
Figure 33. Linear-effects plot for Ethanol on Bag-1 PM: PM and ln(PM) vs. Ethanol Level, with data
averaged by four Ethanol levels and by Vehicle.
COBMT
IMR4LAL3
SILVERADO
57,2011 2
OOBU-T
IMFALAL3
SILVERADO
78
-------
Figure 34. Linear-effects plot for Aromatics on Bag-1 PM: PM and ln(PM) vs. Aromatics Level, with
data averaged by two Aromatics levels and by Vehicle.
ArafrntJca (%)
vahide
-CCflOLLACE
uosrrt
20
'CALIBER «
EXFLOBERXLT PBO
' CO/3SEY *» CUTLOCKXE
40
'CIVIC
'FOCUS
3IEWA
HMFMLALS
SILVERADO
0
vrfiidu "-ALJ1MA
CCflCtlACE
1 CAUBER
EXPLOflEPKLT
PBO
QUTLDCKXE
30
'CIVIC
'FOCUS
SIEN4A
'COBMT
'IMFALAL3
SILVERADO
79
-------
Figure 35. Linear-effects plot for RVP on Bag-1 PM: PM and ln(PM) vs.RVP Level, with data averaged
by two RVP levels and by Vehicle.
FWP (Ib)
wshde
'ALJ1MA
OCRCLLAOE
UBS1TY
CALIBER
EXFLOREFX1T
ccr/ssEY
FW>
OUTLOCKXE
'CIVIC
'R3CUS
SIEM4A
CCBU.T
MMRALALS
SILVERADO
I 1
vahid* '-"ALJIMA
"-OCBOUACE
CALIBER
EXPLCflEIWLT
RWP (Ib)
BC*VW -'-'CIVie
PBO "FOCUS
OUTLOCKXE SIEM4A
CCBU.T
HMRALALB
SILVERADO
80
-------
Figure 36. Linear-effects plot for T50 on Bag-1 PM: PM and ln(PM) vs.TSO Level, with data averaged
byfiveTSO levels and by Vehicle.
OQBVLT
MFWLAL3
SILVERADO
81
-------
Figure 37. Linear-effects plot for T90 on Bag-1 PM: PM and ln(PM) vs.T90 Level, with data averaged
by three T90 levels and by Vehicle.
30
2D
303
vrfiicte
3D
320
340
TSO
AU1MA
OCRCLLACE
CALIBER
EXFLOflBRXLT
CAMRY
PBO
" OinLOCKXE
SIEM4A
CCBALT
MMFKLALS
SILWRADO
OOROLLACE EXPtOflEPWT
340
CCBVLT
MRA1A1S
SILWRADO
82
-------
Figure 38. Bag-1 PM: Interaction plot for etOHxTSO. Data are averaged by Ethanol and T50 levels,
and plotted vs. Ethanol level, with a separate series for each T50 level. Note that not all points on the
plot represent the same number of fuels.
Bhend (%)
T50
so *-" 230 frr^ ato
16
«
14
19
12
11
10
!
07
0.6
05
0.4
03
02
TSO
-D
Bhand
KJ * IB «
83
-------
Figure 39. Bag-1 PM: Interaction plot for TSOxEthanol. Data are averaged by T50 and Ethanol levels,
and plotted vs. T50 level, with a separate series for each Ethanol level. Note that not all points on the
plot represent the same number of fuels.
I 3
tn
Bhund
-BO SCO
Tso
0 "
21)
250
290
240
16
15
14
13
12
11
1O
OH
oa
07
06
05
C.4
oa
02
0.1
oo
"GO 200
TSD (dBQ.F)
22D
230
240
Bhund (*) "-" 0
84
-------
Figure 40. Bag-1 PM: Interaction plot for Ethanol x Aromatics. Data are averaged by
Ethanol and Aromatics levels, and plotted vs. Ethanol level, with a separate series for
each Aromatics level. Note that not all points on the plot represent the same number of
fuels.
15
VI
13
«
11
to
04
i°*
£ 07
06
OB
04
03
e&
01
00
Bhend pfc)
Annuities (%) *** B »
85
-------
Figure 41. Bag-1 PM: Interaction plot for Aromatics x Ethanol. Data are averaged by
Aromatics and Ethanol levels, and plotted vs. Aromatics level, with a separate series for
each Ethanol level. Note that not all points on the plot represent the same number of
fuels.
Bhund (*)
Aromatics (*>)
o ** 10 »»
15
w
13
12
M
10
OS
0«
07
06
05
01
03
02
dl
0.0
30
(%)
Bhund (*)
86
-------
Figure 42. Bag-1 PM: Interaction plot for Aromatics x T90. Data are averaged by Aromatics and T90
levels, and plotted vs. Aromatics level, with a separate series for each T90 level. Note that not all
points on the plot represent the same number of fuels.
20 30
Aramotks (%)
"RO (de&F) *-" 303 »» aa "a 340
15
14
13
12
11
10
OS
I0*
£ 07
06
05
04
03
02
&1
00
20
Arcmatlcs
T90 (da^F) "" 303 **
30
87
-------
Figure 43. Bag-1 PM: Interaction plot for T90 x Aromatics. Data are averaged by T90 and Aromatics
levels, and plotted vs. T90 level, with a separate series for each Aromatics level. Note that not all
points on the plot represent the same number of fuels.
300
ait
330
TOD (dag F)
330
340
A-cmatcs (*)
300
Animates
88
-------
5 Preliminary Modeling
Following construction of the dataset, we conducted an initial round of modeling, primarily for
purposes of outlier detection and influence analysis.
5.1 Assumptions
In the course of the analysis, we fit statistical models to emissions results for selected species,
including THC, NMOG, NMHC, CH4, CO, NO* and PM. The emissions data represent cycle
aggregates (i.e., Bag measurements) for the phases of the LA92 cycle, as described above. To
allow for potential differences in behavior between start and running emissions processes, we
analyzed the data separately by test phase (Bag).
Quantitation. For some emissions in some bags, subsets of observations fell below the limits of
quantification (LOQ) for the measurement techniques used. In practical terms, this outcome
means that the sample measurements, ostensibly from the tailpipe, were lower than background
levels in the ambient air. When this result occurred, the measurements were set to zero. Note
that for NMOG and NMHC, the situation is more complex in that these emissions represent
quantities calculated from sets of speciated hydrocarbons, as described above in Section 2.2. In
any case, values for either of these two emissions can also be considered missing if the
calculation cannot be performed for any reason, such as the absence of a key constituent.
Censoring. For purposes of analysis, we treated these measurements as "censored." Specifically,
we refer to affected datasets as "left-censored," because the lower, or "left tail" of the
distribution was censored by limitations in our ability to quantify very small pollutant
concentrations in the exhaust sample. We assume that a very small but positive measurement
existed but was not captured and quantified. Assigning a value of zero to these observations is
an example of a common approach to censoring of observations, known as "substitution." In this
approach, a small but fixed quantity is substituted for the censored observations. Values used for
substitution include zero, as mentioned, or small but positive quantities such as the smallest
observation, a multiple of the smallest observation, the limit of quantitation (LOQ) or half the
limit of quantitation (LOQ/2)11. At different stages of the analysis, we addressed censoring in
different ways.
The degree of censoring varied widely by emission and bag, as shown in Table 16, although
some patterns were observed, which can be related to the characteristics of the test cycle. The
cold-start phase (Bag 1) of the LA92 is shorter than its counterpart in the FTP. Nonetheless, the
presence of the cold-start increment provided generally larger measurable masses, hence the
fraction of censored values is lowest in Bag 1. The hot-running phase, Bag 2 is longer than the
preceding start bag, as well as the hot-running bag of the FTP. In addition, it is more aggressive,
containing some transient operation with hard acceleration. These characteristics, despite the fact
that the catalyst was lit off, provided sufficient measurable mass that the censoring rates in bag 2
were relatively low. In contrast, Bag 3 represents a hot start condition, involving a repeat of Bag
89
-------
1 for a conditioned engine and control system. Thus, the final bag presented the greatest
challenge to the measurement techniques employed, as shown through the highest censoring
rates and the highest levels of model uncertainty and random error relative to the other two bags.
90
-------
Table 16. Numbers of Censored Measurements, by Emission and Bag (ntotai = 956 measurements).
Emission
THC
NMOG
NMHC
CH4
NO,
CO
PM
Involves speciation
NO
YES
YES
NO
NO
NO
NO
Test Phase
Bagl
0
0
0
0
2
0
45
Bag 2
o
6
44
44
0
4
0
47
Bag3
2
119
119
0
25
0
82
Transformation of Emissions Data. In all models, the response variable is always the natural log
transformation of the emissions result. This step takes advantage of the tendency of regulated
emissions to follow approximately log-normal distributions. In addition, this transformation is a
standard approach to normalizing the distributions of residuals and stabilizing their variance
9
across ranges of fuel properties . Two additional justifications apply to use of the
transformation. First, in interpretation of results, effects of fuel changes are expressed as ratios or
percentages, calculations which are multiplicative on un-transformed data but very conveniently
translated to additive operations on transformed data (i.e., differences of logarithms). Second,
after reverse transformation, no fitted response can give a negative result, whether appropriately
1 9
or inappropriately extrapolated .
Standardization. In all models, the independent or predictor variables are always standardized as
described above (one- and two-stage). One-stage standardization was applied to linear or linear-
effects terms (Equation 5), and two-stage standardization to quadratic or interaction terms
(Equation 6, Equation 7).
Design Model. As described above, the study design was optimized with respect to a model
including 11 terms (12 with the intercept), including five linear-effect terms, two quadratic
terms, and four interaction terms between ethanol and the other four fuel properties (see
Equation 2, page 21).
ExtendedModels. During modeling, six additional terms were included, to explore the
possibility that additional effects could be estimated using this dataset. Throughout this
document, models including all possible terms will be referred to as "full" models. Models
including all 17 terms (excluding the intercept) will be referred to as "17-term extended models,"
and those with the 11 optimized terms will be referred to as the "11-term full model," or the
"design model." All 17 terms and the notation used to identify them are shown in Table 17.
91
-------
Table 17. Description and notation for parameters included in model fitting.
Fuel Parameter
Ethanol content (%)
Aromatics content (%)
RVP (psi)
T50(°F)
T90 (°F)
Model term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Arom x T50
Arom x T90
T90 x T90
T50 x T90
Arom x RVP
RVP x T90
In optimized design
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
NO
NO
NO
NO
NO
NO
Notation
Ze
7-a
7-r
Z5
Z9
ZZee
ZZ55
77
ZjZj QQ
ZZe5
ZZep
77
ZJZJgr
7-.7.a5
ZZap
ZZpp
ZZ^p
77
^^ar
ZZrp
Standardization1
One-stage
One-stage
One-stage
One-stage
One-stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
1 For one-stage standardization, see Equation 5, for two-stage standardization, see Equation 6 and Equation 7.
5.2 Identification of Influential Observations
We fit an initial model for each Bag and emission to allow identification of influential
observations. For this purpose, we used 17-term extended full models. In this initial step,
censored measurements were replaced with the minimum positive value measured for the
emission and Bag.
All models were fit as mixed models. The standardized fuel properties were treated as
continuous numeric variables and assigned as fixed factors; each vehicle was treated as a class
variable and assigned as a random factor. We fit the models as "random coefficients models," in
which the random effect is a random intercept fit for each vehicle. However, we did not attempt
to fit random slope coefficients for individual vehicles.
Random slopes by vehicle would effectively comprise an interaction between vehicle and fuel.
However, the study design and analysis approach do not allow for fitting such interactions. At
the outset, it is clearly logical to fit random intercepts by vehicle, given that "vehicle" is the
sampling unit for the study, for which reason we treat it as the blocking variable with respect to
the fuel parameters, i.e., the "treatments" in the experiment. However, similar conditions with
respect to "fuel" do not apply. As mentioned, we are not analyzing emissions in relation to the
actual fuels used, but rather to fuel properties abstracted from the fuels and assumed to be
92
-------
effectively independent (after design optimization and standardization). Thus, it is inappropriate
to include the carefully designed and optimized fuel property matrix in the random component of
the model because it in no way represents a random sample of available fuels, nor even a
reasonable judgment sample spanning common combinations of fuel properties. Finally, the
purpose of the study is to assess changes in mean emissions with respect to changes in fuel
properties, which makes the fuel properties "fixed," by definition. Similarly, the various vehicles
contribute much variability, which we isolate by treating vehicles as "blocks," which allows us to
estimate the between-vehicle variability, without impairing our ability to estimate the effect of
"treatment," i.e., the fuel parameter effects.
We fit the models using the MIXED procedure in SAS 9.2° using "maximum likelihood" as the
solution method to allow comparison of fit among nested models with different numbers of
parameters. No covariance structures were modeled using the REPEATED statement, reflecting
an assumption that the variance of residuals is uniform throughout.
As a measure of influence, we calculated the externally studentized or "studentized-deleted"
residual (r.,). Observations with externally-studentized residuals greater than or equal to 3.5 in
absolute value (| r.t > 3.5) were flagged for further evaluation. Numbers of influential
measurements are summarized in Table 18, by emission and bag. Individual influential
measurements are listed in Appendix G.
Table 18. Counts of Influential Measurements (out of a total of 956), by Emission and Bag (with
"influential" defined as having a studentized-deleted residual > 3.5 or < -3.5).
Emission
CO
THC
NMOG
NMHC
CH4
NO,
PM
Test Phase
Bagl
0
0
0
0
0
3
1
Bag 2
0
0
0
0
0
7
1
Bag3
0
0
0
0
0
0
0
Measurements identified as influential were reviewed in detail on a case-by-case basis. EPA and
its collaborators in the study agreed that measurements could be considered for removal from the
dataset if (1) a physical reason were found that plausibly indicated that the data points could be
invalid, or (2) the study participants reached a consensus that specific data points were
sufficiently unusual or problematic to justify their removal. Based on these criteria, Two
measurements were removed, both for paniculate matter (PM). One measurement was removed
93
-------
in Bag 1 and another in Bag 2. An additional measurement in Bag 3 was removed, even though
it was not flagged as influential.
In Bag 1, the run selected for removal (run 6247) has a value of 413 mg/mi (corresponding to a
studentized-deleted residual of 4.45) whereas all remaining measurements on the same vehicle
(Liberty) have values less than 35 mg/mi. Further, the replicate measurement (run 6259) on the
same fuel (16) has a value of 20.0 mg/mi, which is lower than that for run 6247 by a factor of
20.7. See Figure 44 below.
Similarly, in Bag 2, the run selected for removal (Run 5284) has a value of ~110 mg/mi
(studentized deleted residual = 4.27) whereas all remaining measurements on the same vehicle
(Explorer) have values of 0.50 or less. The selected run exceeds all other measurements on all
other fuels by a factor of 220. See Figure 45.
Finally, in Bag 3, the run selected for removal (Run 6281) has a value of-62 mg/mi, whereas the
remaining measurements on the same vehicle (Explorer) have values of 5 mg/mi or less. See
Figure 46. Despite its size, this point was not flagged as influential.
Figure 44. PM (Bag 1): Measurements for the Jeep Liberty, by fuel. The measurement identified as
influential and selected for removal (Run 6247, on fuel 16) has an exceptionally high value (413
mg/mi).
500-
inn
f 350-
1"
,-- 300 -
ID
CD
^ 250-
c
g
VI
UJ
1 150-
100-
50-
0-
_ 1
) 8 «
Li_,
> 0 '
) ° <
O
300 S @ § 8
06 00^0 eo°
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Fuel Number
94
-------
Figure 45. PM (Bag 2): Measurements for the Ford Explorer, by fuel. The measurement identified as
influential and selected for removal (Run 5284, on fuel 27) has an exceptionally high value (~110
mg/mi).
125-
100-
E
en
E
- 75-
en
n
S-
0
ift
."> 50 -
HI
5
Q.
25-
0-
t
)
oooooooooooooooo
0 0 0 0 O O O O O
0 0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Fuel Number
Figure 46. PM (Bag 3): Measurements for the Ford Explorer, by fuel. The measurement identified as
influential and selected for removal (Run 6281, on fuel 10) has an exceptionally high value (~62
mg/mi).
70-
E
£ 40-
ra
E.
s=
o
» 30 -
E
HI
Q- 20 -
10-
0-
o
o o o (
1351
!
o
1 ° E
3 8 <
> 8 <
> 8
0 <
o
' 0 S
3 8 <
JLi
S o
8 8
11 13 15 17 19 21 23 25 27 29 31
Fuel Number
95
-------
5.3 Reduced Models
With the full models as a starting point, the next step was to begin the process of fitting
"reduced" models, defined as models containing fewer terms than the full models. The reasons
for this approach are several. A primary goal is to identify models including subsets of terms
limited to those shown to be the most meaningful or useful in explaining and predicting the
emissions measured. The full sets of terms in the optimized design include terms anticipated to
be meaningful for any of the emissions to be measured. However, it was not anticipated that all
the terms included would necessarily be meaningful for all emissions in all bags. A closely
related goal is to develop models that would be, to the extent possible, explicable in terms of
knowledge of the relevant physical and chemical processes. Parsimonious models are preferred
over full models for this purpose, as their simpler structure makes their behavior easier to assess
and explain. Finally, with respect to explicability, it is much preferred to minimize the potential
for overfitting, which could reduce the generality of models selected for prediction. To guide the
process, we adopted several assumptions, described below.
Hierarchy. At the outset, we imposed a requirement that the principle of "hierarchy" be
maintained during the fitting process. Briefly stated, the principle requires that if any two-way
interaction term AxB is retained in a model, both linear terms A and B must also be included,
whether or not the linear terms appear highly significant taken alone. The maintenance of
hierarchy serves to ensure that the reduced model(s) obtained are interpretable. The retention of
an interaction and its linear terms is interpreted to mean that linear relationships exist between
the response variable and both linear terms (A and B), and that the interaction (AxB) describes
how these relationships vary with differing levels of the two factors involved. For example, if
the slope for AxB is positive and significant, it suggests that the slope for A is steeper at a high
level of B than at a low level of B, and vice versa, or that increasing the level of B reinforces the
effect of A, and vice versa. Conversely, a negative and significant slope for AxB suggests that
the slope for A is steeper at a low level of B than at a high level, suggesting that increasing the
level of B may dampen or interfere with the effect of A. In either case, the interaction acts as a
modifier or refinement to the underlying linear relationships, but for this model structure to
remain intelligible, both linear effects must be retained, if the interaction is retained.
Removal of Outliers. Before the outset of model fitting, influential observations identified as
problematic outliers were removed. As described above, this step affected only PM models
(Bags 1-3).
Left-censoring. After deciding that censoring was too important an issue to neglect, the study
participants adopted a consensus on how to address it in model fitting. For minimal levels of
censoring, defined as five or fewer censored measurements (^censored < 5), we elected to substitute
the minimum positive measured value for the missing measurements. After substitution we fit
mixed models as described above.
96
-------
For higher levels of censoring, defined as more than five censored measurements (^censored > 5) in
a given Bag for a given emission, we elected to not rely on substitution, for two main reasons.
The first is that substitution is known to introduce bias in the estimation of summary statistics for
censored distributions13. Secondly, given that the purpose of this analysis is not to estimate
means and variances for single censored distributions, but rather to develop multidimensional
models to represent the response of emissions to simultaneous changes in multiple fuel
parameters, substitution of uniform values for multiple missing measurements could introduce
bias by obviating potential relationships between emissions and fuel properties. In cases of
severe censoring, we adopted another approach commonly applied to left-censored datasets,
known as "censored normal regression" or "Tobit regression." We describe this approach in
greater detail below.
Removal of T9Ox T90 term. For development of reduced models, we removed the interaction
from the full model. Our rationale for this step is that the two fuels containing T90 at the middle
level (325) lie along one edge of the fuel parameter space, and thus lack sufficient depth across
the modeling space to allow for adequate estimation of this term. Thus, in development of
reduced models, we will refer to the "16-term extended model'' as opposed to the "17'-term
extended moder as used in 5.1 and 5.2.
5.3.1 Minimal Censoring (Mixed Models)
For cases involving minimal censoring, we fit mixed models as described above.
We employed two model fitting approaches (1) Backwards elimination, and (2) fitting all
possible models. In both approaches, the goodness of fit of various models was assessed using
the Bayesian Information Criterion (BIC), as calculated by the MIXED procedure. In this
formulation of the parameter, a reduction in the BIC indicates an improvement in fit. Thus, the
goal in model fitting is to identify the model(s) giving the minimum value of BIC. The MIXED
procedure was run using "maximum likelihood" (ML) as the solution method, to allow
comparisons of fit between models with different sets of fixed effects.
Backwards Elimination (based on BIC). This approach was applied as follows. The full model
was fit, including that maximum number of terms (p) and its BIC recorded (BIC(p}). The Type-
Ill tests of effect (^-tests or %2-tests) for the full model were reviewed, and one or more
parameters with insignificant tests at the 10% confidence level (a = 0.10) were identified. The
model was refit after dropping the kj selected parameters, and its BIC recorded. If the BIC for
the reduced model (ElC{p-kj}) was less than that for the full model, the reduced model was
retained as the current "best-fit" model for the first step. After reviewing the tests of effect for
the current reduced model, additional &2 parameters could be selected for evaluation. The
reduced model whhp-ki-k2 parameters was fit, and retained as the "best-fit" if RlC{p-k\-k2] <
BIC{/7-&i}. These steps were repeated until no additional terms could be removed without
increasing the BIC. Often, but not always, the tests of effect for remaining terms in the last
97
-------
model fit had/>-values < 0.10. In some cases, however, linear terms with high ^-values would be
retained to maintain hierarchy. Note that throughout, no terms were dropped based solely on the
tests of effect, but rather only on the goodness-of-fit. The a-level was set somewhat higher than
the commonly used 5% level, to reduce the potential for type-II error in dropping terms, i.e.,
erroneously dropping terms that help explain the results or improve model fit.
Fitting all possible Models. This approach involves fitting all possible combinations of the/?
parameters in the full model. A total of 16 parameters gives a total of 216 = 65,536 possible
models. With 11 parameters the corresponding total would be 2n= 2,048 possible models.
However, in both cases, very high fractions (-95%) of the totals do not respect hierarchy, and
thus are not eligible for further consideration.
In this process, the models were again run as mixed models, with algorithms written specifically
to submit every possible candidate to the MIXED procedure. At the completion of the process a
selected number of models with the best fits (lowest BIC), after excluding models without
hierarchy, were listed and ranked. The model with the lowest BIC typically matched the best fit
model from the backwards elimination process.
5.3.2 Severe Censoring (Tobit Regression)
For compounds and bags with high levels of censoring, we fit "censored normal regression," or
"Tobit" models, a technique commonly used for left-censored data14'15. We fit the models using
the LIFEREG procedure in SAS 9.2, as applied for cases of left censoring.
As with the mixed models, the procedure solves for the model parameters using maximum
likelihood estimation. However, the Tobit approach does not attempt to estimate the missing
values. Rather, the formulation of the maximum likelihood function (L) is modified so as to
compensate for the absence of the censored values and to estimate values for the model
coefficients accordingly13. In the Tobit model, each measurement is represented by its
probability density (standard normal), given an assumed set of parameters, and each censored
value is represented by the cumulative probability that the value would be less than the effective
censoring level.
As the LIFEREG procedure is not able to handle random factors, it was necessary to enter each
vehicle as a fixed factor, represented as an indicator or "dummy" variable. Thus, the model
outputs an intercept for each vehicle, and an estimate of random error variance. It does not
estimate a grand intercept for all vehicles, nor a component of variance representing the between
vehicle-variability (the variance of the random intercepts). These steps were performed
manually.
The procedure outputs the log likelihood (InL) as a goodness-of-fit parameter. It does not output
an estimate of the BIC, but the BIC is readily calculated from L, the number of model terms p
and the total number of (non-missing) observations n, as
98
-------
= -2\nL + p\nn Equation 12
In model-fitting, we started with the full model, and proceeded by backwards elimination. In
each step, one or more parameters were removed, and the model was refit. As with the mixed
models, models with one or more terms removed are referred to as "reduced models." Models
r\
were selected for removal based on the/>-value for their respective % -test of effect (p > 0.10),
starting with parameters with the highest ^-values.
At each step, each successive reduced model was tested for goodness-of-fit against the preceding
reduced model. At each step, if the current reduced model was not a significantly poorer fit than
either the full model or its predecessor, it was accepted as the current "best fit." To interpret the
goodness-of-fit test, the current reduced model was considered a poorer fit than the full or its
predecessor if the/>-value for the likelihood ratio test was < 0.10. The process was repeated until
the current reduced model was a significantly poorer fit than its predecessor.
In performing the likelihood ratio tests, it was necessary that the two models included in the test
be "nested," i.e., that both models have all terms in common except the subset of terms whose
inclusion is the subject of the test. This condition always applied, in that all reduced models were
nested within the full model, and each reduced model was nested within the preceding reduced
model.
For a specific test, the model with more parameters is designated as the "reference" model, and
the model with fewer parameters as the "nested" model. The test was fit in standard fashion,
using the log-likelihood statistics output as the primary fit statistics for the models (all models
were fit by maximum likelihood estimation) as shown in Equation 13. The test statistic is
calculated as the difference in the -21og-likelihood between the nested and reference models, and
which is assumed to be distributed as a %2 statistic with d degrees of freedom, where dis the
difference in the numbers of parameters between the two models (pref-^nested).
= -2 m 4ested - (- 2 In 4eference) ~ %} Equation 13
reference
The test is considered significant if the/>-value was less than 0.10. The process is repeated until
a significant result is obtained or if the remaining fixed effects were significant (p-value < 0.10).
The set of terms remaining following the final step was retained as the "best fit" reduced model.
99
-------
5.4 Initial Modeling: Summary
All models are based on the 16-parameter full model, as shown in Equation 14. All terms shown
in Table 17 above were included at the outset of model fitting, with the exception of T90xT90.
Results for preliminary models are summarized and presented in Appendix E.
Using the notation for standardized predictors, also shown in Table 17, the model is also
expressed in Equation 15.
?2 Arom+/?3 -RVP + /?4 -T50 + /?5 -T90 +
/?6-T502+/?7-etOH2
Equation 14
/?8 -etOHxArom+/?9 -etOHxRVP + /?10 etOHxT50 + /?11 -etOHxT90 +
/?12 Aromx T50 + /?13 Aromx T90 + /?14 T50 x T90 + /?15 Aromx RVP + /516 RVP x T90 +
&ZZ55+/]7ZZee
Equation 15
&ZZea + j89ZZer + j8wZZe5 + A ,ZZe9 +
A2zzfl5 +0l3zza9+pl4zz59+j315zz
e
5.5 Initial Modeling: Influence Analysis
A parallel analysis of these data, performed under contract to the DOE, employed methods and
approaches similar to those described so far in this chapter (chapter 5). This analysis identified
influential observations using an approach very similar to that described above in 5.2. The
approach to analysis of censored measurements, as described in 7.2.2.5, was also adopted based
on guidance from the author of the DOE research . Thus, the criteria for applying mixed models
or Tobit regression is the same in both analyses. Reduced models were also identified in the
DOE analysis, emphasizing the "fitting all models" approach using mixed models when
censoring was minimal, and reporting lists of terms making up potential good candidate sets.
When censoring was severe the DOE analysis also applied Tobit regression, with models fit by
backwards elimination, as described above. On the whole, the "best-fitting" reduced models
reported for the DOE analysis are similar to those reported above in 5.4. However, due to
limitations in time and resources, the DOE research did not extend into the additional influence
analyses described below.
1 Professor Richard F. Gunst, Southern Methodist University, Department of Statistical Science.
100
-------
Following the completion of the initial model fitting, the reduced models were used for purposes
of influence diagnostics. In this round, however, rather than identifying individual influential
observations, the goal was to assess the influence of vehicles, which served as the sampling units
in the project. A broader aim was to assess the extent to which inclusion or exclusion of
particular vehicles might affect the model fits.
To achieve this step, we ran additional models, using the sets of terms retained in the initial best
fits. These models were run as mixed models in all cases, even when the initial reduced model
was obtained using a Tobit model. This change in procedure was adopted as the LIFEREG
procedure used to fit the Tobit models lacks the automated diagnostic features available in the
MIXED procedure. Thus, in running these models, censored values were replaced with the
minimum positive measured value in each bag for each emission.
The influence of each vehicle was assessed using several diagnostic parameters. These included
the "restricted likelihood distance" (RLD), the "multivariate DFFITS" (MDFFITS), the
"covariance ratio" (CovRatio), the "PRESS statistic" (PRESS) and the "multivariate DFFITS for
the covariance parameters" (MDFFITSCP). Each of these parameters assesses the influence of
each vehicle with respect to the various aspects of the model-fitting process16.
At the outset, the RLD is a measure of the influence of each vehicle on the overall fit of the
model. It measures the change in the value of the log-likelihood for the entire dataset, using sets
of parameters generated with and without the subset of data under consideration. The MDFFITS
measures the influence of vehicles on the values of the coefficients of the fixed parameters, i.e.,
the parameter vector. It calculates the change in the set of coefficients, relative to their
uncertainty. Similarly, the MDFFITSCP estimates the influence of vehicles on the values of the
covariance parameters, or the variance of the random vehicle intercepts in this analysis. As
opposed to the values of the coefficients themselves, the covRatio measures the influence of
vehicles on the precision of the estimates, and is calculated as the ratio of the determinants of the
covariance matrix of the parameters (cov/?). The influence of vehicles on the models' predicted
values is assessed using the PRESS statistic (Predicted Residual Sum of Squares). For each
observation, the PRESS residual is calculated as the difference between each observation and the
predicted marginal mean, estimated without the vehicle in question. The marginal mean is the
prediction obtained using only the fixed parameters in the model (grand intercept and fuel-
parameter effects), but excluding the random parameters (individual vehicle intercepts). For a
vehicle, the PRESS statistic is the sum of squared PRESS residuals for all observations on that
vehicle. For the RLD, MDFFITS and PRESS statistics, large values for a vehicle, in relation to
those for other vehicles indicate high influence. For the covRatio statistic, a value of 1.0
indicates "no influence," and a value much larger or smaller than 1.0 indicates a high degree of
influence.
Table 19 shows influence results for NO* (Bag 1). The Ford Focus (FFOC) stands out as highly
influential based on all five statistics. Its strong influence on the overall fit is indicated by its
101
-------
high RLD, followed by the Odyssey. On the basis of MDFFITS however, the Odyssey is the
most influential, with the Focus as a very close second, indicating that both these vehicles
strongly influence the values of the coefficients. For the covRatio, a number of vehicles,
including the Focus, have values greater or larger than 1.0, where a value of 1.0 is a benchmark
of "low influence." The values of the covRatio, suggest that the vehicles strongly affect the
precision of the fixed-effect parameters. For the PRESS statistic, the Focus has the highest value
by a wide margin, but the second place is held by the Sienna rather than the Odyssey. For the
MDFFITSCP statistic, the Focus is clearly most influential, but with the Odyssey much less so.
Review of the dataset clarifies these results to some degree (Figure 12, page 53). The emissions
for the Focus are a full order of magnitude lower than those for all other vehicles, with the
exception of the Sienna, which is also somewhat influential. However, neither the Odyssey nor
the Sienna is nearly as influential as the Focus. Interestingly, the emissions for the Odyssey are
among the highest for all the vehicles. However, the Odyssey stands out in that a subgroup of its
measurements are markedly lower than the main group, substantially increasing the variability of
its measurements overall.
Table 19. NO* (Bag 1): Selected Influence Statistics, by Vehicle.
Vehicle
Cobalt
Impak
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Altima
Outlook
Camry
Corolla
Sienna
n
63
64
65
61
66
61
63
65
63
58
62
65
68
71
61
RLD
1.309
1.366
1.665
1.621
1.619
1.784
59.671
1.105
12.131
1.823
1.658
1.092
2.067
2.3
5.825
MDFFITS
0.0673
0.0460
0.0504
0.0479
0.0936
0.0343
0.5843
0.0748
0.5982
0.0497
0.0379
0.0476
0.0216
0.1148
0.2170
CovRatio
3.608
4.154
4.605
4.342
3.535
4.847
0.039
2.896
0.876
4.001
4.187
3.784
5.737
1.783
0.797
PRESS
16.17
16.39
6.85
4.98
57.69
2.72
405.82
21.09
55.51
94.65
91.64
8.15
31.17
52.85
292.27
MDFFITSCP
0.3446
0.6556
0.8674
0.8584
0.2676
1.1777
59.6382
0.052
3.7042
1.0302
1.0217
0.3904
1.5672
0.6622
2.831
Table 20 shows results for NOX (Bag 2). In this case the Cobalt stands out as most influential by
all measures. Identifying the second- and third-most influential vehicles is more ambiguous, but
the Civic and Odyssey have RLD values slightly higher than the remaining vehicles and
MDFFITS values considerably higher than the remaining vehicles. In bag 2, the set of
measurements for the Cobalt is more variable than for any other vehicle, with measurements on
102
-------
different fuels spanning roughly two orders of magnitude; no other vehicle is nearly as variable
(Figure 22, page 65). Again, the Odyssey stands out somewhat in that it has a noticeable subset
of measurements that stand out from the majority, although in contrast to Bag 1, the subset is
higher rather than lower than the main group.
Table 20. NO* (Bag 2): Selected Influence Statistics, by Vehicle.
Vehicle
Cobalt
Impah
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Aftima
Outlook
Camry
Corolla
Sienna
n
63
64
65
61
66
61
63
65
63
58
62
65
68
71
61
RLD
60.037
0.788
1.318
0.391
0.863
0.134
0.75
2.919
3.612
0.674
0.281
0.406
2.14
1.249
1.698
MDFFITS
1.8257
0.0193
0.0393
0.0386
0.0742
0.0389
0.1631
0.8214
0.8051
0.0317
0.0769
0.0808
0.0105
0.0294
0.0084
CovRatio
0.289
.374
.424
.251
.317
.273
.184
.252
.181
.402
.255
.254
.490
.470
.470
PRESS
747.50
55.92
41.46
27.75
97.78
56.08
169.42
31.15
48.70
8.34
54.90
47.91
27.68
21.97
13.63
MDFFITSCP
65.8804
0.6593
1.0744
0.2519
0.5666
0.008
0.2108
0.2712
0.9911
0.5237
0.031
0.1363
1.8705
1.0328
1.5037
Table 21 shows the results for NMOG (Bag 1). For this compound, two vehicles, the Focus and
Outlook, are roughly tied as most influential by three of five criteria, with the Altima, Corolla,
and F150 following in 3r to 5* places, respectively. These two vehicles stand out in terms of
their influence on the overall fit, the fixed-model coefficients and precision of the estimates.
Examining individual measurements by vehicle, as shown in Figure 47, shows that these two
vehicles are distinguished in the variability and distribution of their observations. The Focus
shows order-of-magnitude variability, largely due the fact that its measurements form two
distinct groups. One group, containing approximately 25% of its measurements is in the same
range as most other vehicles. However, a second group sits lower than emissions for all but three
of the 15 vehicles, with the lowest values in the entire dataset coming from the Focus. The
Outlook, by contrast, is distinguished in that while most of its measurements are in the same
range as most of the vehicles, it has a subset of measurements that are 0.3-0.4 orders of
magnitude higher than the remainder, with this vehicle accounting for the highest measurements
in the dataset, with the exception of the F150. It appears that these patterns may be related to the
relatively high influence of these vehicles. As with NOX, the covariance ratio shows that the two
vehicles having the most influence on the values of the coefficients have the least influence on
their precision.
103
-------
Table 21. NMOG (Bag 1): Selected Influence Statistics, by Vehicle.
Vehicle
Cobalt
Impala
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Altima
Outlook
Camry
Corona
Sienna
n
63
64
65
61
66
61
63
65
63
58
62
65
68
71
61
RLD
1.938
2.605
2.545
1.645
4.741
0.881
14.897
3.822
2.354
2.088
4.575
19.239
0.747
5.984
1.084
MDFFITS
0.0302
0.1825
0.1836
0.0168
0.3845
0.0681
0.5206
0.3071
0.0783
0.0787
0.3822
0.6169
0.0574
0.2777
0.0269
CovRatio
4.062
3.094
3.058
3.806
1.860
2.493
0.475
2.126
2.998
2.819
1.971
0.485
2.203
1.165
3.324
PRESS
0.96
2.98
3.53
2.89
18.53
5.34
40.86
27.61
27.69
23.51
18.04
15.17
6.22
13.73
2.68
MDFFITSCP
1.4196
0.4117
0.3447
1.3058
0.1086
0.0735
8.9811
0.1529
1.3193
1.063
0.0126
12.4328
0.0629
2.5616
0.7003
Figure 47. NMOG (Bag 1): Common logarithm of Measurements, by Vehicle and Fuel.
110, 2011 52
For Bag-2 NMOG, influence statistics are shown in Table 22. These results show the Sienna to
be most influential by all measures except the PRESS statistic, with the Odyssey in second place
for all measures except the PRESS statistic. The F150 has the highest PRESS statistic, which in
104
-------
this case reflects that fact that its measurements are highest of all the vehicles, as shown in
Figure 48. In contrast, the Sienna and Outlook are conspicuous in that they are most variable,
have the lowest measurements and contribute all censored measurements for this dataset. Note
also that the measurements for these vehicles are imputed from NMHCpio, as previously
described in 3.2.1 and shown in Figure 49.
Table 22. NMOG (Bag 2): Selected Influence Statistics, by Vehicle.
Vehicle
Cobalt
Impala
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Aftima
Outlook
Camry
Corolla
Sienna
n
63
64
65
61
66
61
63
65
63
58
62
65
68
71
61
RLD
0.545
1.223
1.885
0.761
2.499
1.092
1.361
0.366
17.019
0.797
1.813
1.885
3.009
1.458
33.597
MDFFITS
0.0560
0.0597
0.0591
0.0035
0.0974
0.0678
0.1360
0.0293
0.1346
0.0339
0.0814
0.0428
0.2151
0.0272
0.8614
CovRatio
1.911
2.525
2.772
2.641
2.281
2.512
2.094
1.770
0.371
2.471
2.727
2.927
1.380
3.219
0.281
PRESS
65.32
83.66
137.40
6.43
340.19
16.64
24.11
62.13
330.09
7.14
60.60
102.54
90.73
18.02
222.50
MDFFITSCP
0.0063
0.5836
1.181
0.6627
1.4651
0.4047
0.0477
0.0766
16.5429
0.4293
0.9221
1.3234
0.891
1.0684
27.4382
105
-------
Figure 48. NMOG (Bag 2): Common Logarithms of Measurements, by Vehicle and Fuel.
r 10,2011 52
Qb&. (sorted by vehicle, fud)
+ + COEOLLACE EXPLOK5BJCLT JJ50
o n o oirY53Eir D a a OUTLOOCXE
106
-------
Figure 49. NMOG (Bag 2): Common logarithms of Observations by Vehicle and Fuel, delineating
speciated measurements (blue) and values imputed from NMHCpm (green).
110,2011 53
Cbs., (Eorted by vehicle, fuel)
dD_BHZ2 once i
dmi ml In-p*;! wJijM
Influence statistics for Bag 1 PM are shown in Table 23. The vehicles with the greatest influence
overall are the Liberty, Sienna, Explorer and Focus. However, in contrast to other emissions and
Bags, the range in the various measures is not as wide. Thus no individual vehicle stands out
dramatically with respect to the others. The Liberty has a markedly higher PRESS statistic, but
this value merely indicates that its emissions are highest among a pool of vehicles with low
between-vehicle variability compared to other emissions (Figure 32, page 77).
For PM (Bag 2), patterns in the influence measures are broadly similar to those in Bag 1 in that
no vehicles stand out dramatically in respect to model fit, parameter values or their precision
(See Table 24). As with the other emissions, the vehicle with the highest measurements has a
conspicuously high PRESS statistic.
107
-------
Table 23. PM (Bag 1): Selected Influence Statistics, by Vehicle.
Vehicle
Cobalt
Impala
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Altima
Outlook
Camry
Corona
Sienna
n
63
64
65
60
66
60
63
65
63
57
62
65
68
71
61
RLD
0.8448
1.1711
0.819
1.24
1.4635
2.6696
1.6378
1.421
1.1893
3.6971
0.4994
1.3185
0.7271
1.5608
3.1149
MDFFITS
0.0640
0.1023
0.0668
0.0159
0.0190
0.1348
0.1889
0.0420
0.1588
0.2224
0.0740
0.1104
0.0324
0.2300
0.0869
CovRatio
1.369
1.348
1.744
1.995
2.128
1.069
1.309
2.070
1.474
0.816
1.587
1.853
1.952
1.465
1.032
PRESS
96.21
102.55
71.50
21.41
24.63
152.49
113.75
29.33
82.89
231.28
67.64
53.70
43.38
135.61
153.88
MDFFITSCP
0.41179
0.48901
0.35215
1.03185
1.20473
1.76658
0.41219
1.03774
0.15175
2.9276
0.02499
0.55396
0.46756
0.0428
2.51621
Table 24. PM (Bag 2): Selected Influence Measures, by Vehicle.
Vehicle
Cobalt
Impala
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Altima
Outlook
Camry
Corolla
Sienna
n
63
64
65
60
66
60
63
65
63
57
62
65
68
71
61
RLD
0.3003
1.0491
0.8582
3.7201
0.671
1.4893
0.4502
0.7138
0.5833
2.4291
0.7088
0.353
1.1214
1.3307
1.6511
MDFFITS
0.0262
0. 1604
0.0989
0.5356
0.0516
0.1711
0.0438
0.0158
0.0050
0.3058
0.1595
0.0733
0.0241
0.1663
0.2862
CovRatio
1.280
1.048
1.355
1.177
1.545
1.126
1.483
1.578
1.565
0.872
1.373
1.434
1.632
1.179
1.236
PRESS
126.44
265.14
142.87
138.80
53.98
172.71
67.71
40.17
38.64
314.47
87.19
75.24
38.98
201.48
132.14
MDFFITSCP
0. 17797
0.39073
0.39695
1.37541
0.41104
0.7334
0.2415
0.58636
0.51223
1.22254
0.02283
0.03507
0.91643
0.57426
0.39142
108
-------
Influence diagnostics for CH4 (Bag 1) are shown in Table 25. Three of the five measures, except
the CovRatio and PRESS, show the Focus to be the most influential vehicle. Of all the vehicles,
the Focus shows the strongest influence on the overall fit and the values of the coefficients. With
respect to the precision of the coefficients, the Focus is in the four most influential vehicles,
although its influence opposite in direction. The Corolla, F150 and Liberty have the largest
values of the PRESS statistic. Review of the data points in Figure 50 shows that the Corolla has
the lowest values in the dataset, and the other two vehicles the maximum values. Thus, these
results for the PRESS are expected. The values for the Focus are at the lower end of the range,
but this vehicle is on par with two others that are not nearly as influential (Civic and Odyssey).
However, the Focus has a subset of points that sit considerably higher than the majority, which
may account for its influence.
Table 25. CH4 (Bag 1): Selected Influence Statistics, by Vehicle.
MDFFITSCP
1.816
16.408
22.734
4.436
60.58
7.301
19.503
31.095
33.47
1.286
0.431
0.082
1.130
0.359
0.238
0.215
0.054
0.061
0.223
0.098
0.081
0.121
0.192
0.017
1.124
0.941
0.081
0.260
0.649
0.860
0.302
109
-------
Figure 50. Cm (Bag 1): Common logarithm of Measurements, by Vehicle and Fuel.
Common logarithm of Bag 1 CH4, g/mi by Index
-OS
-0.6
-0.7:
-0.8
-OS
-1.0
-1.1
-1.2
-13
-\A
-\B
-1.6
-1.7:
-IjB
-\a
-2.0
«*
*$«
0° "l 8
%8>0o go °o
an8 °£o "
*»
c?f
20 40 60 BO 100 120 140 160 180 200 220 240 283 260 930 320 340 360 380 400 433
Obs., (sorted by vehicle, fuel)
h ooo CCOB ooo CMP oo* CSIL o o o DCAL o .:. .:. F150 FESP FFOC HCIV
HOCY MB oo n HALT ° ° Q SOUT ° D o TCAM ° ° ° i^OR D a a TSffi
For CH4 (Bag 2), influence measures are shown in Table 26. Again, the Focus is most influential
by all measures except the PRESS and covRatio. However, the differences between the Focus
and the other vehicles are not nearly as wide as in Bag 1. Viewing the Focus in the context of
the other vehicles, as shown in Figure 51, shows the results for this vehicle to be unremarkable.
Its measurements are on par with those for five other vehicles, and are not unusually variable.
The reasons for this vehicle's influence are thus unclear based on an initial review. Again, the
vehicles highest (Liberty) and lowest measurements (Odyssey, Corolla) have the largest PRESS
values.
110
-------
Table 26. CH4 (Bag 2): Selected Influence Statistics, by Vehicle.
Vehicle
n
0.0514
0.0528
0.1495
0.0733
0.1035
0.2067
0.1115
0.1037
PRESS MDFFITSCP
Cobalt
Impala
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Altima
Outlook
Camry
Corolla
Sienna
63
64
65
61
66
61
63
65
63
58
62
65
68
71
61
3.0700
124.7140
59.9610
55.1470
28.3030
59.1910|
86.9690
138.3610
32.1670
3.6390
4.2500
49.1980
204.3760
72.4840
0.3057
0.1939
0.2222
0.6034
0.6997
0.8980
2.624
0.1556
0.4563
0.2484
0.0319
0.1068
0.4080
0.0995
0.1304
Figure 51. CH4 (Bag 2): Common Logarithm of Measurements, by Vehicle and Fuel.
Common logarithm of Bag 2 CH4, a/mi by Index
-1/1
-«
-1.7
-1.8
-IS
-2.0
-2.1
-2.2
-23
-2/4
-25
-2.6
-2.7
-2.8
-2J8
-310
-3.1
-3.2
-9.3
-3xt
0 20 /O 60 30 lQD12014QieD1B020Q22D3a02S)9BD3303SOa40aBa3804004aQ
Obs., (sorted by vehicle, fuel)
Yeb ooo CCOB o o o CWP
-------
For CH4 (Bag 3), influence measures are shown in Table 27. In this case, the Corolla is
conspicuous with respect to all five measures. As the vehicle with the lowest measurements, as
shown in Figure 52, it has a large PRESS, followed by the F150, which has the highest
measurements.
Table 27. CH4(Bag3): Selected Influence Measures, by Vehicle.
Vehicle
Cobalt
Impala
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Altima
Outlook
Camry
Corolla
Sienna
it
63
64
65
61
66
61
63
65
63
58
62
65
68
71
61
RLD MDFFITS
0.637 0.0919
0.786 0.0306
1.049 0.1372
1.175 0.0485
2.129 0.2044
1.449 0.2209
3.922 0.3757
1.906 0.2950
4.417 0.6303
1.055 0.0921
1.378 0.0174
0.794 0.1099
1.103 0.1504
36.655 1.0974
1.526 0.0399
CovRatio | PRESS
1.5736 6.0930
1.7951 49.8830
1.6826 44.5850
1.9223 23.3200
1.4788 163.2620
1.5706 12.0210
1.2113 10.7820
1.5508 55.2680
1.3534 30.8480
1.7339 46.5800
2.0765 1.0980
1.5330 6.5010
1.7131 38.4500
^^^MSS^^^H
2.0142 21.4020
MDFFITSCP
0.0385
0.5357
0. 1494
0.7871
0.7787
0.0314
1.4645
0.0139
0.3214
0.4262
1.1424
0.0809
0.1199
32.1543
1.1480
112
-------
Figure 52. CH4 (Bag 3): Common Logarithms of Measurements, by Vehicle and Fuel.
Common logarithm of Bag 3 CH4, g/nri by Index
0 20 40 fC SO 1CD 1JO UO 1fO 160 300 233 240 260 533 330 350 340 3S3 3EO
Obs., (sorted by vehidg, fuel)
As an outcome of the influence analyses, we performed additional detailed review of the
measurements for the vehicles identified as highly influential, summarized in Table 28. The
review and its results are discussed below in Section 6.
Table 28. Vehicles Selected for detailed review for subsets of NOX, NMOG and CH4 results.
Vehicle
Cobalt
Impala
Silverado
Caliber
F150
Explorer
Focus
Civic
Odyssey
Liberty
Altima
Outlook
Camry
Corolla
Sienna
NOx
Bagl
Bag 2
NMOG
Bagl
Bag 2
CH4
Bagl
Bag 2
Bag3
113
-------
6 Measurement Issues
6.1 Data Quality at Very Low Emission Levels
Looking at the results of the influence analysis alongside plots of the measured values revealed
that the most influential vehicles tended to have a large number of very low measurements.
These observations led to a closer examination of measurement error in the dataset.
All measurement processes have error associated with them as a result of the physics of mixing
and sampling from a gas stream or noise in electronic components such as optoelectronic
detectors or signal amplifiers. This means that repeated measurements taken under identical
process conditions would produce a range of results, their average being the true response of the
instrument and the range around it representing the measurement variability. The dilute bag
method requires measurement of concentrations in both the sample and background (ambient or
dilution air) bags, followed by a calculation that subtracts the background from the sample after a
dilution factor correction. Thus, the net result contains a linear combination (sum) of variability
from two measurements. For the analyzers used in this test program, this variability is generally
of a fixed size in terms of concentration, resulting in a relative error that increases as the
concentration being measured decreases. This fact is important to consider when using
measurements to produce relative difference models.
Results such as NMOG and PM mass involve more complex measurement processes, making
sources of error harder to quantify. NMOG is a calculated result, comprised of measurements of
many different aggregated and individual species made using different types of instruments. PM
mass is measured by technicians who must carefully handle and weigh filters, presenting
opportunities for other types of errors and biases.
6.1.1 NOX
Southwest Research Institute reported that the site NOX analyzer used for this program was
expected to have an inherent noise level of about ±10 ppb based on earlier experiments involving
repeat measurement of very low concentrations. Two separate but related issues exist to varying
degrees within the NO* dataset. One is the issue of very low sample measurements for some
vehicles, resulting in sufficient inherent noise to cause a large relative measurement error.
Another is the overlapping ranges of the sample and background measurements for these same
vehicles, such that their net results may be smaller than the measurement variability. If this
program were simply trying to quantify the magnitude of NOX emissions from such vehicles, this
level of error may be acceptable. However, since we are looking for meaningful differences in
emissions between fuels, this large relative error is particularly problematic.
Figure 53 shows bag 1 background and sample NO* concentrations, with the medians for the two
lowest-emitting vehicles (Focus, Sienna) shown in blue text below their points. A measurement
114
-------
variability of ±10 ppb represents about 10% of the sample medians for these two vehicles.
Moreover, performing a subtraction of background would double this margin, for a relative error
in the net result of more than 20% for these two vehicles, because the measurement variability
applies to both the sample and background measurements. This figure is of the same magnitude
as the fuel effect the program was designed to detect. For the other vehicles the relative error is
generally less than 5%.
Figure 53. Bag 1 Sample and Background NO* Concentrations by Vehicle (median values in blue).
ppm NOx Bag 1 (blue=sample, red=ambient)
10.0000
1.0000
0.1000
0.0100
0.0010
CCOB
CIMP
F150
HODY
JLIB
NALT
DCAL
HCIV
ffi FFOC
TCAM
..'. SOUT
Oft - TCOR
TT r"*1
TSIE
*>*" * o «0 O °0°" n" 0% o* oftO * %** d
ee
DO
B 00
Also noticeable for the two lowest-emitting vehicles is the substantial overlap of the range of
sample and background measurements, with differences in some cases being less than the
measurement variability. This condition would be expected to give a zero result as discussed in
Section 3.1. Meanwhile, for the other vehicles the range of sample measurements is separated by
a few hundred ppb or more, generally giving a net result that is discernible from measurement
variability.
As described in Section 5.5, vehicles having strong influence on model fitting were identified.
For bag 1 NOX, results show that a highly influential vehicle, the Focus, also has sample
measurements in the range of background levels. This suggests it likely has higher measurement
noise than data from the other vehicles, and thus many measurements may not be reliably
distinguishable from background levels. Since each vehicle is subjected to the same number of
tests on the same fuel set, under the same test conditions, a vehicle can reasonably be considered
an integral sampling unit. Thus, the decision was made to remove the measurements on the
115
-------
Focus from subsequent Bag-1 modeling analysis. The data for the Sienna, though similar in their
range of sample and background measurements, were not found to be exceptionally influential to
model-fitting and therefore were not removed from the dataset.
Figure 54 shows background and sample NO* concentrations for Bags 2 with medians for the
two lowest-emitting vehicles (Cobalt, Focus) shown below their data in blue text. The
measurement variability of ±10 ppb represents more than 20% of the median sample
measurement for the Cobalt, and more than 10% for the Focus. In addition, subtraction of
background results in the majority of Cobalt net results being smaller than the variability.
As described above, the influence analysis identified the Cobalt as very strongly influential in
model fitting (Table 20). This graphical analysis suggests that the Cobalt measurements are, like
those for the Focus in Bag 1, affected by a high level of measurement uncertainty, and may not
be reliably distinguished from background levels. Thus, based on judgment parallel to that
applied for bag 1, the subset of measurements for the Cobalt were removed from the dataset used
for subsequent modeling analyses.
Figure 55 shows Bag 3 NO* data. In this case, emission levels for most vehicles are lower than
in Bag 2, and show even more overlap with background measurements. This dataset as a whole
is expected to contain a large amount of measurement error, and models produced from it would
be of questionable value.
116
-------
Figure 54. Bag 2 Sample and Background NO* Concentrations by Vehicle.
10.0000 -
1.0000
0.1000
0.0100
0.0010
ppm NOx Bag 2 (blue=sample, red=ambient)
FEXP
CIMP pier,
DCAL HODY
CSIL HCIV JLIB
CCOB J.7 j8B ^$ .£5 \? & !v A:
fc. ;" ' *'* *« ir. VjJ *a js, *?
&£ °* ° ^ °*» * °° ^ *«v °^ »°* ^
** v ':; \ c; :£ v« * .;» fc
?* V # f.° ' " -J * 5h '
%. j'- '. % : °V '5
O O O 0 O O
O O 0 00
00000 W 000
0 000
0.038 0.072
NALT
*
**:
a
Jboo
0°
o
SOUT TCAM
."... . TCOR
W> t*A
& " ^«
' ' ' : S5
» l^lJB
* o» e °
| .> $
. r ^ j.
o° oo^r
r -
o o o
> 00
TSIE
If
1
Figure 55. Bag 3 Sample and Background NO* Concentrations by Vehicle.
ppm NOx Bag 3 (blue=sample, red=ambient)
0.0100
CIMP F150
'." DCAL '
CCOB H
A\ CSIL Lv p
/ *.\. 4t rft
g *j9 " "B* ^.Hh
gf Ij,* c^t *f*
1 ii! 1 ! :f
%l *«h° >.«> »o -V
V ?. ./» s * °1°
^»f 2!' * > -:
W § « Jj|
0° Qj ^ » 0 *" -."f
°° 00% 800 S
0 00 00" 0 0»
OP 0««
0 00 0,1
0 00
«0
03 00
0 0
TCOR
TCAM
TSIE
/ ':.
iTI
dj*» f °
°i* fW" ^
v* L*» f. °
»«° *0 oJ
>? g;» 4*
^ J3n ^
o^p° ^ aV
° oo xofr °o°
o | * . .V
0 * 0
OO 0 0
D 00
0 00
0 0
117
-------
6.1.2 Particulate Matter
Unlike most gaseous pollutants, measurement of paniculate matter (PM) mass involves handling
filter media, including pre- and post-weight measurements done, in the case of this program, by a
human operator using a microbalance in a clean room. Prior to a test, new filters are acclimated
for a period of at least 12 hours in the clean room, and each is marked with a serial number then
weighed three times to produce an average pre-test weight. The filters are placed into clean
containers, transported to the test cell, and inserted into the sampling apparatus. After the test is
conducted, the filters are removed, placed back into the clean containers and returned to the
clean room. The acclimation and weighing procedures are repeated as before to produce an
average post-test weight. The difference (in micrograms) is referred to as the filter weight gain,
which is then combined with dilution and sample flow data to produce a PM emission rate for
the test period (by bag). In this program, dilution air was HEPA-filtered and presumed to be free
of PM, so there was no background filter sample collected for later subtraction as is typical with
other emissions. Thus, the net PM result is generated from the clean and dirty weights of a
single sample filter.
Particulate emission rates from vehicles are highly sensitive to driver behavior, and thus
significant variability is expected in the measured parameter itself. One source of variability in
the measurement process is unintended gain or loss of material during filter handling. Material
gain may occur from accidental exposure to ambient dust or stray particles released from the
sampling apparatus, while loss may occur from exposure to elevated temperatures or air currents.
Two measurement process parameters, filter temperature and sample flow velocity at the filter
face, were captured during each test as part of data quality assurance. Other measurement
artifacts may occur due to insufficient neutralization of static charge on the filter, as well as
errors in accounting for temperature, humidity, and barometric pressure during filter storage and
weighing. The process of averaging triplicate weights attempts to mitigate some of these.
Discussion with EPA staff experienced with PM measurement suggests that for the data as
collected in this program, where typical certification procedures and other best practices were
generally followed, a variability of approximately ±1 ug should be assumed for all filter
weights.11 Considering that the net PM result is calculated by subtracting two filter weights (dirty
minus clean), it should be understood to have a variability range of ±2 ug, as the measurement
error applies to both weights. Therefore, a net weight gain of 10 ug would have a relative error
of 20% associated with it, a figure of the same order of magnitude as the fuel effects we are
attempting to capture.
m These data are available in the "Output-QA" sheet of the individual test files.
n See, for example, Figure 2 in Chase, R., Duszkiewicz, G., Lewis, D., and Podsiadlik, D., "Reducing PM
Measurement Variability by Controlling Static Charge," SAE Technical Paper 2005-01-0193, 2005,
doi: 10.4271/2005-01-0193.
118
-------
Filter weight gain data for bag 1 spans a range of 1 to 100 ug as depicted in Figure 56. The
majority of results are below 10 ug and thus contain substantial error relative to the fuel effects
being investigated. Across the dataset, most vehicles' PM results fall into a range of one order of
magnitude over a range of 2.0-20 ug (with the possible exception of the Liberty). Looking at the
figure, no vehicles stand out as having a large number of very low measurements. Additionally,
no particularly influential vehicles were found in the influence analysis (Table 23) . Thus, while
there were concerns that the model may contain artifacts due to the uncertainty in the data, no
vehicle-specific subsets of data were more suspect than others. Therefore, all bag 1 PM data
were retained for subsequent model fitting.
Figure 56. Bag 1 Particulate Mass Filter Weight Gain by Vehicle.
PM bag 1 filter weight gain (}ig)
1000.0
100.0
10.0
1.0
0.1
JLIB
TCOR
CSIL °CAL HCIV HODY f NALT
F150 FEXP FFOC ,' «"j
' £
SOUT
CCOB CIMP
»
-------
for a few seconds. This operation mode is known to have greatly increased emission rates of
PM, which may account for the relatively high PM masses shown here for a portion of the test
that is fully warmed-up and would otherwise have the vehicle's emission controls working at
high efficiency.
Figure 57. Bag 2 Particulate Mass Filter Weight Gain by Vehicle.
PM bag 2 filter weight gain (\ig)
1000.0
100.0
10.0
1.0
0.1
HODY
CIMP
CSIL
JLIB
V
TCOR
SOUT
CCOB
DCAL
FEXP
FFOC HCIV
F150
jg &
I
*
ft *.
v.
&
r
*.
NALT
TCAM
TSIE
"'
10
15
20
25
Filter weight gain data for bag 3 is shown in Figure 58, with lower median magnitudes and
spanning a smaller range than in bags 1-2. Nearly all the data here are below 10 ug, with the
majority of masses below 5 ug. PM emissions in bag 3 are small because the hot restart has
much less over-fueling than the cold start seen in bag 1, and the driving schedule is much milder
here than in bag 2.
Influence analysis was again unable to discern any subsets of the data that were likely to contain
more measurement noise than others. In this case, measurement variability is considered large
relative to the fuel effect being investigated and modeling of the data should be done with
caution. Indeed, model fits presented in Section 7 show that magnitude and sign of coefficients
are highly variable depending on which terms are included in the model, behavior that is
consistent with a dataset containing substantial uncertainty.
120
-------
Figure 58. Bag 3 Particulate Mass Filter Weight Gain by Vehicle.
PM bag 3 filter weight gain
1000.0
100.0
10.0
1.0
0.1
CCOB CSIL
LLUb CIMP
DCAL
F150 FEXP
HODY NALT
TCOR
TCAM . TSIE
. i
.
v> v * "
s
v »'
10
15
20
25
Tunnel blanks were performed periodically throughout the data collection period as a screen for
significant contamination or other measurement issues. Results are shown in Figure 59. This
procedure is done by setting up and running the dilution and sampling systems as if an emission
test were occurring, but without collecting any vehicle exhaust (the intake is typically capped).
The results of this procedure show background levels of all the pollutants at the time the blank is
being run, combined with any capture or release of material from internal surfaces of the
sampling equipment. Additionally, for pollutants like PM and speciated compounds, the results
include any contamination or variability introduced during handling and analysis of sample
media. Generally speaking, tunnel blanks are of limited utility in assessing the contribution of
material released from sampling system surfaces during an actual test, since the blank test
procedure does not produce the same range of temperature and humidity conditions within the
system.
121
-------
Figure 59. Particulate Mass Tunnel Blank Filter Loading for Bags 1-2.
5
-------
emission level of the individual species is not independent from the THC measurement (of which
they are a subset).
The speciated carbonyl and alcohol results were subjected to limits of quantitation (LOQs) by
Southwest Research Institute before being used in the NMOG calculations. These LOQs were
set by tracking the behavior of media blanks (the cartridges used to make the measurements).
Background and sample results were determined to be different from zero only if the blank-
corrected measurement was greater than three times the standard deviation of a running five-day
set of media blanks. The final result was then reported as nonzero if the sample exceeded
background.0 This process was developed to accommodate the fact that the concentration of
species of interest in dilute exhaust was often of similar magnitude to levels present as low-level
contamination in the measurement media. However, this additional rigor resulted in a relatively
conservative standard for quantification, such that nonzero mass emissions should contain
relatively low measurement error.
The THC results are produced by FID analyzers in the test cell, and have measurement errors on
the order of 10 ppb. In the range of data being measured (background levels around 2 ppm,
samples in Bag 1 >10 ppm and in Bag 2 around 3 ppm), this represents less than 1%
measurement error. Figure 60, Figure 61, and Figure 62 show sample and ambient THC
measurements for bags 1, 2, and 3, respectively. Figure 63 and Figure 64 show net
measurements as a percentage of sample for bags 2 and 3, as an attempt to understand the
magnitude of the nets relative to the measurement error where nets are small. Several vehicles
show a portion of tests with net concentrations less than 10% of sample measurement, which
would put them in the same order of magnitude as the estimated error of measurement. Two
vehicles (Odyssey, Sienna) have the majority of points below those of other vehicles.
The methane results are similar in magnitude and relationship to background as THC, though
methane generally shows less cold start effect than total hydrocarbons because it is not a fuel
component and therefore is not directly emitted during initial start-up. Measurement error for
methane is expected to be similar to that for THC. Plots of ambient and sample for bags 1-3 are
shown in Figure 65, Figure 66, and Figure 67, followed by examination of nets as percentage of
sample for bags 2 and 3 in Figure 68 and Figure 69. In this case, six vehicles have most or all of
their nets falling under 10% of sample concentrations. For the two vehicles mentioned above
(Odyssey, Sienna), measurement error may again play a significant role in the variability of the
net measurement.
While a number of vehicles have measurement error concerns, influence analysis for Bag 2
NMOG identifies the Odyssey and Sienna as highly influential vehicles in terms of overall fit,
effect on fixed-model coefficients and precision of fixed-model coefficients (Table 22, page
105). Subsequent review of the measurements as shown in Figure 63 and Figure 68 again
0 Details on this LOQ method are described in Appendix L of the EPAct/V2/E-89 testing report.
123
-------
suggest that high measurement variability impairs our ability to discern tailpipe emissions from
background levels. Thus, data from these two vehicles were removed from subsequent bag 2
NMOG models.
124
-------
Figure 60. Bag 1 Total Hydrocarbon Sample and Ambient Concentrations by Vehicle (ppmC).
100.0000
10.0000
1.0000
ppmTHC Bag 1 (blue=sample, red=ambient)
F150
DCAL .
NALT SOUT JCAM
CCOB
-
-
FFOC
,
.-. :? :.-. ~
« fc *
Figure 61. Bag 2 Total Hydrocarbon Sample and Ambient Concentrations by Vehicle (ppmC).
10.0000
ppm THC Bag 2 (blue=sample, red=ambient)
1.0000
CIMP
F150
fl
DCAL
CSIL
TCOR TSIE
FEXP NALT SOUT
CCOB
.,
HCIV HODY JUB :< I* TCAM
* O«
125
-------
Figure 62. Bag 3 Total Hydrocarbon Sample and Ambient Concentrations by Vehicle (ppmC).
10.0000
ppm THC Bag 3 (blue=sample, red=ambient)
1.0000
F150
CCOB c|Mp
JLIB NALT
SOUT
TCAM
TCOR TSIE
80.0
70.0
50.0
50.0
40.0
30.0
20.0
10.0
0.0
Figure 63. Bag 2 Total Hydrocarbon as Percent of Sample Measurement.
Bag 2 net THC as percent of sample
F150
CIMP
CSIL
DCAL
r,
NALT SOUT
FEXP
FFOC
JLIB
CCOB
HCIV
FT
.*«
HODY «fj
$
r f'J TCAM
TCOR TSIE
$
A't
-hf-
126
-------
50.0
Figure 64. Bag 3 Total Hydrocarbon as Percent of Sample Measurement.
Bag 3 net THC as percent of sample
70.0
F150
CCOB
niwiD «V JLIB
C MP r^n A
DCAL J?
40.0
30.0
20.0
V* " *" FEXP
** «. ** c hhUL Sc
c* £ <^ ^ i_, £
f rs
FFOC
NALT SOUT
r? * ."I* §^ ^% ^v
i$ ?** :?. >r -J
HGV HOOV T J :., -M TCOR TS,E
10.0 *^ytf-A
^-rn
0.0
127
-------
Figure 65. Bag 1 Methane Sample and Ambient Concentrations by Vehicle (ppm).
ppm CH4 Bag
10 0000
1 nnnn
F150
DCAL
CCOB
CIMP
CSIL «
« L **
£* iX-
I* '. v» V
- A i< & ?:,
lib **i t£* -* v
. ! ^P » '
fit »tjj
'*"
*
* i i ii i
1 (blue=sample, red=ambient)
JLIB
FFOC
"S
ft
V NALT
X t)
,'' HODY " v
*& . HCIV ll'
ftM £?
«* !
?: / tf g
<2t jifi V j
%! ^V S
* * y^t
* 0
* x° rf **
&! %& arf $£e J^°
w W *B 35B 9ft M|
e *W ^W(P
TCOR
TSIE
SOUT TCAM
(
. . .5
/ J f *
:.V 'if .' */j
MS -Q& .- *».
*V» V* .A .
VI §J
w.
4! * S i
Figure 66. Bag 2 Methane Sample and Ambient Concentrations by Vehicle (ppm).
10.0000
ppm CH4 Bag 2 (blue=sample, red=ambient)
1.0000
CIMP
F150
DCAL
CSIL . FEXP
CCOB
FFOC
HCIV
HODY
«1»
H
Bl
TCOR
NALT SOUT
TSIE
TCAM
!
i
128
-------
10.0000
Figure 67. Bag 3 Methane Sample and Ambient Concentrations by Vehicle (ppm).
ppm CH4 Bag 3 (blue=sample, red=ambient)
F150
DCAL
CIMP CSIL FEXP
JLIB
FFOC
CCOB .
NALT SOUT
HODY
TCOR
' TSIE
TCAM
1.0000
Figure 68. Bag 2 Methane as Percent of Sample Measurement.
50.0
50.0
40.0
30.0
20.0
10.0
0.0
CH4 Bag 2 net as percent of sample
F150
CIMP
DCAL
JLIB
CSIL
FEXP
NALT
SOUT
CCOB
/*'. i
*""' *
TCAM
FFOC
TSIE
HODY
TCOR
«ft
129
-------
Figure 69. Bag 3 Methane as Percent of Sample Measurement.
CH4 Bag 3 net as percent of sample
10.0
0.0
F150
CIMP
CCOB CSIL DCAL \*
;" " .^. ]£: FEXP
*J. %-*r *
.r *| -x :* :*
.; s» s -* >*
* *. *S - " Vi
r- -* -* - *i
SM * * y"*~
"*" *
FFOC
;"\ HCIV HODY
.4 : "*
r? ^
'-' 1 $
JLIB
.;.
-X
£?
. *
? " NALT TCAM
" . SOUT
$L ' TSIE
» 4i '
** ^ .' TCOR
-"' ?^ % ^1
W""
$&
6.2 Analyzer Drift
During investigation of influential vehicles in the NOX dataset, concentration data were examined
for both sample and background measurements. In addition to the very low emission levels seen
for two vehicles, it was also noted that a large proportion of background concentrations for
several vehicles were reported as zero. Further investigation revealed that these vehicles were
those added to the test fleet later in the program, such that their testing had taken place after
August 2009. The timeline of addition of vehicles and fuels to the test scheduling is shown in
Table 29.p Since it appeared this issue may have been related to testing procedures or conditions,
the analyzer zero check results (available in the QA sheet of the individual test files) were plotted
chronologically along with the background measurements.q These data are shown in Figure 70
p The order in which vehicle-fuel combinations were tested in this program was randomized to minimize
effects of any systematic bias on the subsequent analyses. However, due to funding limitations early in
the program, not all vehicles began testing at the same time. As funding was received, vehicles were
added to the randomization schedule.
q Before each test the NOx analyzer performs an automated calibration process whereby the "zero" and
"span" levels are set by flowing gas streams of known concentrations through the analyzer. Zero and
span checks are subsequently made after each bag is analyzed to verify that the calibration held.
130
-------
and Figure 71. This presentation shows that the zero point on the NO* analyzer had been
experiencing an offset of seemingly random magnitude during each test beginning in August
2009 (affecting both sample and background). The offset values were distributed over a range of
about 300 ppb, mostly negative for bag 1 and mostly positive for bags 2-3. As there was no
discernible pattern of offset behavior among bags within a test, or tests within a day, it wasn't
clear that the zero check measurement taken at the end of each bag was representative of the
measurement process immediately preceding it. Thus, no drift corrections could be applied to
the dataset. More details of this analysis are available in Appendix F.
131
-------
Table 29. Timeline of Addition of Fuels and Vehicles to the Test Scheduling.
Phase 3
Week
Weekl
Week 2
WeekS
Week 4
Week5
Week 6
Week?
WeekS
Week 9
Week 10
Week 12
Week 25
Week 37
Week 55
Week 60
Fuels
Added
2, 7, 8, 9 and 15
None
None
1, 12, 13
None
22,24
None
3,4,5, 11, 14, 16,20,21,23,30
None
None
6, 10,25,26,27,28,31
None
None
29 (E85)
Vehicles3
Added
CCOB, TCAM, FEXP, DCAL,
HODY
CSIL, TSIE, DLIB, HCIV, NALT
None
None
None
None
None
None
None
None
None
FFOC, SOUT
CIMP, F150, TCOR
DCAR
End of Phase 3 testing
Vehicle/Fuel
Assignments
EPA
Randomized for
rest of program,
except for E85
Last fuel tested
a - Vehicle designations are explained in Section 2.
Figure 70. Bag 1 NO* Background and Zero Check by Date.
0.40
0.30
0.20
0.10
0.00
-0.10
-0.20
-0.30
Phase 3 EPAct - Bag 1 NOx ambient (blue) vs. zero check (red)
<»o<> 9«»B P «ooxooo o" OCOIB>O>OO«DO<> . go. o§mo°° oo
Mar-09
»»?
" j
o_ . _ o
o oo o o o a
Jun-09
Sep-09
Dec-09
Mar-10
Jun-10
132
-------
Figure 71. Bag 2 NO* Background and Zero Check by Date.
0.40
0.30
0.20
0.10
0.00
-0.10
-0.20
PhaseBEPAct- Bag 2 N Ox ambient (blue) vs. zero check (red)
i°
O CD IB O 0 O OMB)
00 00 C
OOO OQD OO O
O O CD OCD O O OO O
O OO OOOO OOO
00 O O O
OOO O
I OOOO O O
Mar-09
Jun-09
Sep-09
Dec-09
Mar-10
Jun-10
The root cause of the offset was never fully determined, but after some troubleshooting
with SwRI staff, the most likely explanation was intermittent malfunction or failure of a
component in the analyzer. Immediately following the completion of testing, and before the
discovery of this zero offset issue, the analyzer underwent scheduled maintenance that included
replacement and recalibration of several components around the reaction cell and related
measurement electronics. Thus, follow-up testing to further characterize the malfunction
behavior was not possible.
As a result, each NOX result has a random noise component included, which is more
significant on a relative basis for the lowest-emitting vehicles, adding to the measurement
concerns discussed in Section 6.11.
133
-------
7 Final Modeling
7.1 Design Efficiency for Extended Models.
The optimized design for the fuel matrix and its development was previously described above
(Section 2.1, page 19). As mentioned, the design was optimized for a model including eleven
terms, with a resulting G-efficiency of 51.6%. This level of efficiency is considered to be at the
low end of the range considered adequate for efficient estimation of effects17.
For purposes of analysis, additional parameters were allowed into the full model, to explore the
possibility that important effects might not have been included in the design model. All effects
considered for inclusion are listed above in Table 17, here reproduced as Table 30.
As an additional step, we retrospectively evaluated the design efficiency of the extended model
(design terms plus additional terms). In addition to the 11-term design model, we evaluated the
efficiency for two additional models, one of which includes 17 terms, and a second that includes
16 terms, after omitting the T90xT90 term. In Table 30, these models are denoted as the "11-
term," "17-term" and "16-term" models, as mentioned in 5.land 5.3, respectively. The G-
efficiencies of the three models are shown in Table 31. Results show that inclusion of the
additional six or five terms reduces the effective G-efficiency to values between 20 and 23%.
In terms of design, this outcome implies that had some or all of the additional terms been
included in the design model, the fuel matrix would have required additional fuels to achieve an
adequate G-efficiency. In terms of analysis, this result does not necessarily or absolutely
preclude the additional terms as candidates for inclusion. However, it does suggest that caution
is required in the fitting of the additional terms, as the additional terms may be estimated less
precisely than the design terms. As an initial precaution, it is valuable to review whether any of
the additional terms are highly correlated with each other, or with any of the design terms,
expanding the analysis shown earlier for the design terms (Table 6 - Table 9).
A correlation matrix for standardized model terms, incorporating one-stage standardization for
linear terms and two-stage standardization for quadratic and interaction terms is shown in Table
32. The table shows that no high correlations exist among the additional terms, or between
additional terms and design terms, with two exceptions. Thus, with appropriate cautions and
qualifications, it is possible to consider candidate terms from among the additional terms for
inclusion in reduced models.
It appears that fitting models based on the extended full model may yield important insights.
However, given the fact that design efficiency for the extended full model is low, it is not certain
that reduced models based on the extended model will be retained for use in prediction or
description, following additional review and interpretation.
134
-------
Table 30. Description and notation for parameters included in model fitting
Term
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Fuel Parameter
Ethanol content (%)
Aromatics content (%)
RVP (psi)
T50(°F)
T90 (°F)
Model term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Arom x T50
Arom x T90
T90 x T90
T50 x T90
Arom x RVP
RVP x T90
In
optimized
design?
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
NO
NO
NO
NO
NO
NO
11 -term
17-term
16-term
Table 31. Design Efficiencies for Sets of Candidate Model terms.
Model
11 -term
17-term
16-term
G-efficiency (%)
51.6
20.6
22.3
135
-------
Table 32.
matrix.
Correlation matrix for Standardized linear-effect (one-stage) and interaction (two-stage) terms in the full-design fuel-parameter
Ethanol
Afom
RVP
T50
T90
Banz
Okftns
atOHxetOH
T50:-,T50
T90>.T90
T50*T90
atOH--.T50
Arom?.T?C
atOH>.T90
RVP>.T90
Arcm.*-T90
stOft.Arom
etOH.-.RV?
Atcin.'.RVP
EEhancl
i.oo
Aram
-0.04
:.oo
RVP
-0.15
0,05
1.00
T50
-O.S7
-o.:o
-0.26
1.00
T90
-o.o;
-0.02
0.13
-0.02
1.00
Banz
0.75
-0.07
-0.15
-0,60
0.00
:.oo
Olafms
O.OS
-0.54
-0.36
-0.16
-O.OS
0.39
:.oo
atOHxatOH
-0.03
-0.1.
-0.01
-O.OS
-0.03
-0.18
-0.04
1.00
T50:«.T50
-0.01
-0.10
0.16
0.12
0.05
-0.13
-0.2S
-0 ' ^
1.00
T90xT90
- .18
- .23
- .07
.26
\~>
o^
i r
Of,-!
. J .!
-0.13
_.oo
T50«T90
-0.02
-0.31
0.06
0.04
O.-O
-0.17
0.02
0.07
0.07
0.02
1.00
stOH:-.T50
-O.OS
0.15
-0,06
-0.01
-0.03
0. -0
0.21
-0.6S
-0.44
0.18
-0.05
1.00
Aiomr.lSO
0.12
0.00
-0.34
-O.OS
-0.30
0. 5
0. 4
0. 4
-0. 1
0.37
0.01
-0.06
1.00
atOH.-.T90
-0.02
0.02
0.09
-0.02
-0.07
0.07
0.04
-0.02
-0.02
-0.12
-O.S3
0.04
-0.04
1.00
RVP«T90
0.09
0.15
-0.02
0.06
-0.02
0 '
-0.18
0.02
0.21
0.10
-0.31
-0.10
-0.1S
-0.13
1.00
AromsTsfl
0.02
0.01
0.15
-0.30
-O.OS
0.05
-0.35
0.04
0.03
0.15
0.04
-0.06
0.03
-0.16
0.10
1.00
atOH:-.Arcm
-O.OS
0.03
-0,02
f| * T
0.02
-0.06
0.02
-O.OS
0.06
-0.3*
-0.04
-0.01
-0.59
0.01
0.14
-0.04
1.00
atOH:-.RVP
-0 0"
-0.02
0.02
-0.05
O.OS
0.04
0.10
-0.07
0.05
0.06
-0.16
-0.04
0.27
0.14
-0.02
0.13
-0.05
1.00
AromxRVP
-0.02
0.05
0.01
-0.34
0.15
0.23
0.30
-0.12
-0 4^
0.14
-0.20
0.40
-0.19
0.14
0.02
0.15
-0.20
-0.09
1.00
136
-------
7.2 Fitting Reduced Models
7.2.1 Guiding Assumptions
Incorporating results from analyses performed up to this point, we performed a final round of
model fitting. The goal of the process was to identify reasonable sets of reduced models, both to
describe and represent relevant processes, and for use in prediction. At the outset, we express a
preference for reduced over full models, for several reasons.
1. The candidate terms in the design model were selected because we anticipated that
they could be important for one or more of the measured emissions. However, we
did not anticipate that all fuel properties would be important for all the compounds
selected for measurement.
2. The descriptive or representational aspects of the models are very important.
Insofar as possible, it is highly desirable that the models selected be intelligible
and interpretable in terms of physical and chemical processes. It is also important
that the terms in the models be limited to effects that describe important processes
affecting emissions. Accordingly, we place emphasis on inclusion of parameters
that can be precisely estimated.
3. Consistent with (2), it is important to avoid overfitting of models. We prefer to
avoid inclusion of parameters that may represent study artifacts or random noise,
unique to this study that might not be replicated in other studies or in real-world
emissions.
For these reasons, the models reported in this section are as parsimonious as the data and subject-
matter knowledge allow. That is, they include the minimum number of terms needed to avoid a
decline in model fit to the Phase-3 data.
In addition, this round of fitting incorporated the outcomes of analyses previously described. We
took the following specific steps:
1. We excluded the T90xT90 term, as mentioned in 5.3 (page 96). Upon
consideration, we concluded that the fuel matrix was not adequate to estimate this
term. Only two fuels (30 and 31) were assigned the middle T90 value (325 °F)
needed to model the quadratic term. In addition, the remaining fuel properties for
these two fuels were not well balanced across the parameter space. Both fuels had
high aromatics and low T50 (< 165 °F). With respect to ethanol and RVP, one
fuel has middle and high values, and the other high and low values.
2. We dropped selected influential observations (5.2). As mentioned, this step
affected only PM (Bag 1).
3. Following influence analysis for vehicles (5.5), we dropped selected vehicles in
selected analyses (Section 6). We took this step based on conclusions that subsets
137
-------
of measurements for these vehicles were affected by measurement issues that
called their validity into question.
In the two subsections below, we report results based on both the 16-term full model as well as
the 1 1 -term design model.
7.2.2 Methods
In the final round of model fitting, we followed a series of steps:
1) Fit all possible models.
2) Identify a set of leading candidates
3) Construct a "superset" of terms from the set of leading candidates
4) Identify the best fit model, using the "superset" model as a starting point.
These steps were performed for the gaseous emissions: THC, NMOG, NMHC, CO, NOX and
CH4. As before, the modeling was performed separately for each of bags 1, 2 and 3. These steps
were not performed for subsets of data affected by "severe censoring," as defined in 5.3.2 (page
98). Below, we describe each of these steps.
Fit all possible models: We fit all possible models respecting hierarchy, based on a designated
full model, either the 1 1-term design model (Equation 16) or the 16-term extended model
(Equation 17) (See also Table 30, page 135). The models were fit using the SAS MIXED
Procedure, as previously described, and models were ranked on the basis of the Bayesian
Information Criterion (BIC).
/?6ZZee+/?7ZZ55+ Equation 16
&ZZ55+/]7ZZee
Equation 17
&ZZea + j89ZZer + j8wZZe5 + A ,ZZe9 +
&2ZZa5 +fll3ZZa9+fll4ZZ59+fll5ZZar+j316ZZr9 +
£
138
-------
Identify a subset of leading candidates: Based on inspection of a plot of BIC vs. the number of
terms in each model,/? (including the intercept), we identified a subset of the top 5 to 10
candidate reduced models.
Construct superset of terms: We constructed a set of terms that included all terms in any of the
models in the set of candidates identified in the previous step. We designated this set of terms as
the "superset" and used it as a de facto full model in subsequent modeling.
Identify a "bestfit" model: Starting with the superset, we performed model fitting by backwards
elimination. We used likelihood-ratio tests to evaluate goodness of fit, at a 90% confidence level
(a = 0.10). As before, we dropped terms or groups of terms if the/?-value for the test was greater
than 0.10.
The process will be illustrated using NOX (Bag 1) and CO (Bag 1). Following these examples,
the model-fitting process using Tobit regression will be illustrated using PM (Bag 1) as an
example.
7.2.2.1 NOX (Bag 1): Model fitting based on the 11-term Design Model
Inclusion of 11 terms in the "full" model gives a total of 211 = 2,048 possible models. However,
of these, a total of only 294 models (14.5%) respect hierarchy. Results for these models are
shown in Figure 72. The plot shows a measure of goodness-of-fit (BIC) versus the number of
terms in the model, including the intercept (p). This view shows the set of models forming two
major groups differing in terms of BIC. In this case, the model with minimum BIC is not clearly
visible at this scale.
Figure 73 shows a close-up view of the same results, focusing on the major group with lowest
BIC. This view shows that the major group is composed of an additional 3-4 subgroups. At this
scale the model with minimal BIC, having five terms (p = 5) is apparent.
To get a closer view of the subset of models with lowest BIC, Figure 74 shows a close-up
focusing on the lowest edge of the lowest main group of models. The "best" model is clearly
visible, with two models fairly close in second and third places. In this case, we selected the six
best-fitting candidates (BIC < 913) to construct the superset for subsequent model fitting. Table
38 shows the/?, BIC and specific terms included in the 35 best-fitting models out of 294.
Final model fitting began with the 6 terms included in the "superset model" shown in Table 34.
In this case, only one additional reduced model was fit. Two terms, with ^-values substantially
higher than 0.10, were selected for removal. The/>-value for the associated likelihood ratio test
was also well above the designated a level. Accordingly, the reduced model was retained as the
"best fit" model. Note that the model selected by the goodness of fit testing also has the lowest
BIC, as shown in Table 33.
139
-------
Figure 72. NO* (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 11 terms in the design model.
1030-
1020-
1010'
1000-
990-
980-
s 97°
960-
950-
940-
930-
920-
910-
(
! ! f
i * E
' S ' I i
§
o § I
i i
j
i !
i f
i i
i fe
i i
i
i i
1
1
2 3 4 5 6 7 8 9 10 11 1;
P
Figure 73. NO* (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy (CLOSE-UP of Figure 72).
940-
o
01
910-
t
(
> ;
<
3
<
<
>
!
<
> <
<
J i
C
I
C
c
i
3
j
s t
^ c
E
c
J
c
t :
c
)
1
£
I \
i i
j
' !
; f
£
;
i
!
1
;
!
/
\ \
3
>
' i
j
j (
\
i
j <
3
,
I S
^
i
5 1
) \
\ \
i
D 1
)
) c
>
\
1 1;
140
-------
Figure 74. NO* (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy (CLOSEUP of Figure 73).
u
915;
914;
912;
91 1:
i
o °
s
c
e
<
i
{
c
c
J
c
)
f
J t
c
> c
>
8
o
c
I
>
)
15678
P
141
-------
Table 33. NO* (Bag 1): Number of terms (p), Goodness-of-fit (BIC) and terms included in the 35 best-
fitting candidate models (out of a total of 294 possible models with hierarchy). (Terms included in
models ranked 1-6 comprise the "superset" for final model-fitting).
Rank
1
2
i
j
i
5
6
7
8
9
10
11
12
13
i ^
15
16
1 i'
18
19
20
21
22
23
24
25
26
27
23
29
30
31
32
33
34
35
P
5
6
4
6
5
5
7
6
6
6
6
-
/
5
5
j
7
5
6
~
8
8
6
7
6
6
6
6
7
8
/
/
8
7
7
BIC
911.00
911.51
911.73
912.05
912.35
912.35
912.44
913.01
913.13
913.34
913.52
913.70
913.90
913.92
913.94
914.04
914.11
914.48
914.53
914.56
914.59
914.61
914.62
914.63
914.73
914.76
914.93
914.97
915.02
915.19
915.43
915.45
915.47
915.57
Design Terms
O
"o
*
«
*
*
*
*
«
*
*
*
P
*
*
E
3.
*
*
*
*
>
iV
.
LA
.
i
0
O
X
O
"o
*
*
m
ir
X
₯'
m
m
m
m
m
m
m
m
m
E
o
<
X
O
~0
*
K
»
m
m
m
r
*
>
£/
X
O
o
U~':
X
O
o
*
«
*
*
w
*
m
*
*
*
m
X
O
o
142
-------
Table 34. Models fit for NO* (Bag 1): (all models include an intercept term).
Model Term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Ze
7-a
Zr
7-5
7-9
7.2ee
ZZ55
2Zea
Zj/JQf-
ZZe5
ZZeg
Model
Superset
SM21
X
X
denotes "Superset minus 2."
Table 35. NO* (Bag 1): Model Fitting History, starting with the 7-term Superset model.
Fit Parameters
Model
Superset
SM22
P
1
5
-21nL
890.288
892.527
BIC1
914.039
911.001
1 A lower value indicates a better fit.
2 Best fit with respect to the 11-term design model.
Test with respect to
Previous Model
Dev.
2.240
d
2
Pr>5c2
0.326
Table 36. NO* (Bag 1): Coefficients and Tests of Effect for the Superset and Reduced Models, with
respect to the 11-term design model.
Full Model (Superset)
Estimate
-2.8598
0.06830
0.1368
0.04678
0.00634
-0.02343
-0.01495
Std.Err.
0.2061
0.01688
0.01333
0.01688
0.01899
0.01302
0.01857
d.f.
14
879
879
879
879
879
879
f-value
-13.87
4.05
10.27
2.11
0.334
-1.80
-0.805
Pr>f
<0.0001
O.0001
<0.0001
0.0057
0.74
0.072
0.42
Reduced Model (SM2)
Estimate
-2.8594
0.06750
0.1339
0.04783
-0.02369
Std.Err.
0.2061
0.01568
0.01320
0.01619
0.01290
d.f.
14
879
879
879
879
lvalue
-13.87
4.30
10.15
2.95
-1.84
Pr>t
O.0001
<0.0001
O.0001
0.0032
0.067
143
-------
7.2.2.2 NOX (Bag 1): Model Fitting starting with the 16-term Extended Model
Inclusion of 16 terms in the "full" model gives a total of 216 = 65,536 possible models.
However, of these, a total of only 2,964 models (4.5%) respect hierarchy. Results for these
models are shown in Figure 75. The plot shows a measure of goodness-of-fit (BIC) versus the
number of terms in the model, including the intercept (p). The view shows the set of models
forming two major groups differing in terms of BIC, similar to those for the design model.
However, in this case, the model with minimum BIC, having eight terms (p = 8) is visible even
when the results are viewed at this scale.
Nonetheless, Figure 76 shows a close-up view of the same results, focusing on the major group
with lowest BIC. This view shows that the major group is composed of an additional two
subgroups. Of course, the model with minimum BIC is clearly visible.
To get a closer view of the subset of models with lowest BIC, Figure 77 shows a close-up
focusing on the lowest edge of the lowest main group of models. The "winning" model is
obvious, as the only model with BIC < 907, with no "close" ties. Nonetheless, we
conservatively selected the six best-fitting candidates to construct the superset for subsequent
model fitting. Table 37 shows the/?, BIC and specific terms included in the 35 best fitting
models out of 2,964.
Final model fitting began with the 9 terms included in the "superset model" shown in Table 38.
In this case, two reduced models were fit. In the first step, two terms with high ^-values (T50
and etOHxTSO) were selected for removal. The/>-value for the associated likelihood ratio test
was well above the designated a level. Accordingly, these two terms were dropped. In the
second step, four additional terms, including RVP, T90 and their interaction terms aromxT90
and RVPx T90 were tested, due to the high ^-values for both linear terms. For this test, the
result was highly significant, suggesting that this set of terms to contribute to model fit. The
insignificant linear terms were thus retained, as well as the significant interaction terms, to
maintain hierarchy. Based on these results, the first reduced model (SM2) was retained as the
"best fit" model. Note that the model selected by the goodness of fit testing also has the lowest
BIC, as shown in Table 39. Coefficients and Type-Ill tests of effect for the superset and reduced
models are shown in Table 40.
144
-------
Figure 75. NO* (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 16 terms in the extended model.
1 ron
ffi <-)t-r)
920
900
D F
c
c
! f
> <
. ;
C
i |
> \
I *
j i
c
j
' !
c
i £
1 1
! |
j i
> i
> (
; |
|
i 6
1 1
! |
i i
i
i
i i
! 1
1 !
i f
!
h
i
i i
] i
1
f
J c
] f
j f
i 1
i
i r
||
>
! i
>
||
J "
1 '
II
) (
>
1
I
I
i
\ <
12345
9 10 11 12 13 14 15 16 17
Figure 76. NO* (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 16 terms in the extended model (CLOSE-UP of Figure 75).
940
930
s 92°
910
900
(
> !
J i
8 °
a o °
e g § a g e
i
9 8 E
3
f
1
J g 0
0 °
o i
f
J 1
]
o e o g j
o B
(
r
j t
i
i
i f
j i
s
i
? t
' 0
' S
1 B
1
i
i
;
i
k
I
j
I
I
fer
o
i °
' O
a
§ o
0 g
(.
8 s
I
I
234567
9 10 11 12 13 14 15 16 17
145
-------
Figure 77. NO* (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 16 terms in the extended model (CLOSE-UP of Figure 76).
U
CD
91V
914
913
912
9ns
<
<
<
D
<
I
3
)
1
C
,
' C
c
c
t
j 1 1 1
§ !
1 1
3 j
e A
i i
3 a
1 |
5 S a
1 *
5 ° g 1
' ° a
c, 0 {
@ S i
5 (
°
j
!
j
1
i i
j E
i
1
[
!
!
!
i (
1 l
)
f
j
l i
1
c
(
1
(
1
) {
1
)
I
I
\
10
11
12
13
14
146
-------
Table 37. NO* (Bag 1): Number of terms (p), Goodness-of-fit (BIC) and terms included in the 35 best-
fitting candidate models (out of a total of 2,964 possible models with hierarchy). (Terms included in
models ranked 1-6 comprise the "superset" for final model-fitting).
Rank
i
2
T
3
4
5
6
i
8
9
10
ii
12
; »
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30'
31
32
33
34
35
P
8
8
/
6
5
g
!
-
9
9
6
10
9
8
9
9
8
1
!
11
9
8
8
6
9
f
f
8
9
10
8
/
9
6
9
BIC
906.51
907.37
907.57
907.94
908.17
908.22
908.27
908.33
908.34
908.35
908.48
908.60
908.62
908.69
908.86
908.89
908.90
908.98
909.06
909.08
909.13
909.26
909.35
909.46
909.46
909.49
909.59
909.67
909.69
909.74
909.80
909.82
909.82
909.88
Design Terms
O
o
*
m
m
*
*
*
*
»
«
*
<5
m
'
m
m
-
m
m
m
*
*
*
*
>
IT
*
»
*
»
ft
»
O-.
,
*
*
*
«
m
O
O
X.
y
O
m
ft
»
P
X
IT
£
JV
X
ft
,
ft
*
ft
ft
ft
ft
ft
ft
147
-------
Table 38. Models fit for NO* (Bag 1): (all models include an intercept term).
Model Term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Ze
Za
2r
25
Z9
ZZee
ZZ55
77
ZjZj QQ
ZZer
ZZe5
ZZe9
Model
Superset
SM21
X
X
SM6
X
X
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
77
t^^ar
ZZa5
ZZag
ZZ.5P
ZZrp
denotes "Superset minus 2, etc."
X
X
Table 39. NO* (Bag 1): Model fitting history, starting with the 9-term superset model.
Fit Parameters
Model
Superset
SM22
SM6
P
10
8
4
-21nL
876.813
880.118
901.210
BIC1
908.482
906.509
917.044
A lower value indicates a better fit.
2 Best fit with respect to the 16-term extended model.
Test with respect to
Previous Model
Dev.
3.305
21.09
d
2
4
Pr>5C2
0.192
0.0003
148
-------
Table 40. NO* (Bag 1): Coefficients and Tests of Effect for the Superset and Reduced Models, with
respect to the 16-term extended model.
Effect
Intercept
77
Z-,Z-,g,
ZZ
55
77
ZjZjgQ
Full Model (superset)
Estimate
-2.8603
0.04635
0.1313
-0.01084
0.01472
0.004922
-0.02774
-0.02040
Std.Err.
0.02063
0.01749
0.01350
0.01469
0.01935
0.01314
0.01292
0.01331
d.f.
14
879
879
879
879
879
879
879
lvalue
-13.87
2.65
9.73
-0.74
0.76
0.37
-2.15
-1.53
Pr>f
O.0001
0.0082
O.0001
0.46
0.45
0.71
0.032
0.13
Reduced Model (SM2)
Estimate
-2.8602
0.03718
0.1258
-0.01452
0.005211
-0.02699
Std.Err.
0.2064
0.01303
0.01316
0.01341
0.01316
0.01292
d.f.
14
879
879
879
879
879
/-value
-13.86
2.85
9.56
-1.08
0.40
-2.09
Pr>/
O.0001
0.0044
O.0001
0.28
0.69
0.037
-0.04640
0.02677
0.01415
0.01367
879
879
-3.28
1.96
0.0011
0.051
-0.04990
0.03160
0.01324
0.01334
879
879
-3.77
2.37
0.00018
0.018
7.2.2.3 CO (Bag 1): Model Fitting based on the 11-term Design Model
Inclusion of 11 terms in the "full" model gives a total of 211 = 2,048 possible models. However,
of these, a total of only 294 models (14.5%) respect hierarchy. As with NO*, Figure 78 shows a
measure of goodness-of-fit (BIC) versus the number of terms in the model, including the
intercept (p). This view shows four major groups of models differing in terms of BIC. In this
case, the model with minimum BIC is not clearly visible at this scale.
Figure 79 shows a close-up view of the same results, focusing on the major group with lowest
BIC. This view shows that the major group is composed of an additional four subgroups. At this
scale the model with minimal BIC, having five terms (p = 9) is apparent, although not strikingly
so.
To get a closer view of the subset of models with lowest BIC, Figure 80 shows a close-up
focusing on the lowest edge of the lowest main group of models. The "best" model is clearly
visible, with five additional models forming a second tier. In this case, we selected the six best-
fitting candidates (BIC < 324) to construct the superset for subsequent model fitting. Table 41
shows the/?, BIC and specific terms included in the 35 best fitting models out of 294.
149
-------
Final model fitting began with the 6 terms included in the "superset model." In this case, only
several reduced models were fit. In the first trial, a single term (etOH>-value for
the linear effect is well over the critical value and that for the interaction is close to the critical
value (p = 0.091). With the result of this test giving a significant value, both the linear and
interaction terms were retained, to maintain hierarchy. Similarly, a third trial (SM3b), tested the
removal of aromatics and its interaction, etOHxArom. As with the test for RVP, the test of fit
returned a highly significant value, with the result that both terms were also retained. A fourth
and final trial (SMS) tested the removal of both aromatics and RVP and their two interactions
with ethanol. As in the two previous trials, this test also returned a significant result (p =
0.00082). Accordingly, the initial model, "SMI" was selected as the "best fit." Note that the
model selected by the goodness of fit testing has the second lowest, rather than the lowest BIC,
as shown in Table 41. Were the BIC used as the sole criterion, "SM3a", rather than "SMI"
would have been selected as the "best fit."
Figure 78. CO (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 11 terms in the design model.
800;
700;
600;
u
ffl
500 :
400;
300:
3 i
c
c
] i
) fc
) t
i <
' i
' I
(
j <
l !
i i
8 |
! !
i 1
! i
(
] C
3
)
3 2 ° ° °
9 10 11 12
150
-------
Figure 79. CO (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 11 terms in the design model (CLOSEUP of Figure 78).
U
ffl
370-
360-
350:
340;
330-
3201
10 11 12
Figure 80. CO (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 11 terms in the design model (CLOSEUP of Figure 79).
328 f
327-
326-
U
CO
325-
324-
323-
322:
321 1
10
11
12
151
-------
Table 41. CO (Bag 1): Number of terms (p), Goodness-of-fit (BIC) and terms included in the 35 best-
fitting candidate models (out of a total of 294 possible models with hierarchy). (Terms included in
models ranked 1-6 comprise the "superset" for final model-fitting).
Rank
1
2
3
4
5
6
7
S
9
10
11
12
13
14
15
16
17
IS
-.9
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
P
9
11
7
10
S
10
8
9
12
9
8
11
10
9
9
10
10
11
3
7
8
7
f
6
6
6
5
9
8
9
9
f
S
10
7
BIC
321.10
322.84
323.36
323.42
324.59
324.60
324.99
325.08
325.14
325.26
325.63
326.76
327.06
327.70
333.26
334.32
334.38
334.50
334.54
334.68
334.76
334.S5
335.15
335.38
335,45
335,63
335.66
335.79
336.14
336.32
336.50
Design Terms
Q
o
_
<
r»/
8
E
»
S
*
1
o
^
w
o
?
E-
X
U"l
E
c
4
X
5C
Q
o
(V
X
Q
U
g
Q
CJ
*
S
X
Q
O
152
-------
Table 42. Models fit for CO (Bag 1): (all models include an intercept term).
Model Term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Ze
7-a
Zr
25
Z9
ZZee
ZZ55
2Zea
Zj/JQf
ZZe5
ZZeg
Model
Superset
SMI1
X
SM3a
X
X
SM3b2
X
X
SM53
X
X
X
X
denotes "Superset minus 1."
Not nested within SM3a; test with respect to SMI .
Not nested within SM3b or SM3a; test with respect to SMI .
Table 43. CO (Bag 1): Model Fitting History, starting with the 11-term Superset model.
Fit Parameters
Model
Superset
SMI2
SM3a
SM3b3
SM53
P
12
11
9
9
7
-21nL
287.074
287.271
291.311
294.809
298.471
BIC1
324.986
322.475
321.010
324.597
322.843
1 A lower value indicates a better fit.
2 Best fit with respect to the 1 1-term design model.
3 Test with respect to SMI .
Test with respect to
Previous Model
Dev.
0.1970
4.041
7.538
11.200
d
1
2
2
4
Pr>3C2
0.657
0.0444
0.0060
0.00082
153
-------
Table 44. CO (Bag 1): Coefficients and Tests of Effect for the Superset and Reduced Models, with
respect to the 11-term design model.
Effect
Intercept
Ze
77
z,z,g(
zz
55
77
ZjZjgQ
Full Model (Superset)
Estimate
1.3467
-0.1051
-0.01248
-0.0081
-0.03285
-0.1565
0.07290
0.05362
0.02074
0.01535
0.1062
0.003963
Std.Err.
0.1618
0.01305
0.009092
0.01038
0.01310
0.009095
0.01751
0.01311
0.008894
0.009073
0.01879
0.008928
d.f.
15
941
941
941
941
941
941
941
941
941
941
941
lvalue
8.32
-8.06
-1.37
-0.780
-2.51
-17.20
4.16
4.09
2.33
1.69
5.65
0.444
Pr>f
O.0001
O.OOOl
0.170
0.436
0.0123
O.OOOl
O.0001
O.OOOl
0.0199
0.0911
O.OOOl
0.657
Reduced Model (SMI)
Estimate
1.3466
-0.1049
-0.01242
-0.00762
-0.03273
-0.1571
0.07304
0.05358
0.02086
0.01596
0.1064
Std.Err.
0.1619
0.01304
0.009092
0.01033
0.01310
0.008992
0.01750
0.01311
0.008891
0.008967
0.01878
d.f.
15
941
941
941
941
941
941
941
941
941
941
941
f-value
8.32
-8.05
-1.37
-0.737
-2.50
-17.47
4.17
4.09
2.35
1.78
5.67
Pr>f
O.OOOl
O.OOOl
0.172
0.461
0.0126
O.OOOl
O.OOOl
O.OOOl
0.0192
0.0753
O.OOOl
7.2.2.4 CO (Bag 1): Model Fitting based on the 16-term Extended Model.
Inclusion of 16 terms in the "full" model gives a total of 216 = 65,536 possible models.
However, of these, a total of only 2,964 models (4.5%) respect hierarchy. Results for these
models are shown in Figure 81. The plot shows a measure of goodness-of-fit (BIC) versus the
number of terms in the model, including the intercept (p). This view shows four major groups of
models differing in terms of BIC, similar to those for the design model. As with the design
model, the model with minimum BIC is not clearly visible.
Figure 82 shows a close-up view of the same results, focusing on the major group with lowest
BIC. This view shows that the major group is composed of five additional subgroups, differing
greatly in size. Within groups there is a general trend of decline in fit with increasing numbers
of terms. However, the model with minimal BIC in each group tends to have more terms as the
BIC range for each group declines. At this scale the model with minimal BIC, having thirteen
terms (p = 13) is obvious, as occupying the low end of a trend composed of models having 13-17
terms. The extended model (p = 17) as also a member of this group. The BIC for the extended
model is substantially higher than that for the "best fit," but it is also clear that the full extended
model has a better fit than the majority of models with fewer terms.
To get a closer view of the subset of models with lowest BIC, Figure 83 shows a close-up
focusing on the lowest edge of the lowest main group of models. The "winning" model is clearly
visible (BIC=297), with two full units between the "best fit" and the 2nd ranked model
(BIC-299). We selected the five best-fitting candidates (BIC < 300) to construct the superset for
154
-------
subsequent model fitting. Table 45 shows the/?, BIC and specific terms included in the 35 best
top-ranking models.
Final model fitting began with all 16 terms included in the "superset model." Three reduced
models were fit. In the first trial, four terms were tested for removal based on high ^-values for
effect (0.24-0.83). As expected, the test of fit indicated that these terms did not contribute
significantly to fit, and they were accordingly dropped. In the initial model, however, three linear
terms, aromatics, RVP and T50 had high p-values. Thus, for thoroughness, a second trial tested
aromatics and four interaction terms. Not unexpectedly, the result of this test was highly
significant, and the aromatics linear term was retained on the strength of its interactions.
Similarly, the third trial tested the removal of T50, its quadratic term and two interactions, with
the result that the T50 linear term was also retained on the strength of its 2nd order terms. On this
basis, the initial model, "SM4" was selected as the "best fit," as shown in Table 47. Note that
the model selected by the goodness of fit testing also has the lowest BIC of all possible models,
as shown in Table 45. For the superset and "best fit" models, coefficients and tests of effect are
shown in Table 48.
Figure 81. CO (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 16 terms in the extended model.
U
CO
800
700
600
500
400
300
200
8 8
0 8 g §
o
o o o o
8 II II I I ! j !
12345
10 11 12 13 14 15 16 17
155
-------
Figure 82. CO (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 16 terms in the extended model (CLOSE-UP of Figure 81).
O
CO
370
360
350
340
330
320
310
300
290
o
! !
34567
9 10 11 12 13 14 15 16 17
Figure 83. CO (Bag 1): Bayesian Information Criterion (BIC) vs. Number of terms (p) for all models
respecting hierarchy, selected from the 16 terms in the extended model (CLOSE-UP of Figure 82).
310
309
ono
307
306
305
304
303
302
301
299
297
i
c
3
i
c
'
' c
1
O
0 ,
c
3
3
0
0
r
0 °
a
8
1
I
°
i
o
j
10
11
12
13
P
14
15 16 17
156
-------
Table 45. CO (Bag 1): Number of terms (p), Goodness-of-fit (BIC) and terms included in the 35 best-
fitting candidate models (out of a total of 2,964 possible models with hierarchy). (Terms included in
models ranked 1-5 comprise the "superset" for final model-fitting).
Rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
P
13
14
14
14
14
15
15
15
15
15
15
16
16
16
16
11
12
10
17
11
13
12
12
12
13
11
13
12
13
11
9
12
14
10
12
BIC
297.09
299.13
299.50
299.59
299.65
300.72
301.69
301.77
301.85
302.03
302.22
303.13
303.37
304.31
304.49
304.83
304.98
305.77
305.79
306.35
306.59
306.81
307.10
307.53
307.62
307.63
307.65
307.95
308.09
308.14
308.43
308.58
308.60
308.93
308.98
Design Terms
o
ttf
O
<
Q_
C±
8
i
8
o
td
X
o
ttf
O
LO
X
s
I
o
<
X
o
w
Q_
5
x
o
ttf
0
LO
I
X
O
ttf
0
O)
X
o
ttf
Extended Terms
Q_
5
X
o
<
o
LO
I
X
O
<
o
O5
I
X
O
<
o
O)
X
s
I
0
O)
X
Q_
C±
157
-------
Table 46. Models fit for CO (Bag 1): (all models include an intercept term).
Model Term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Ze
Za
Zr
Z5
Z9
77
ZjZj QQ
zzx
77
ZjZjea
77
ZjZjgr
ZZe.5
ZZep
Model
Superset
SM41
X
X
SM9
X
X
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
ZZar
zzas
ZZap
ZZ^p
ZZr9
X
X
X
X
X
SM82
X
X
X
X
Denotes "Superset minus 4, etc."
2 Formed by dropping aromatics and its interactions.
3 Formed by dropping T50 and its remaining 2nd order terms; Nested within SM4, not SM9.
Table 47. CO (Bag 1): Model Fitting History, starting with the 16-term Superset model.
Fit Parameters
Model
Superset
SM42
SM9
P
11
13
8
-21nL
254.340
256.473
297.508
BIC1
305.793
297.094
324.588
Test with respect to
Previous Model
Dev.
2.134
41.034
d
4
5
Pr>3C2
0.711
0.000000092
SM83
9
302.800
332.589
1 A lower value indicates a better fit.
2 Best fit with respect to the 16-term design model.
3 Test with respect to SM4.
46.327 4 0.0000000021
158
-------
Table 48. CO (Bag 1): Coefficients and Tests of Effect for the Superset and Reduced Models, with
respect to the 16-term extended model.
Effect
Intercept
ze
za
zr
Z5
Z9
ZZee
ZZ55
77
ZjZj g/
0.000001
0.000006
0.44
0.18
0.23
0.000000
0.000004
0.000002
0.000034
0.59
0.000000
0.24
0.00067
0.0017
0.000072
0.27
0.83
Reduced Model (SMI)
Estimate
1.3472
-0.06967
-0.01156
0.02063
0.02160
-0.1469
0.08535
0.07222
0.06244
0.1117
0.05370
0.05859
0.03861
0.3926
0.06981
Std. Err.
0.1620
0.01462
0.008946
0.01273
0.01652
0.009473
0.01725
0.01382
0.01410
0.01857
0.01476
0.01657
0.009502
d.f.
15
941
941
941
941
941
941
941
941
941
941
941
941
/ -value
8.32
-4.76
-1.29
1.62
1.31
-15.5
4.95
5.23
4.43
6.02
3.64
3.54
4.06
Pr>/
0.000001
0.000002
0.20
0.11
0.19
0.000000
0.000001
0.000000
0.000011
0.000000
0.00029
0.00042
0.000052
7.2.2.5 PM (Bag 1): Model fitting based on the 11-term Design Model (Tobit
Regression)
In the three tables below, we present the results of a Tobit model-fitting process, using Bag 1 PM
as an example. Each of the reduced models is identified by the number of terms having been
removed from the full model, i.e., "FM5" is read as "Fullminus 5," etc.
Table 49 shows the set of reduced models fit, starting with a full model including the 11 terms in
the design model. Five reduced models were fit, removing two to six terms, with each
successive model nested within its predecessor. For each, the terms removed from the previous
model are indicated by the "x" symbol.
Table 50 illustrates the model-fitting results for this process. The left-most block in the table
"Fit Parameters" shows the number of terms in each model p-1, excluding the intercept. It then
shows the -2 log likelihood statistics (-21nL) for each model.
159
-------
The rightmost block shows the results of goodness of fit tests for each model and its preceding
reduced model, i.e., FM6 is tested against FM5, and FM7 against FM6, etc. The first column
9 _
shows the "deviation," which is equal to the test statistic % test as shown in Equation 13, between
each reduced model and its predecessor. The second column shows the difference in the
numbers of terms between the two models d, and the third shows the p-value for the ^ test with
d degrees of freedom. A value greater than the assigned level (0.10) indicates an insignificant
result, i.e., we retain the null hypothesis that there is no significant difference in fit between the
two models. Stated differently, the additional terms in the full model do not give a corresponding
improvement in fit.
The tests are continued until either the test result is significant (p < 0.10), or the tests of effect for
the remaining terms are all significant (also/? < 0.10). In the test between FM5 and FM4, the
result is almost significant at the 0.10 level, but we nonetheless proceed to drop the etOHxT90
term based on our established criterion. In the final step, we test the removal of the RVP term
(FM6 vs. FM5); the result is insignificant (p = 0.625), suggesting that this term does not improve
the fit. Thus, we drop RVP and accept the resulting five-term model (FM6) as the "best fit." The
linear term for T50 is insignificant, but we retain it due to the significance of its quadratic term.
Table 51 shows coefficients, standard errors and tests of effect for the best-fit model, with results
for the full model included for reference. Note that the values of coefficients for terms in
common for both models are similar, but not identical. The standard errors, on the other hand,
are generally somewhat lower in the reduced model than for corresponding terms in the full
model, indicating that the terms are estimated with greater precision in the reduced model. The
improvement in precision corresponds to a reduction in the uncertainty (standard error) relative
to the mean (coefficient), which is also reflected in substantial increases in the test statistics and
corresponding reductions in the ^-values (higher significance). For example, the ethanol
coefficient in the full model is fairly large and its/?-value quite low. However, in the reduced
model, its standard error drops by 20%, and its/>-value decreases by an order of magnitude or
more.
160
-------
Table 49. Models fit for PM (Bag 1): (Grand Intercept not fit by Tobit models).
Model Term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Ze
Za
Zr
Z5
Z9
zzee
ZZ55
77
ZjZjgQ
77
ZjZjg;-
zze5
ZZep
Model
Full
FM21
X
X
FM3
X
FM4
X
FM5
X
FM6
X
1 Denotes "Full minus 2," etc.
Table 50. PM (Bag 1): Model fitting history, starting with the 11-term design model.
Fit Parameters
Model
Full Model
FM2
FM3
FM4
FM5
FM62
P-11
11
9
8
1
6
5
-21nL
2789.671
2790.118
2791.074
2793.414
2796.069
2796.308
Total number of terms excludes the intercept, as the Tobit
models do not fit intercepts.
Best fit with respect to the 11 -term design model.
Test with respect to
Previous Model
Dev.
0.447
0.956
2.34
2.655
0.239
d
2
1
1
1
1
Pr>5C2
0.800
0.328
0.126
0.103
0.625
161
-------
Table 51. PM (Bag 1): Coefficients and Tests of Effect for the Full and Reduced Models, with respect to
the 11-term design model.
Effect
Intercept1
ze
Za
Zr
Z5
Z9
7.7.
ZZ55
77
^^ ea
77
z,z, er
zze5
ZZeg
2 !
°"veh
^e
Full Model
Estimate
0.1365
0.3840
-0.0227
0.0338
0.2965
-0.0401
0.0700
0.0508
0.0295
-0.0482
0.0503
1.0321
Std. Err.
0.05030
0.03510
0.04000
0.05050
0.03510
0.06750
0.05050
0.03430
0.03500
0.07230
0.03440
d.f.
1
1
1
1
1
1
1
1
1
1
1
%2- value
7.35
119.96
0.32
0.45
71.48
0.35
1.92
2.19
0.71
0.44
2.14
Pr>I2
0.0067
<0001
0.57
0.50
<0001
0.55
0.166
0.139
0.40
0.51
0.14
Reduced Model (FM6)
Estimate
0.6559
0.1582
0.3833
0.0550
0.2923
0.0935
0.4251
1.0359
Std. Err.
0.04130
0.03480
0.04310
0.03440
0.03420
d.f.
1
1
1
1
1
% - value
14.7
121
1.63
72.2
7.46
Pr>x2
0.00010
<0001
0.20
<0001
0.0063
Not fit by Tobit model; calculated manually from individual vehicle intercepts.
7.2.2.6 PM (Bag 1): Model Fitting based on the 16-term Extended Model (Tobit
Regression)
Table 52 shows the set of reduced models fit, starting with a full model including the 17 terms in
the extended model. Seven reduced models were fit, removing three to nine terms, with each
successive model nested within its predecessor. For each, the terms removed from the previous
model are indicated by the "x" symbol. The model fitting history is shown in Table 53.
Coefficients and tests of effect are shown in Table 55. The final model (FM10) contains seven
terms, six from the design model, and one from the extended model (arom>
-------
Table 52. Models fit for PM (Bag 1): (Grand Intercept not fit by Tobit models).
Model Term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Ze
7-a
Zr
25
Z9
zzee
2Z55
77
ZjZjgQ
77
ZJZJer
ZZe5
ZZeg
Model
Full
FM31
X
X
FM5
X
FM6
FM7
FM8
FM9
X
FM10
X
T90 x T90
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
ZZpp
77
t^^ar
2Za5
2Zag
ZZ.5P
ZZrp
X
X
X
X
X
1 Indicates "Full minus 3," etc7
Table 53. PM (Bag 1): Model fitting history, starting with the 17-term extended model.
Fit Parameters
Model
Full Model
FM3
FM5
FM6
FM7
FM8
FM9
FM102
P-11
17
14
12
11
10
9
8
7
-21nL
2804.455
2804.778
2805.586
2806.743
2809.264
2811.736
2813.991
2814.388
Total number of terms excludes the intercept, as the Tobit
models do not fit intercepts.
2 Best fit with respect to the 17-term extended model.
Test with respect to
Previous Model
Dev.
0.323
0.808
1.157
2.521
2.472
2.255
0.397
d
3
2
1
1
1
1
1
Pr>5C2
0.956
0.668
0.282
0.112
0.116
0.133
0.529
163
-------
Table 54. PM (Bag 1): Coefficients and Tests of Effect for the Full and Reduced Models, with respect to
the 17-term extended model.
Effect
Intercept1
z.
1a
Zr
Z5
Z9
77
Z>Zj QQ
77 ,,
/,/,«
77
AZj ea
ZZ^
ZZe5
ZZe9
77
z>z> ar
77
LL as
ZZa9
ZZ}9
ZZr9
7 1
fveh
X2
0.016
<0001
0.2897
0.96
<0001
0.7653
0.17
0.24
0.76
0.55
0.033
0.41
0.67
0.17
0.45
0.22
Reduced Model (FM10)
Estimate
0.1807
0.3792
0.1004
0.3034
0.0806
0.0647
0.0665
1.0443
Std. Err.
0.0425
0.0351
0.0470
0.0353
0.0346
0.0349
0.0380
d.f.
1
1
1
1
1
1
1
%2- value
18.0
116.6
4.6
74.0
5.4
3.4
3.1
Pr>X2
<0001
<0001
0.0326
<0001
0.0198
0.0636
0.0804
Not fit by Tobit model.
7.2.3 Coefficients for Reduced Models
The tables below present sets of coefficients for reduced models. The tables present models
representing cold-start emissions (Bag 1), hot-running emissions (Bag 2) and hot-start emissions
(Bag 3). Parameters for each model are shown in Table 55, including numbers of observations,
numbers of vehicles included, and numbers of observations censored, missing or removed. In
addition, the model indicates the model type. Sets of model coefficients for THC, NMOG,
NMHC, CH4, CO, NOX and PM are presented in Table 56-Table 69. In reviewing the
coefficients it is important to remember that the values presented represent "standardized
coefficients," i.e., coefficients relating the change in the natural logarithm of emissions to
standardized fuel properties (see 2.3.1, page 28). Generally, the coefficient represents the change
in the logarithm of emissions associated with a change in the fuel property of one standard
deviation, calculated with respect to the fuel matrix used in this program.
164
-------
At this point it is important to note that the results reported below differ from those described
above in 5.3, as well as from those reported for the DOE analysis9. As mentioned, we excluded
the T90xT90 term, whereas the DOE analysis fit reduced models with and without it, and
reported that differences in candidate models resulted in some cases. With respect to influential
observations, the DOE analysis removed the same outlying measurements, based on the same
evaluation. However, the model fitting reported in the DOE analysis does not reflect the
influence analyses and subsequent evaluation reported in section 5.5 and Chapter 6.
Consequently, the DOE analyses retain all 15 vehicles for all analyses, whereas we have dropped
selected vehicles for specific models as described above. This difference in approaches does
lead to differences in some model fits. Finally, when ranking candidate models, the DOE
analysis applies the Mallow's Cp criterion, whereas we have applied the Bayesian Information
Criterion (BIC). Ranking by these two criteria is expected to give similar but not necessarily
identical results.
165
-------
Table 55. Design and Modeling Parameters for Final Model-fitting.
Compound
Bag
«obs
«veh
^censored
^missing
^removed
Model Type
CO
1
2
3
956
956
956
15
15
15
0
0
0
0
0
0
0
0
0
Mixed
Mixed
Mixed
NO,
1
2
3
893
893
931
14
14
15
0
0
25
0
0
0
0
0
0
Mixed
Mixed
Tobit
PM
1
2
3
908
906
873
15
15
15
45
47
82
2
2
0
1
1
1
Tobit
Tobit
Tobit
THC
1
2
3
956
832
954
15
13
15
0
0
2
0
0
0
0
0
0
Mixed
Mixed
Mixed
NMOG
1
2
3
956
832
837
15
13
15
0
0
119
0
0
0
0
0
0
Mixed
Mixed
Tobit
NMHC
1
2
3
956
832
828
15
13
15
0
0
128
0
0
0
0
0
0
Mixed
Mixed
Tobit
CH4
1
2
3
956
956
885
15
15
14
0
0
0
0
0
0
0
0
0
Mixed
Mixed
Mixed
166
-------
Table 56. THC: Reduced Models, based on the 11-term design model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
Ze
z.
Zr
zs
Z9
zzee
ZZ55
zzea
77
f^f^er
ZZe5
ZZep
0 veh
02s
Bagl
-0.8664
0.0548
0.0676
-0.0445
0.1288
0.0183
0.0436
0.0736
0.0179
0.0445
0.0214
0.1325
0.06872
Bag21
-4.6533
0.0327
-0.0195
-0.0355
0.0501
0.0514
0.0337
0.8384
0.06717
Bag 3
-4.2300
0.0079
-0.0612
-0.0142
0.0360
0.0490
0.0167
-0.0313
0.8860
0.09218
1 Fit excluding the Odyssey and Sienna.
Table 57. THC: Reduced Models, based on the 16-term extended Model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Notation
Intercept
Ze
za
Zr
Zs
zg
zzee
ZZ55
77
^^ea
77
f-if-ier
ZZeJ
ZjZjQf)
Bagl
-0.8658
0.06793
0.08344
-0.04669
0.1490
0.01434
0.04065
0.07555
0.02297
0.03426
0.04782
Bag21
-4.6543
0.03470
-0.01968
-0.02641
0.05122
0.06077
0.03392
0.02892
-0.02184
Bag 3
-4.2300
0.0079
-0.0612
-0.0142
0.0360
0.0490
0.0167
-0.0313
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
77
t^^ar
ZZas
ZZag
ZZjp
ZZrp
O veh
02s
0.02553
0.02260
0.05004
0.1317
0.06687
0.03112
0.8395
0.6646
0.8860
0.09218
1 Fit excluding the Odyssey and Sienna.
167
-------
Table 58. NMOG: Reduced Models, based on the 11-term design model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
Ze
z.
zr
Z5
Z9
zzee
ZZ55
77
ZjZjgQ
77
z,z,er
ZZe5
ZZeg
02veh
02s
Bagl
-0.95209
0.080186
0.087823
-0.04224
0.134524
0.044316
0.075786
0.016927
0.04653
Bag21
-5.2360
0.02673
0.03634
-0.04786
0.04915
0.07252
0.05349
0.02171
0.02586
0.8502
0.1310
Bag3
-5.S4492
0.0339
-0.0572
0.0783
0.1467
0.0707
-0.0728
1 Fit excluding the Odyssey and Sienna. See 6.1.3.
2 Not fit by the Tobit model; calculated manually from individual vehicle intercepts.
Table 59. NMOG: Reduced Models, based on the 16-term extended model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
Notation
Intercept
Ze
za
Zr
Z5
Z9
zz
ZZ55
77
t-'t-'ea
77
t^t^er
ZZgj
ZjZjeg
77
t^^ar
ZZaj
ZZa9
ZZjp
ZZrp
2
0 veh
02e
Bagl
-0.9513
0.0927
0.1051
-0.0483
0.1541
0.0112
0.0420
0.0787
0.0217
0.0357
0.0476
0.0272
0.0205
0.0544
0.1215
0.07335
Bag21
-5.2369
0.02947
0.03540
-0.03415
0.05338
0.08637
0.05652
0.04544
-0.02795
0.03080
0.04528
0.8507
0.1300
Bag33
-5.S4492
0.0339
-0.0572
0.0783
0.1467
0.0707
-0.0728
Fit excluding the Odyssey and Sienna. See 6.1.3.
Not lit by the Tobit model; calculated manually from individual vehicle intercepts.
3 Results identical to those for design model above.
168
-------
Table 60. NMHC: Reduced Models, based on the 11-term design model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
2etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
ze
Za
Zr
Z5
Z9
zzee
ZZ55
zzea
77
t^^er
zze5
ZZep
0 veh
02s
Bagl
-1.0315
0.03094
0.09461
-0.04568
0.13689
0.02160
0.04612
0.07534
0.02045
0.04729
0.02441
0.1266
0.07624
Bag21
-5.3253
0.03987
-0.05881
0.04548
0.08202
0.04774
0.9691
0.1708
Bag 3
-0.05810
-0.03130
0.1356
0.1546
0.07730
Fit excluding the Odyssey and Sienna. See 6.1.3.
Not fit by the Tobit model; calculated manually from individual vehicle intercepts.
Table 61. NMHC: Reduced Models, based on the 16-term extended model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Notation
Intercept
Ze
za
Zr
Z5
Z9
zzee
ZZ55
77
^^ea
77
f-if-ier
ZZeJ
ZjZjQf)
Bagl
-1.0308
0.04439
0.11119
-0.04765
0.15733
0.01682
0.04274
0.07769
0.02594
0.03579
0.05125
Bag21
-5.3256
0.01334
0.03743
-0.03909
0.05566
0.09799
0.05909
0.05087
-0.03153
0.02865
Bag33
-0.05810
-0.03130
0.1356
0.1546
0.07730
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
77
t-'t-'ar
ZZ<3.5
Z^ap
ZZjp
/j/jrg
0 veh
028
0.02820
0.02068
0.05225
0.1257
0.07430
0.05429
0.9723
0.1684
Fit excluding the Odyssey and Sienna. See 6.1.3.
Not lit by the Tobit model; calculated manually from individual vehicle intercepts.
Results identical to those for design model above.
169
-------
Table 62. CH4 : Reduced Models, based on the 11-term design model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
ze
Za
Zr
Z5
Z9
zzee
ZZ55
zzea
77
f^f^er
ZZe5
ZZep
0 veh
02s
Bagl
-3.0074
0.06994
-0.1053
-0.03275
0.07554
0.02844
0.05170
0.02088
0.01082
0.03048
0.2855
0.03014
Bag 2
-5.7075
0.05860
-0.09836
-0.02049
0.04394
0.02575
0.01227
0.008769
1.1108
0.02518
BagS1
-4.4742
0.02805
-0.09578
0.03025
0.01691
0.01528
-0.02079
0.4477
0.03439
1 Fit excluding the Corolla.
Table 63. Cm : Reduced Models, based on the 16-term extended model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Notation
Intercept
Ze
za
Zr
Zs
zg
zzee
ZZ55
77
t-'t-'ea
77
/-i/-*er
ZZej
/j/jeg
Bagl
-3.0068
0.08877
-0.09816
-0.02455
0.1059
0.008573
0.03133
0.05882
0.03977
0.02883
0.02655
Bag 2
-5.7076
0.06076
-0.09211
-0.02082
0.04477
0.02445
0.01398
0.01047
BagS1
-4.4742
0.02805
-0.09578
0.03025
0.01691
0.01528
-0.02079
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
77
t-'t-'ar
ZZa5
ZZap
ZZjp
/j/jrg
0 veh
02s
0.02791
0.02585
0.03072
0.02280
0.2853
0.02883
0.01374
-0.009921
1.1109
0.02486
0.4477
0.03439
Results identical to those for the design model above.
170
-------
Table 64. CO: Reduced Models, based on the 11-term design model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
Ze
z.
Zr
zs
Z9
zzee
ZZ55
zzea
77
f^f^er
ZZe5
ZZep
0 veh
02s
Bagl
1.3466
-0.1049
-0.01242
-0.00762
-0.03273
-0.1571
0.07304
0.05358
0.02086
0.01596
0.1064
0.3920
0.07214
Bag 2
-1.3893
0.0913
0.0299
0.0261
0.0440
1.9187
0.1256
Bag 3
-1.1409
-0.0815
0.0719
0.0239
0.0578
2.4412
0.1819
Table 65. CO: Reduced Models, based on the 16-term extended model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
Notation
Intercept
Ze
za
Zr
Zs
zg
zzee
ZZ55
77
t-'t-'ea
77
t-'t-'er
ZZej
/j/jeg
77
t-'t-'ar
ZZaj
2Zap
ZZjp
ZZrp
0 veh
02s
Bagl
1.3472
-0.06967
-0.01156
0.02063
0.02160
-0.1469
0.08535
0.07222
0.06244
0.1117
0.05370
0.05859
0.03861
0.3926
0.06981
Bag 2
-1.3895
0.09800
0.02839
0.02484
0.04177
0.02177
1.9196
0.1252
BagS1
-1.1409
-0.0815
0.0719
0.0239
0.0578
2.4412
0.1819
Results identical to those for design model above.
171
-------
Table 66. NOX: Reduced Models, based on the 11-term design model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
Ze
z.
Zr
zs
Z9
zzee
ZZ55
zzea
77
f^f^er
ZZe5
ZZep
0 veh
02s
Bag I1
-2.8594
0.06750
0.1339
0.04783
-0.02369
0.5925
0.1458
Bag22
-4.5692
0.06299
0.04407
0.4720
0.1836
Bag3
1 Fit excluding the Ford Focus.
2 Fit excluding the Chevrolet Cobalt.
Table 67. NO*: Reduced Models, based on the 16-term extended model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Notation
Intercept
Ze
za
zr
Z5
Zp
zzee
ZZ55
77
^^ea
77
ZjZjer
ZZe5
ZjZjQf)
Bag I1
-2.8602
0.03718
0.1258
-0.01452
0.005211
-0.02699
Bag22
-4.5692
0.06299
0.04407
Bag33
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
zzar
zza5
ZZag
ZZ^g
ZZrg
-0.04990
0.03160
0 veh
02s
0.5939
0.1437
0.4720
0.1836
1 Fit excluding the Ford Focus. 2 Fit excluding the Chevrolet Cobalt.
3 Results identical to those for design model above.
172
-------
Table 68. PM : Reduced Models, based on the 11-term design model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
ze
za
zr
zs
Z9
zzee
ZZ5,
zzea
77
t^^er
LLej
7.7.eg
0 veh
G\
Bagl
0.1582
0.3833
0.0550
0.2923
0.0935
Bag 2
0.1126
0.1662
0.1072
Bag 3
0.0173
0.0216
-0.1098
0.0167
0.1023
-0.1218
Table 69. PM : Reduced Models, based on the 16-term extended model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Arom x RVP
Arom x T50
Arom x T90
T50 x T90
RVP x T90
Notation
Intercept
Ze
zfl
Zr
Z5
zg
77
ZjZjee
ZZ5J
77
t-'t-'ea
77
t-'t-'er
ZZej
ZZep
77
t-it-iar
ZZaj
ZZa9
ZZjp
ZZrp
2
0 veh
02e
Bagl
0.65291
0.1807
0.3792
0.1004
0.3034
0.0806
0.0647
00665
Bag 2
-1.31211
0.1158
0.1988
-0.003700
0.09530
0.1059
Bag 3
0.0884
0.0667
0.1692
-0.0224
0.085
0.0637
0.0894
1 Not fit by the Tobit model; calculated manually from individual vehicle intercepts.
173
-------
7.3 Detailed Review and Interpretation
At this point, it is appropriate to give selected models and terms additional review and scrutiny,
with particular focus on interaction terms. We illustrate this involved and intensive process
through two examples: (1) NOX (Bag 1) and CO (Bag 1).
7.3.1 Example 1: NO* (Bag 1)
The reduced model based on the 11-term design model contains three linear terms and one
interaction (etOHxArom) (see Table 36, page 143). The corresponding model based on the 16-
term extended model contains two additional interactions, arom>
-------
stronger, and vice versa for the positive effect in (e) and the negative effect in (f). The visual
effect is that of a "mirror image" between (e) and (f), which does not apply between (a) and (b)
or (c) and (d).
Figure 84 also shows linear effects, in which the data is averaged across the target levels of only
one fuel parameter. Subplots (b) and (d) show the strong (and significant) aromatics effect,
whereas (c), (e) and (f) show the relatively small (and insignificant) effects for RVP and T90.
The graphic presentation, in conjunction with model fitting, makes it clear that RVP and T90 are
retained in the model solely on the strength of their interactions, for the maintenance of
hierarchy. For the etOHx Arom interaction, however, both underlying linear effects contribute to
the goodness-of-fit without their interaction, although including the interaction does improve the
fit.
Figure 84 shows interaction plots for an additional pair of parameters, etOH and T50. This term
does not qualify as an "interaction" as no interaction term is retained in reduced models, whether
fit with respect to the 16-term or 11-term full models. However, the position of the T50 term is
unclear in that it is dropped during model fitting based on the 16-term model, but retained in
model fitting based on the design model. In subplot (g) In NOX is portrayed with respect to
ethanol, with different series by T50 level. Due to the non-orthogonal relation between ethanol
and T50 in the design (Figure 1, page 27), the plot appears disjointed, with no complete series
across the entire range of ethanol. Also, in this presentation, three T50 levels are paired with only
one ethanol level (150°F, 10%; 165°F, 20%; and 240°F, 0%). The remaining two T50 levels,
190°F and 220°F, are both paired with 0% and 10% ethanol levels, with the series for 190°F
higher than that for 220°F, apparently suggesting a negative effect for T50. The picture is
complicated by the fact that the point for 240°F is higher than the 190°F series, and the point for
150°F is lower than the 220°F series. The complementary view in subplot (h) also shows
apparent "local" decreases with increasing T50 between 190°F and 220°F, both for 0% and 10%
ethanol. However, it also shows apparent local increases between 150°F and 190°F for 10%
ethanol, and between 220°F and 240°F for 0% ethanol. The overall apparent trend in means over
all T50 levels and across all ethanol levels is positive, which is reflected in the reduced model fit
with respect to the design model, but which is absent from the reduced model fit with respect to
the 16-term extended model.
The conclusions from the graphic presentation are confirmed by model fitting parameters shown
in Table 70-Table 72. These tables represent the results of model fitting, if the 5-term linear
effects model is treated as the full model. The results show that the fit is improved by removing
the RVP and T90 linear terms, which is not a surprising outcome in view of Figure 84.
Another way of viewing the interactions is to average and plot the residuals of the linear effects
model. Figure 85 shows the residuals averaged and plotted by target fuel properties. For all three
interactions (subplots (a)-(f)), the results reflect the interference interactions with trends that
cross each other, although the residual plots are "detrended", with the trends for individual factor
175
-------
levels crossing the 0.0 line on the^-axis. For the etOHxTSO pair, the view is similar to that in
Figure 84, with the residuals "detrended" around the 0.0 line, although the patterns of points
underlying the subtrends do not change. However, the "mean" lines reflect the results of the
linear effects model in that they are close to the 0.0 line, with the trend against ethanol (g)
showing smaller deviations than the trend against T50 (h). However, within the "zig-zag" effect,
the main points are apparently balanced around the no-trend line, consistent with retention of a
positive T50 term in the linear effects model.
In Figure 86, the residuals are averaged and plotted against the two-stage standardized
interaction terms (denoted by ZZ). In subplots (a) to (f), the existence of interactions is
suggested by the trends across the zero line. Further, the direction and sign of the trends reflect
the sign and magnitude of the interaction terms when they are included in the model (Table 36,
Table 40). For the etOHxTSO pair (subplots (g) and (h)), despite the irregular patterns of the
points when arranged in ethanol and T50 series, the residuals appear more or less balanced
across the no trend line, suggesting why the models do not fit an interaction for these two
parameters.
To further explore the effect of T50, it is possible to view a subset of data balanced on ethanol
and aromatics levels, which are the remaining two properties with clear effects on NO*. Figure
87 shows a subset of data for four fuels, averaged by T50 and vehicle. The fuels included are 1,
3, 4, and 6, which have target T50 levels of 150, 220, 220 and 190°F, respectively. These fuels
have ethanol and aromatics levels of 10% and 15%, respectively, but have varying levels of RVP
and T90 levels. The plot shows a separate trend for each vehicle, with apparently mixed results
but no apparent overall trend. Patterns do, however, appear to vary by vehicle. For example,
some vehicles, including the Liberty, Impala, Cobalt and Odyssey, show apparent negative
trends, while others, including the Explorer, F150, Outlook and Civic, show apparent positive
trends. One interpretation of these results could be that behavior with respect to T50 is vehicle
specific, depending on factors such as vehicle design and calibration. Another possibility is that
the effect of T50 in this presentation is masked by variation in RVP and T90 levels across these
fuels. In any case, this review of results for the selected fuels does not clarify the relation
between NOX emissions and T50 for Bag 1.
Viewed in physical terms, it seems plausible that bag 1 NO* as a function of T50 may be highly
sensitive to some particulars of vehicle design or calibration. Engine design could be a factor,
given that NOX is primarily a function of combustion temperature, and different vehicles could
have differences in design features such as the location of fuel injection in the intake port, rate of
heat-up of the manifold and cylinder block, spark timing algorithms (retarding=hotter exhaust
gases), etc., all of which could interact with the T50 parameter to produce differences in
combustion temperature during the first 30 seconds of operation before the catalyst is active.
176
-------
Figure 84. ln(NOx) (Bag 1): Two-way Conditional Effects Plots for Four Pairs of Terms, viewed with
respect to both Fuel Parameters : (a) EthanolxAromatics, (b) AromaticsxEthanol, (c) T90xAromatics,
(d) AromaticsxT90, (e) RVPxT90, (f) T90xRVP (g) etOHxTSO, (h) TSOxetOH.
OI
t .
it
^^=
~^~^_
^
^___- '
II
l»
^^ * Arom=15
--Arom=35
0.0 5.0 10.0 15.0 20.0 25
Ethanol(%)
£ -2.85 -
s
"
, >
^_ '
, r -~~~
=11
. ---~t
^ ^*^ * Arom-15
Arom-35
« LinearEffect
290.0 300.0 310.0 320.0 330.0 340.0 35C
7-90 (-F)
S "2'9° '
<
I
1
> p^^T' "
^ T90 300
6.0 7.0 8.0 9.0 10.0 11.0
RVP(lb)
C
S -2.90 4
^^
| ^^f** _i
*^^- '1:iiJ
(g)EtOHxTSO
,,, «
A 150=165
750=190
750=220
750=240
<*> Mean
(b)Ar
3m x EtO
T -<
]***"
T X
f
H
..-
^'X
/
-X
x
A
/
ear Effect -
20.0 25.0 30.0
Aromatics(%!
(d)Ar<
'
1
i
>m xT90
^~~^~
.,-'
' ^
x^
^^<-
.-'/
/
^
y
X
~^^^
l
» 190=300
m T90=
* Linea
'40
r Effect
20.0 25.0 30.0
Aromatics (%)
-2.65
-2.70
-2.75
-2.80
-2.85
-2.90
-2.95
-3.00
-3.05
-3.10
-3.15
(f)T90xRVP
» RVP=7
- RVP=10
*" Linear Effect"
320
T90(°F)
-3.05 -
/.
*
(h) T50
, ^
/ ..
/ ^
,..©..
*-
^
"~\^^
,
* etOH=o
xEtOH
etOH^lO
* etOH=20
O" mean
' ^
/
130.0 200.0 220.0
T50('F)
111
-------
Table 70. Models fit for NO* (Bag 1) (all models include an intercept term).
Model Term
etOH
Arom
RVP
T50
T90
Notation
Ze
7-a
Zr
7-5
7-9
Model
Linear
Effects
LM1
X
LM2
X
Table 71. NO* (Bag 1): Model Fitting History, starting with the 5-term linear effects model.
Fit Parameters
Model
Linear Effects
LM1
LM22
P
6
5
4
-21nZ
895.2
895.5
895.9
BIC1
916.4
913.9
911.7
1 A lower value indicates a better fit.
2 Best fit respect to the 5-term linear-effects model.
Test with respect to
Full Model
Dev.
0.202
0.444
d
1
1
Pr>5C2
0.653
0.505
1 The deviation is the difference in
the -21oglik statistics for the
nested and reference models,
respectively, per Equation 13.
Table 72. NO* (Bag 1): Coefficients and Tests of Effect for the Full and Best-Fit Models, with respect to
the 5-term Linear Effects model.
Effect
Intercept
za
7.5
7-9
Full Model
Estimate
-2.8616
0.06477
0.1322
-0.006509
0.04080
0.009373
0.5929
0.1462
Std.Err.
0.2062
0.01693
0.01301
0.01448
0.01738
0.01298
d.f.
14.00
879
879
879
879
879
f-value
-13.88
3.83
10.16
-0.45
2.35
0.72
Pr>f
0.0001
0.00014
0.0001
0.65
0.019
0.47
Best-Fit Model (LM2)
Estimate
-2.8616
0.06743
0.1322
0.04381
0.5931
0.1463
Std.Err.
0.2062
0.01567
0.01301
0.01573
d.f.
14.00
879
879
879
f-value
-13.88
4.30
10.16
2.78
Pr>f
0.0001
0.0001
0.0001
0.0055
178
-------
Figure 85. NO* (Bag 1): Mean Residuals for the Linear Effects Model, vs. Target Fuel Properties for
four pairs of terms: (a) Ethanol x Aromatics, (b) Aromatics ethanol, (c) Aromatics x T90, (d)
T90xAromatics, (e) RVP x T90, (f) T90 x RVP, (g) ethanol x T50, (h) T50 x etOH.
E 0.00 -
0
0.25
- 0.10
3
r .
(a) Etc
' -H
H x Aror
0
(c)Aro
i
5.0
1-=^
. *
HI
-»-Arom=35
E 0.00
(b)A
fr- * ~
romx Etl
^
i
3H
^^ ^^ *
10.0 15.0 20.0 25.0 10.0 15.0 20.0 25.0 30.0
Ethanol (%) Aromatics (%)
I
,
nxT90
mo i5.o
!5 0-05 -
S -0.10 -
4
(e)RV
6.0
B
P
_
^3»
*=s
>
~~ HI
--190=340
K
E 0 00
^ -0 10
4
(d)T9C
1 .
t -
ixArom
» -g=r:
-»-et
--et
-*-et
OH=10
OH=20
35.0 40
-1
1
-» A
--A
20.0 25.0 30.0 35.0 40.0 290.0 300.0 310.0 320.0 330.0
Aromatics (%) T30(°F)
F^^
xT90
7.0
I
>*«rr-
~~
(g)EtC
IHx
0.0
^ «S
. II
<>
--190=340
E 0 00 -
^ -0 10
4
1
l
/ft TOO v RVD
Z.
-z.
t
t
rom=35
340.0 35(
1
- -1
-»-F
--F
8.0 9.0 10.0 11.0 290 300 310 320 330
RVP (Ib) T90 (°F)
.
T50
'
\
5.0
1
150=190
150=220
O T50=240
..<^.. Mean
i? A AH
m^
^^-^
r^* * * *
^M*^
'(iiiL--
(h)TSOxEtOH
* etOH
...5.. Mea
=c
-i
i
Xs^
t
»
VP=10
340 3=
A \
jS"
^
10.0 15.0 20.0 140.0 160.0 180.0 200.0 220.0 240.0
Ethanol (%) T50(°F)
179
-------
Figure 86. NO* (Bag 1): Mean Residuals for the Linear-Effects Model, vs. Two-stage Standardized
Predictors: (a) Ethanol x Aromatics (b) Aromatics x Ethanol, (c) Aromatics x T90, (d) T90 x Aromatics,
(e) T90 x RVP, (f) RVP x T90, (g) ethanol x T50, (h) T50 x ethanol.
-0.25
(a)EtOHxArom
=^
f=
* Arom=15
-B-Arom=35
notrend
^ -
=»
-0.25 :
(i
)Aro
mx E
tOH
* -.
1
T
-^
-» etOH=0
==r~
etOH=
etOH=
notre
=10
nd
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
I
-0.25 -
lf\
w
1
T90 x Aro
k .
==~.
-^=
» Arom=15
Arom=35
notrend
1 *
*
-
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
TS °-05 ;
(«
K»
!)T90xRV
S=
P
^-^
«^c=
-
>
-« RVP=7
-»-RVP=10
no trend
-
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
- 0.10
Z ~0-05 '
-n T; -
1,
9
;}etOHxT
A
SO
1
1
^
1
> T50=150
i 150=165
^
1
1-T50-220
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5
ZZe5
1.0 1.5 2.0 2.5
V)
-0.25 ;
(t
1
) Arom x "
90
==s-
«
»- 790=300
--190=340
notrend
' *
~*
-
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
V)
(f)
*i
RVP x T9(
^^^
^^B
.
1
-»- 190=300
--190=340
no trend
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
2^
(B UllU "
M
n i^ -
(h
«L
)T50
s^
S
xetC
f
)H
,-
^ -
T
4
Tl
^ £
tOH=0
tOH=l
tOH=2
otrenc
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
zz,.
180
-------
Figure 87. InNO* vs. T50 for 4 selected fuels: 1 (T50=150), 6 (T50=190) and 3,4 (T50=220), by
Vehicle. Ethanol = 10%
140
150
vehicle
160
ALTIMA
COBALT
IMPALALS
SIENNA
170
1BO
CALIBER
COROLLACE
LIBERTY
SILVERADO
190
vnn
:] n
230
T50
CAMRY
EXPLORERXLT
ODYSSEY
CIVIC
F1SO
OUTLOOKXE
7.3.2 Example 2: CO (Bag 1)
The reduced model based on the design model retains two quadratic terms and three interaction
terms: etOHxArom, etOHxRVP and etOHxTSO (Table 44, page 154). The reduced model based
on the extended model includes three additional interactions, AromxRVP, AromxTSO and
AromxT90 (Table 48, page 159).
Review of the coefficients shows that all five interaction terms can be considered as
"interference" terms. For the etOHx Arom and AromxT90 terms this interpretation is
straightforward because the linear effects are negative in sign while the interaction is positive.
As another example, the etOHxTSO term also qualifies as an interference because while the sign
of the interaction itself is positive, one of the linear terms (ethanol) has a negative sign, while the
other (T50) has a positive sign, although they differ in magnitude. To review the interactions in
detail, we selected three terms for review, plus the quadratic term etOHxetOH.
We begin with interaction, or conditional-effects plots. As before, we calculated these plots by
averaging the data by target levels of the two fuel properties of interest, and across the levels of
181
-------
the remaining three fuel properties. These plots are similar to the ones viewed previously, with
some slight modifications. For effects involving ethanol, we omitted the 15% ethanol level and
for T90, we omitted the 325°F level. The selected levels of these two properties represent small
numbers of fuels, not well balanced across the remaining fuel properties. After omitting them,
the underlying patterns detected by the models are simpler to illustrate.
Interaction plots for the selected terms are shown in Figure 88. These plots display the
"interference effects" described by these terms. For example, in subplot (a) the trends in InCO
are displayed by aromatics levels. The trends are close to each other and appear close to parallel.
A strong interaction is not necessarily obvious in this presentation. Subplots (b) and (c) show
trends vs. aromatics level, by levels of ethanol and T90, respectively. In (b), as in (a), a strong
interaction is not visually apparent and the relatively strong ethanol effect is shown by the spaces
between the trends. In (c), however, the interaction between aromatics and T90 is strongly
suggested by the non-parallel slopes for the two T90 levels. In both (b) and (c) the almost level
linear effects also portray the weak aromatics linear effect shown by the models. In contrast, (d)
and (f) show a strong negative T90 effect, which is the single strongest effect in the model. It is
interesting to note that while (d) and (f) appear somewhat similar visually, the model considers
the arom>
-------
Finally, for the etOHxetOH quadratic term (e) the effect does not appear dramatic visually, but is
nonetheless highly significant.
In Figure 90, the residuals are averaged and plotted against the two-stage standardized
interaction terms (denoted by ZZ) used in model fitting. In subplots (a) and (c) the etOHx Arom
and AromxT90 interactions are clearly suggested by the positive trends across the no-trend line.
The remaining plots for the RVPxT90 interaction and the etOHxetOH quadratic term appear
similar visually, but as noted the former is not significant whereas the latter is. This result may
follow from the fact that only one trend for RVPXT90 differs markedly from the no-trend line
(T90=340).
183
-------
Figure 88. ln(CO) (Bag 1): Two-way Conditional Effects Plots for Three Interactions and one Quadratic
term, viewed with respect to Both fuel Parameters : (a) EthanolxAromatics, (b) AromaticsxEthanol,
(c) AromaticsxT90, (d) T90xAromatics, (e) RVPxT90, (f) T90xRVP, (g) etOHxetOH.
V
1 IAO ;
01 i 30 ;
5
§ 1.20 ;
o>
^^s,
(a)etOh
'^
fca^.
^"^Si
x Arom J
« Arom=15
B Arom=35
LinearEffect
0 5 10 15 20 2
Ethanol (%)
at
s 1'30 -
§ 1.20 -
O" ;
790=340
LinearEffect
»
-
(c) Arom x T90
.
'
£ 1.60
o>
E 1.50
| 1.30
§ 1.20
5 1.10
1.00
01
£
^ 1 30 -
EtOH=10 r
* EtOH = 20
.«.. Li
learEffect ^
(b) AromxetOH
»
' A
0 10 20 30 4
Aromatics (%)
i
1
ls^5>>s.
^Sj,
(d)T9ux Arom
t Arom=15
Arom=35
* LinearEffect
^
1
0 10 20 30 40 290 300 310 320 330 340 35
Aromatics (%) T90 (°F)
01
8 ':
O> i 30 _
5
o
i
i
«
(e)RVPxT90
» 790=300
1790=340
>
. LinearEffect
c
Oi
-------
Table 73. Linear-Effects Models fit for CO (Bag 1) (all models include an intercept term).
Model Term
etOH
Arom
RVP
T50
T90
Notation
Ze
Za
Zr
25
Z9
Model
Linear
Effects
LM1
X
LM2
X
LM3
X
Table 74. CO (Bag 1): Model Fitting History, starting with the 5-term linear effects model.
Fit Parameters
Model
Linear Effects
LM1
LM22
LM3
P
6
5
4
3
-21nL
323.94
325.65
327.34
340.25
BIC1
345.60
344.60
343.58
353.79
1 A lower value indicates a better fit.
2 Best fit respect to the 5-term linear-effects model.
Test with respect to
Full Model
Dev.
1.709
1.691
12.915
d
1
1
1
Pr>5C2
0.19
0.19
0.0003
1 The deviation is the difference in
the -21oglik statistics for the
nested and reference models,
respectively, per Equation 13.
Table 75. CO (Bag 1): Coefficients and Tests of Effect for the Full and Best-Fit Models, with respect to
the 5-term Linear Effects model.
Effect
Intercept
za
'"veh
Full Model
Estimate
1.3479
-0.1315
-0.01192
-0.01304
-0.04751
-0.1549
0.3906
0.07501
Std.Err.
0.1616
0.1167
0.008995
0.009967
0.01198
0.008970
d.f.
15
941
941
941
941
941
f-value
8.34
-11.27
-1.33
-1.31
-3.96
-17.26
Pr>f
<0.0001
0.0001
0.185
0.191
0.0001
0.0001
Best-Fit Model (LM2)
Estimate
1.3479
-0.1243
-0.03892
-0.1561
Std.Err.
0.1616
0.01079
0.01079
0.008913
d.f.
15
941
941
941
f-value
8.34
-11.52
-3.60
-17.51
Pr>f
0.0001
0.0001
0.0003
0.0001
185
-------
Figure 89. CO (Bag 1): Mean Residuals for the Linear Effects Model, vs. Target Fuel Properties for
three Interactions: (a) Ethanol x Aromatics, (b) Aromatics x T90, and (c) RVP x T90, and the quadratic
term etOHxetOH.
10 15
Ethanol (%}
RVP(lb)
fll "U'1U :
n ^ '-
+ 190=300
* <-
5^1^
(c)ArornxT90
. ' " ~.
;^s
10
30
40
Aromatics (%)
3 U.1D
QI "U-1IJ
(d)etO
-\ x etOH
i
,
t
0 5 10 15 20' 2
Ethanol (%)
Figure 90. CO (Bag 1): Mean Residuals for the Linear-Effects Model, vs. Two-stage Standardized
Predictors: (a) Ethanol x Aromatics interaction, (b) RVP x T90 Interaction (insignificant), (c) Arom x
T90 Interaction, and (d) quadratic term etOHxetOH.
^ 0.00
r^^~
(J
-»1
£=
)RVPxT
90
^9
i^»
^^H
-JL,
«
_|
-T90=
-T90=
-Refe
300
340
re nee
-2.2 -1.8 -1.4 -1.0 -0.6 -0.2 0.2 0.6 1.0 1.4 1.8 2.2
zzrt
-5 U'1U :
-
(c
-T90=340
Referenct
» «
) Arom x T90
^*
^^
^
»
i
Mean Residual
-2.2 -1.8 -1.4 -1.0 -0.6 -0.2 0.2 0.6 1.0 1.4 1.8 2.2
0 20 ^
0.00
-0 05
_n ^R
r^^^
(d
-*^
^^i
JetOHxetOh
^^m
1
^^
^^^H
H^^^m
X=
»
-2.2 -1.8 -1.4 -1.0 -0.6 -0.2 0.2 0.6 1.0 1.4 1.8 2.2
186
-------
8 Analyses for Speciated Hydrocarbon Compounds and Air Toxics
8.1 Measurements
During the Phase-3 project, hydrocarbons were speciated for specific subsets of vehicles and
fuels (See Table 5, page 20). The processes and methods for hydrocarbon speciation are
described in greater detail in the testing report5. A subset of these compounds selected for
statistical analysis include acetaldehyde, formaldehyde, acrolein, ethanol, benzene, 1,3-
butadiene, and ethane.
This chapter describes the development of models relating the emissions of selected species to
changes in fuel properties. The approaches and methods used are similar to those used for the
regulated pollutants, except as specifically described. One difference is that for the speciated
compounds, models were fit to Bag 1 and Bag 2 results only, representing "cold-start" and "hot-
running" processes. No models were fit for Bag 3 results.
8.2 Parameters and Design Efficiency
Due to limitations in budget, the entire study design was not applied to speciated hydrocarbons,
including those discussed in this chapter. For the speciated compounds, the volume of data
collected varies by Bag, compound and vehicle. For some compounds, measurements for Bag 1
were taken for all vehicles over the entire fuel set, thus encompassing the entire study as
designed, including replication. However, for the remaining compounds in Bag 1 and for all
compounds in Bags 2 and 3, measurements were taken for a smaller number of vehicles over a
reduced set of fuels, without replication. The combinations of fuels and vehicles included for
each compound analyzed are summarized in Table 76.
Note that the level of effort applied to Bag 1 measurements is variable. Four compounds were
measured under the "full design" i.e., 15 vehicles over 27 fuels, with replication. Two
compounds were measured under the "reduced design" i.e., 15 vehicles over 11 fuels, without
replication. A single compound was measured under the reduced design, i.e., on five vehicles
over 11 fuels.
Throughout this chapter, the set of 27 fuels included in the full design will be denoted as the "full
fuel-parameter matrix," as it includes all the fuel parameter points for which the design was
optimized. Similarly, the set of 11 fuels included in the "reduced design" will be denoted as the
"reduced fuel-parameter matrix," as it covers a set of fuel parameter points narrower than that for
which the design was originally optimized. The study fuels included in the full and reduced fuel
matrices are shown in Table 77.
187
-------
Table 76. Features of the Study Design Applied to Speciated Compounds Selected for Analysis.
Compound
Acetaldehyde
Formaldehyde
Acrolein
Ethanol
Benzene
1,3 -Butadiene
Ethane
Bagl
No. vehicles
15
15
15
15
15
15
5
No. Fuels
27
27
27
27
11
11
11
replication
YES
YES
YES
YES
NO
NO
NO
Bag 2
No. vehicles
5
5
5
5
5
5
5
No. Fuels
11
11
11
11
11
11
11
replication
NO
NO
NO
NO
NO
NO
NO
Table 77. Nominal Target parameters for fuels in the Phase-3 EPAct program.
Fuel1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
20
21
22
23
24
25
26
27
28
30
31
In Reduced Matrix
YES
NO3
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
etOH (%)
10
0
10
10
0
10
0
0
0
10
10
10
0
0
0
10
20
20
20
20
20
20
15
15
15
10
20
Aromatics (%)
15
15
15
15
35
15
15
15
35
35
35
35
35
15
35
35
15
35
15
15
15
35
35
15
35
35
35
RVP (psi)2
10
10
7
10
7
7
7
10
10
7
10
10
7
7
10
7
7
7
10
7
10
10
10
7
7
10
7
T50 (°F)
150
240
220
220
240
190
190
220
190
220
190
150
220
190
190
220
165
165
165
165
165
165
165
220
220
150
165
T90 (°F)
300
340
300
340
300
340
300
300
340
340
300
340
340
340
300
300
300
300
300
340
340
340
340
340
300
325
325
Note that numbering of fuels is not entirely sequential throughout.
This parameter was measured as "DVPE," but for simplicity, will be referred to as "RVP" in this document.
Fuel 4 was originally considered for inclusion in the reduced fuel set. However, it was excluded as the only fuel with an RVP
level of 1 0 psi. All other fuels in the reduced matrix have RVP values of 7 psi.
188
-------
The efficiency of the full design was evaluated during the design of the Phase-3 program. It was
evaluated for a model design that included eleven terms. The "G-efficiency" for the full design
was estimated at 51.6% for the eleven design parameters, as previously described in 2.1 above
(page 19).
However, in considering analysis of fuel effects under the reduced design, it was clear that
implementation of the reduced fuel set represents an effective design change, and that its design
efficiency would not necessarily be adequate for all the original 11 model terms. We therefore
reevaluated the efficiency of the reduced design, to identify a set of fuel parameters for which it
could estimate effects with reasonable efficiency. While application of results is beyond the
scope of this report, this de facto design change warrants attention.
The model terms considered for the full and reduced designs are summarized in Table 78, with
their respective design efficiencies. The reevaluation of the reduced design commenced with the
five linear effects, evaluated over 12 fuels, including the 11 identified in Table 76, plus Fuel 4
(design la). Because Fuel 4 had an RVP level of 10 psi, whereas the remaining 11 fuels had
values of 7 psi, a design with four linear effects, excluding RVP, was also evaluated, for sets of
12 and 11 fuels (designs Ib and Ic, respectively). As excluding Fuel 4 from the fuel set and
dropping RVP as a model term gave a marked increase in G-efficiency, design Ic was selected
as the full model for the reduced design. At this point, design efficiency for the inclusion of one
quadratic term and three interaction terms was evaluated (designs 2-5). As shown in the table,
the design efficiency drops sharply with inclusion of 2n order terms, to less than 5% for designs
four and five. Based on this analysis, we concluded that the reduced design would not give
adequate design efficiency for evaluation of the 2n order terms. Therefore, we selected design
Ic for modeling of fuel effects with the reduced fuel-parameter matrix.
Table 78. Model terms and associated G-Efficiency for the full design and selected reduced designs.
Design
Full
la
Ib
Ic1
2
3
4
Fuels
27
12
12
11
11
11
11
Model terms
ffi
i
i
1
o
H
*
*
o
H
*
*
S
tj
X
g
tj
*
0
H
X
o
H
*
g
<
*
g
tj
*
£
*
ffi
9
-------
5
11
3.0
Design Ic was selected for fuel effects modeling with the reduced fuel set.
8.3 Additional Terms for Modeling
A model including all possible candidate terms will be referred to as a "full" model.
Accordingly, the full model including eleven terms listed in Table 78 will be referred to as the
"11-term design model." In addition to the eleven design parameters for which the full
parameter matrix was optimized, an additional six parameters were considered for inclusion
during model fitting. A full model including these additional terms will be referred to as the "17-
term full model." Similarly, the model including only the four linear effects (excluding RVP)
will be referred to as the "4-term full model." Candidate terms considered for inclusion for the
full and reduced designs are shown in Table 76. For compounds measured under the full design,
model fitting started from the 17-term full models; for compounds measured under the reduced
design, model fitting started from 4-term full models.
8.4 Correlations Among fuel parameters
In modeling the toxic compounds, the fuel parameters were standardized as discussed above in
2.3.1 (page 28).
8.5 Measurement Issues
Measurement of individual speciated hydrocarbons, including the toxics discussed in this
document, involved measurement of small to very small quantities, particularly for the Bag 2
measurements. For measurements at the lower end of the range, questions exist as to whether the
concentration of compounds measured at the tailpipe, i.e., "sample" measurements, exceeded
measurable concentrations in ambient air, i.e., "background" measurements.
In addition, for some compounds, there was an additional issue concerning whether the
measurement media used to collect mass from the sample measurements emitted measurable
amounts of the compounds to be measured. Compounds affected by this issue include the
carbonyls, acetaldehyde, formaldehyde and acrolein. These issues are discussed in greater detail
in Appendix L to the testing report5.
In the analysis of these compounds, the issue of censoring applied, as for the regulated emissions
(see 5.1 (page 89), 5.3 (page 96) and 7.2.2.5 (page 159)). For some speciated compounds in Bag
2, censoring was severe enough to preclude the development of models.
Nonetheless, after some consideration, we elected to combine three approaches to censoring.
The first is substitution (for minor censoring). The second is another classic approach to left-
censoring (Tobit regression) which we applied in cases of more severe censoring. In addition, for
the compounds affected by media contamination, we applied a modeling approach that allows
190
-------
estimation of uncertain low-level measurements while acknowledging measurement uncertainty
(for all measurements) and factoring it into the model-fitting process. This approach, the
"estimated dependent variable model" (EDV) will be described below.
At the outset, to estimate measured values, and to identify censored values, we followed the
process described here:
First, we assume that the true, but unknown, tailpipe measurements (7,) are confounded by media
(k) and background (b) contamination, but can be represented as
Y = 7 + h.+ k, Equation 18
where the 7. are the apparent "contaminated" sample values. But because both k and b have
been measured, we can attempt reasonable estimates of the true values 7., as
Yt = Yt - bt.- kt Equation 19
These equations describe the concept. The implementation was somewhat more complicated, as
described below.
It is necessary to emphasize that the measurements of background and media concentrations do
not correspond with the sample measurements on the one-to-one basis. Both were measured on a
basis corresponding to compound and day, rather than to fuel and vehicle (on specific days).
Thus, the background measurements were obtained using sample media for each compound on
a daily basis, and can be thus associated with sample measurements (by vehicle and fuel) on a
chronological basis. Similarly, the estimates of media contamination were taken as 5-day
moving averages of daily measurements from "media blanks" over the previous five days (ks),
as described in Appendix L to the testing report, and depicted in Equation 20.
- k ,+&. d+k. , 7 ,
k5 = -^ '=* ^ ^ ^- Equation 20
In addition, some measurements were collected on two media in series, to account for the
possibility that the first medium (7i) could become saturated before the end of the test, in which
case the second medium (₯2) would be present to collect the "overflow" mass from the first.
Both sample (7^) and background (b,) measurements were adjusted for media contamination in
compilation of the dataset. For a process with two measurement media in series, this process can
be represented as
191
-------
j5=fc-*5)-(*i-*5) + fa-k5)-(b2-k5) Equation 21
which simplifies as
Yi=Yl-bl+Y2-b2
_ _ Equation 22
= Y,+Y2-b,-b2
In addition to estimating the sample measurements, as shown above, we estimated two variances,
the first being the variance of the 5-day moving average of the media blanks
(of), and the second being a variance of random errors (a2£).
With respect to the media-blank variance, we make three assumptions: (1) the variance of media
blanks can vary over time, (2) media variance is not correlated with the fuel properties, and (3)
media variance is not correlated with the random error. With respect to the random error, we
also make three assumptions: (1) the variability of the random errors does not vary with time, (2)
random error is not correlated with the fuel properties, and (3) random error is not serially auto-
correlated.
With these assumptions in place, we estimated the variance of the mean five-day media
contamination for each measurement day / as
o
k -k }
" i~i 5' Equation 23
5-1
where y ranges from 1 to 5, representing the media blank measurements over the previous five
days. The application of these variances will be described below.
Equation 22 above applies to all compounds, whether or not they were affected by media
contamination, as all measurements were potentially affected by background interference. After
application of Equation 22, if the estimated value of 7. was < 0, the measurement was considered
censored and assigned a missing value. Prior to model development, sets of measurements
estimated as described above were plotted, with and without aggregation, as described in the
following section. Table 79 shows the numbers of censored measurements by compound and
Bag. Note that the numbers of total measurements are consistent with the numbers of fuels and
vehicles shown in Table 76.
192
-------
Table 79. Numbers of individual measurements, by compound and Bag, indicating the total number of
measurements (ntotai) and the number of censored measurements (ficensored), as calculated using
Equation 22.
Compound
Acetaldehyde
Formaldehyde
Acrolein
Ethanol
Benzene
1,3 -butadiene
Ethane
Bagl
"total
913
913
913
913
176
62
62
"censored
0
0
23
193
0
0
0
Bag 2
"total
63
63
63
63
62
62
62
"censored
i
i
34
24
41
42
1
8.6 Review of Data
8.6.1 Linear Effects
The data collected in this study are difficult to visualize, in that they encompass variation of
emissions in the five-dimensional fuel-parameter space. Due to human limitations, it is practical
to view the data in only two dimensions at a time, in some cases including multiple series to
represent levels in a third dimension. Despite the risk of misinterpreting graphic portrayals that
may oversimplify the actual emissions behavior in all dimensions, it is valuable to review the
data visually before model development.
At the outset, it is helpful to get an overview of the raw results, sorted by vehicle and fuel, which
gives an initial impression of variability among vehicles and fuels, as well as within vehicles.
This view also gives an initial impression of vehicles or observations that may prove influential.
Figure 91 (a) shows the set of observations for Bag 1 Acetaldehyde, with the data portrayed as
common logarithm of the measurements (base 10). Across all fuels, the range of variability for
most vehicles spans just over one order of magnitude. The set of measurements for one vehicle,
the Focus, are low relative to the other vehicles. For another vehicle, the Outlook, the main body
of measurements is in the same range as the other vehicles. However, for this vehicle, a subset of
measurements is exceptionally high, relative to those from other vehicles. Figure 91(b) shows a
similar depiction of the acrolein results for Bag 1, which depicts and highlights the censored
measurements for this compound, which are represented by a uniform low value, to make them
visible on the logarithmic plot. For acrolein, the variability both among and within vehicles is
greater than for acetaldehyde, with the Focus and Outlook also having the lowest and highest
measurements, respectively.
193
-------
In addition, we averaged and plotted the data to check for evidence of linear effects, i.e., patterns
of emissions across all levels of a single fuel parameter. We constructed these views by
averaging the data by the levels of one fuel parameter and by vehicle, across all levels of the
remaining four parameters, repeating the process for each fuel parameter in turn. We took this
step for the emission results themselves (i.e., in "linear space"), as well as for natural-log
transforms of the data (i.e., "In space"). We made a point of examining the In-transformed
results, as the statistical models were developed using the transforms, rather than the raw results.
The study design anticipates the possibility that the response of emissions to changes in multiple
fuel parameters may involve several 2-way interactions, which suggests that limiting our
examination to "Linear Effects" may be simplistic. To examine 2-way emissions responses, we
also averaged and plotted the data by two fuel parameters simultaneously to examine potential
"conditional" or interaction effects, to examine how the effect of each fuel parameter varied with
the levels of the other parameters.
Below, we illustrate these concepts using results for Acetaldehyde (Bag 1). For this compound,
the ethanolxTSO interaction gives an excellent example of two interrelated variables and the
importance of supplementing "linear-effects" plots with "interaction" plots.
Figure 92 shows "linear-effects" plots for ethanol. By "linear effect" we mean the effect of one
factor (ethanol) across all levels of all other factors. In this case we have averaged the data by the
four ethanol levels and by vehicle. A strong ethanol effect is visible, which is not surprising for
this compound, considering the structural affinity between acetaldehyde and ethanol. The
variability among vehicles, also not surprisingly, is fairly wide, spanning about a factor of three
at each ethanol level. Thus, absolute variability increases with ethanol level, but relative
variability remains fairly stable. This pattern is confirmed by the trends in the transformed
results. In logarithmic space, the trends for individual vehicles track closely, whereas in linear
space they show a characteristic "fan" shape. These results suggest that the effect of ethanol on
acetaldehyde emissions can be expressed multiplicatively, and that the multiplicative factor is
similar across the selection of vehicles measured. The gradual down-sloping of the logarithmic
trends between 10% and 20% ethanol also suggests that fitting a quadratic term for ethanol
would be appropriate.
The linear effects plot for acetaldehyde in relation to T50 (Figure 93) illustrates why viewing the
results of a multidimensional experiment in a single dimension can be misleading. The trend
shows an increase in emissions from 153 °F to 165 °F, followed by a strong decrease at higher
T50 levels. Closer examination and modeling show that this pattern is an example of "Simpson's
Paradox," in which averaging emissions across all levels of the secondary variable, ethanol in
this case, gives an apparent inversion of the actual trend(s). In this case, the apparent inversion is
caused by the relationship between ethanol and T50 levels in the fuel matrix (Figure 1, page 27),
compounded by the fact that the effect of ethanol is far stronger than the effect of T50.
194
-------
These interrelationships are illustrated in the interaction plots, which show the data averaged by
levels of two fuel parameters, but not by vehicle. The first two plots (Figure 94) show
acetaldehyde vs. ethanol, by T50 level, in linear and logarithmic space. These views show a
strong ethanol effect (in the trends) and a smaller T50 effect (spaces between trends). In
addition, the shapes of the trends are very similar to those in the previous linear-effects plots.
However, the reverse does not hold in the remaining two plots (Figure 95), which show
acetaldehyde and In(acetaldehyde) vs. T50 by ethanol level. These two plots show a very
different picture from their corresponding linear-effects views. Specifically, these views show a
slight increase in emissions with increasing T50 at each ethanol level, with the strong ethanol
effect shown as wide gaps between the trends. It is also apparent why averaging across ethanol
levels gives an apparent declining trend, as mentioned above. Review of these data show that it
may not be possible to interpret the effect of one fuel parameter without accounting for the levels
of one or more additional parameters.
Similar plots for the remaining compounds are presented in Appendices K to Q.
195
-------
Figure 91. Common logarithms of (a) Acetaldehyde, and (b) Acrolein, by vehicle and fuel (Bag 1, full
design).
-V
-2
I
!
-
s
-3-
4-
1
2-
-3-
|
1
1
-4
(
*
4*
;** +
1 £
t <-
C °
*+ "Of
** !
-*
3
«4id«
* >
* v
">
+ 1*t 0
H + *b
f%J
'i^ «*
> *
M
* a
*»
3
nbde
(a) Acetaldehyde
D
a
8 i+ o on ji v± Da
£ % + j D °
r^*0 n, ^\ *? ** t * *j*o JSD "c£|:
1 °° *^*" dS' * S* *
^^^ «Vi! ft%i$»: s v '"
.' fi : 1 fi
$
100200300400500600700800900130
Obs,, (sorted by vehicle, fuel)
* + * ALIMA 000 CALIBER ODD CAMRY « » CIVIC as* COBALT
COROLLACE EjaPLORiRXLT f 130 * * » TOOJ5 a n o IMPALALS
* -f * THim-l- 000 ODVSSEV ° D o OUTLOOKXE " SIENNA SILVERADO
(b) Acrolein
%° °
°5 a
8 3$
a D_ £,
aft D *-*i ^m o00^*1
1%, B SB »Vt» JW»«W*
f&
, *
*"
i:
*
o a * fi £d * H- ww* * * *=« *
fl020C3C040Q500eOC'7009CCSOOtX
Obs.. (sorted by vehicle, fuel)
+ + * ALTTMA oon CALIBER o o a CAMRY *** CMC « a - COBALT
- COROLLACI EXPLORERS.! F130 * * FOCGS " » n IMPALALS
* + * LIBERT- 000 ODYSSEi' n ° n OUTLOOKXE " SIENHA SILXBIADO
196
-------
Figure 92. Linear-effects plots for Acetaldehyde (g/mi) and ln(Acetaldehyde) vs. ethanol content (%)
(Bag 1, full design); data are averaged by four ethanol levels and by vehicle.
vehicle
* ALT] MA
COROLLACE
T LIBERTY
'CAUSER
EXPLOFERXLT
'ODYSSEY
Ethand
= CAMRY
F150
= QUTLCOKXE
avic
FOCUS
slew*
'COBALT
IMBMALB
-4
-8
COROLLACE
LIBERTY
CALIBER
EXPLOHERXLT
'OCTfSSEY
Etfianol
CAMRY
F150
-OUTLOOKXE
CMC
FOCUS
SIENNA
'COBALT
IWKAAIH,
SIIVERACO
197
-------
Figure 93. Linear-effects plot for Acetaldehyde and ln(Acetaldehyde) vs. T50 (°F) (Bag 1, full design);
data are averaged by five T50 levels and by vehicle.
290
240
vehicle
COROLLACE
LIBERTY
CALIBER
EXPLCRERXLT
'ODYSSEY
-CAMRY
F150
OUTLOOKXE
OVIC
FOCUS
SIENNA
COBALT
-IMFALA1S
SllVERADO
-4
-5
-6
-7
150
vehicle
160
ALT1MA
COROLLACE
-LIBERTY
170
130
'CALIBER
EXFLCflERXLT
ODYSSEY
190
200
210 220
290
240
T50
CAMRY
F150
-OUTLDOKXE
FOCUS
SIENNA
-COBALT
HMFftLALS
SILVERADO
198
-------
Figure 94. Interaction plots of Acetaldehyde and In(Acetaldehyde) vs. ethanol, by T50 level; data
averaged by four ethanol and five T50 levels.
~[so (deg.Fl 150
240
-5
o>
3Z
-6
-7
150 (deg.F)
150
ID
Ethanol (%)
BO 190
20
220
240
199
-------
Figure 95. Interaction plots of Acetaldehyde and ln(Acetaldehyde) vs. T50, by Ethanol level; data
averaged by five T50 and four Ethanol levels.
O.OM
0.013
0.012
0.011
0.010
0.009
0.008
0.007
0.006
0.005
0.004
0.003
0.002
0.001
160 160 170 180 190 200 210
150 (deg.F)
Etil&nol {%$ '""'""'' 0 *"*"* 10 °***f 15 ""*"* 20
220
230
240
-4
-5
a
32
-7
160
160
180
Ettionol (%) l|~l~tl
SO 200 210
T50«teg.F)
0 10 15 20
220
230
210
200
-------
8.7 Model Fitting
In all cases the response variable was the natural logarithm of estimated emissions
measurements, (In Yt), and the predictors were the standardized or doubly standardized fuel
properties, as described above (see Equation 4 and Equation 7, page 29). The vehicles were
included as class variables, as described below. The predictors considered for inclusion in
models are listed in Table 80, along with the notation used to identify them.
Table 80. Description and notation for parameters included in model fitting.
Fuel Parameter
Ethanol content (%)
Aromatics content (%)
RVP (psi)
T50(°F)
T90 (°F)
Model term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Arom x RVP
Arom x T50
Arom x T90
T90 x T90
T50 x T90
RVP x T90
In optimized design
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
NO
NO
NO
NO
NO
NO
Notation
ze
za
zr
zs
Z9
zzee
ZZ55
77
^^ea
22er
22e5
22e9
zzar
zzas
ZZap
ZZpp
ZZjp
ZZrp
Standardization1
One-stage
One-stage
One-stage
One-stage
One-stage
Two-stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
Two-stage
Two -stage
Two -stage
Two -stage
Two -stage
Two -stage
1 For one-stage standardization, see Equation 5, for two-stage standardization, see Equation 6 and Equation 7.
For each compound and bag, the procedures for model-fitting varied, depending on the degree of
censoring, and whether media contamination applied.
With respect to censoring, the following rule was applied. If the number of censored
measurements was < 5, we substituted the smallest measured positive value for the missing
values, and proceeded with model fitting, using a mixed-model approach. However, if the
number of censored measurements was > 5, we fit a model using Tobit regression (i.e.,
"censored normal regression"), an established technique for analysis of left-censored datasets.
For compounds affected by media contamination, we integrated the approach to censoring with
an "Estimated Dependent Variable Model," an approach to modeling datasets with measurement
uncertainty in the response variable. Table 81 summarizes the modeling approaches used, by
compound and bag. Note that if the level of censoring was considered too severe to allow for
model fitting, "modeling approach" is assigned as "none."
201
-------
Table 81. Summary of Modeling approaches, by Compound and Bag.
Compound
Acetaldehyde
Formaldehyde
Acrolein
Ethanol
Benzene
1,3 -butadiene
Ethane
Media
contamination
YES
YES
YES
YES
NO
NO
NO
Bagl
No. censored
values
<5
<5
>5
>5
<5
<5
<5
Modeling
approach
mixed
mixed
Tobit
Tobit
mixed
mixed
mixed
EDV
YES
YES
YES
YES
NO
NO
NO
Bag 2
No. censored
values
<5
<5
>5
>5
>5
>5
<5
Modeling
approach
mixed
mixed
none
Tobit
none
none
mixed
EDV
YES
YES
YES
NO
8.7.1 The Estimated Dependent Variable Model (EDV)
This approach has been developed for situations in which measurement uncertainty plays a
1 &
substantial role in generating the set of values for the response variable . After estimating
measurements and variances as described above, development of the EDV involves two steps.
The first step is to fit a preliminary model of the compound in terms of the fuel properties. For
this purpose, we fit "full" models, containing all candidate fuel properties. The full models were
the "17-term" full for the full design, and the "4-term" full model for the reduced design. In
addition, the model included a dummy variable for each vehicle.
We solved the model by least squares (using Proc Reg in SAS 9.2) and obtained simple residuals
rt. Using the residuals, we re-estimated the random error (a2£) using the following expression
(Equation 24),
Equation 24
n p \
where n = the total number of measurements (on all fuels and vehicles) andp = the total number
of parameters in the model, including the intercept. After re-estimation of the random error, we
recombined it with the variance of the media contamination to calculate a set of "variance-based"
weights, as shown in Equation 25.
1
Equation 25
In calculation of the weights, the variance of the media contamination is multiplied by 4.0, to
account for the four times that the contamination can affect the total measurement, for the two
202
-------
sets of two media used to measure sample and background, respectively (Equation 21). When
applicable, the weights wt were applied to all subsequent models during the model-fitting
process, for compounds affected by media contamination.
8.7.2 Fitting by Backwards Elimination
The process started with the appropriate full model (17-term or 4-term), and proceeded by
backwards elimination. In each step, one or more parameters were removed, and the model was
refit. Models with one or more terms removed will be referred to as "reduced models." Terms
were selected for removal based on the/?-value for their respective t-tesi of significance (p >
0.10), starting with parameters with the highest ^-values.
At each step, we tested the goodness-of-fit of each reduced model against that of the full model
using a likelihood ratio test. In addition, each successive reduced model was tested for goodness-
of-fit against the preceding reduced model. At each step, if the current reduced model was not a
significantly poorer fit than its predecessor, it was accepted as the current "best fit." To interpret
the goodness-of-fit test, the current reduced model was considered a poorer fit than its
predecessor if the/>-value for the likelihood ratio test was < 0. 10. The process was repeated until
the current reduced model was a significantly poorer fit than its predecessor.
In performing the likelihood ratio tests, it was necessary that the two models included in the test
be "nested," i.e., that both models have all terms in common except the subset of terms whose
inclusion is the subject of the test. This condition always applied, in that all reduced models were
nested within full models, and each reduced model was nested within the preceding reduced
model. For a specific test, the model with more parameters is designated as the "reference"
model, and the model with fewer parameters as the "nested" model. The test was fit in standard
fashion, using the log-likelihood statistics output as the primary fit statistics for the models (all
models were fit by maximum likelihood estimation). The test statistic is calculated as the
difference in the -21og-likelihood (-21nL) between the nested and reference models, and which is
assumed to be distributed as a %2 statistic with d degrees of freedom, where d is the difference in
the numbers of parameters between the two models (pref- ^nested).
XL =-21n -tsL. |=-21nZnested -(-21nZreference)~^2 Equation26
"'reference
The test was considered significant if the/>-value was less than 0.10.
8.7.3 Mixed models
For the compounds and bags indicated in Table 81, we fit mixed models. Fuel properties were
treated as continuous numeric variables and assigned as fixed factors; each vehicle was treated as
a class variable and assigned as a random factor. We fit the model as a random coefficients
model, in which the random effect is a random intercept fit for each vehicle. However, we did
203
-------
not attempt to fit random slope coefficients for individual vehicles. For the speciated
compounds, the mixed model structure was applied as described in 5.3.1 (page 97).
Table 82 shows the set of nested models fit for Bag 1 Acetaldehyde. The model including all
potential candidate terms, both those included in the optimized design, plus additional terms, is
denoted as the "Full model." The sequence of models fit by sequentially removing terms from
the full is listed in order. The reduced models are identified by the number of terms removed
from the full. For example the first model fit by removing the aromaticsxT50 interaction from
the full is denoted as "Full minus 1," abbreviated as "FM1." In this case, eight reduced models
were fit, from "FM1" to "FM8," with each successive model nested in its predecessor. In this
case, it is interesting to note that only one parameter not in the optimized design is included in
final four models (FM5-FM8).
For each model listed in Table 82, Table 83 portrays the fit statistics and tests of fit. The table
shows the "fitting history" of the model, beginning with the full model and ending with the
selection of the reduced model giving the "best fit." In this context, the best fit model represents
the model giving the best statistical fit to the EPAct Phase-3 dataset, under the assumptions and
procedures adopted and implemented during the model-fitting process, as described. In the
block containing "fit parameters" the table includes the number of terms p (including the
intercept), the -2 log likelihood (-21nL), which represents the basic fit criterion from the
maximum-likelihood fitting procedure, and the Bayesian Information Criterion (BIC), as
reported by the MIXED procedure. Based on these parameters, likelihood-ratio tests of fit were
conducted at each fitting step (Equation 26). The first block shows results of tests conducted
using the full model as the reference model, and each successive reduced model as the nested
model. The first column, the "deviation," represents the difference in the -21nL statistics
between the nested and reference models. The second column, "d" represents the difference in
the number of terms between the reference and nested models; this value increases by 1 for each
successive model. The third column represents the/?-value for the test statistic, given as a chi-
square statistic with d degrees of freedom (%/). The third block of the table also contains results
of likelihood-ratio tests, with each model tested not against the full but against its immediate
predecessor in the series. Within this block, the deviation, d and/?-value are calculated in the
same way, except that the value of dis 1 for all tests.
In this procedure, the BIC and "tests-against-full" are included for completeness and as
corroborating information. However, the "test-against-previous" is governing for selection of
the best-fit model. Note that the test-against-full qualifies as the test-against-previous for the
first reduced model (FM1). At each step, the models were fit, and the tests conducted. If thep-
value for the test-against-previous is greater than the critical value, the null hypothesis of no
significant difference in fit between the reference and nested models is retained, and the nested
model is retained as the current best fit. This process was repeated to the point of "one step past
best" i.e., to the point at which the test-of-fit was clearly significant. When this result is
obtained, the null hypothesis of no difference in fit is rejected in favor of the alternative
204
-------
hypothesis of a significant difference in fit, and interpreted to mean that dropping the last model
term led to a decrease in goodness-of-fit. The last model with an insignificant result is selected
as the "best-fit."
In this example, the BIC and tests-of-fit generally corroborate closely, in that the BIC drops
steadily from the full to FM7, and then increasing at the end of the series. However, Table 83
shows an apparent anomaly at step "FM6." For FM6, the BIC increases slightly relative to FM5,
suggesting a decrease in fit, and the test-against-previous is marginally significant, suggesting
that FM6 might qualify as the best fit. However, when pushing "one-step-past," and fitting FM7,
the BIC drops again to a value lower than that for FM5, and the test-against-previous again
exceeds the critical value. Seeing this result, we proceeded to an additional step, fitting FM8.
For FM8, the BIC increases sharply, and the test-against-previous is well below the critical
value, indicating a significant decline in fit. Based on these results, the reduced model FM7 was
selected as the best fit.
For the full and best-fit models, Table 84 shows the coefficients and type-Ill tests of effect. For
the most part, coefficients for terms in both models are similar in value. Exceptions include the
T50 (Z5), etOHxaromatics (ZZea) and the T5QxT90 (ZZ59) terms, which change by margins of-
13%, -39% and 45%, respectively. These changes might be explainable in terms of the dropping
of interactions involving etOH, aromatics, T50 and T90, including etOHxTSO, aromaticsxT50,
etOHxT90 and aromaticsxT90. In general, though, the relative stability of the other coefficients
suggests that the process of standardization induces the model terms to act as though effectively
independent, as required by the multiple regression model. For all parameters retained in the
best-fit, all standard errors are lower than corresponding values in the full, suggesting that
model-fitting has resulted in improved precision of estimation for these parameters. In contrast,
note that all terms dropped during model fitting had high type III ^-values in the full, with the
exception of RVPXT90, which has a low/>-value in the full, but a higher value at step FM5,
when it was dropped (0.087). Despite the fact that this type-Illp-va\ue is less than 0.10, the
decision to drop this term is based on the corresponding tests-of-fit between FM6 and FM7, as
previously described, not the type-Ill test at FM5. Another result of interest is that for this
model, the principal of hierarchy is maintained with full consistency, in that all linear terms are
highly significant, as are all included interactions. In reviewing these models, an advantage of
the standardization is that the coefficients for different terms can be compared in terms of
magnitude. When reviewing coefficients, then, a striking result is that the effects for etOH (Ze)
and its squared term (ZZee) are by far the strongest effects in the model; this result is not
surprising, however, after having reviewed Figure 92 through Figure 95.
205
-------
Table 82. Models fit for Bag-1 Acetaldehyde
Model term
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x T50
etOH x T90
etOH x RVP
Arom x RVP
Arom x T50
Arom x T90
T90 x T90
T50 x T90
RVP x T90
Notation
Ze
za
zr
zs
Z9
ZZee
ZZ55
zzea
zze5
ZZeg
z,z,er
Z-'^-'ar
ZZa,
7.7.a9
L^/^gg
7.7.59
7.7.rg
Model
Full
FM11
X
FM2
X
FM3
X
FM4
X
FM5
X
FM6
X
FM7
X
FM8
X
1 Represents "Full minus 1," etc.
Table 83. Fitting history for Bag-1 Acetaldehyde - with "FM7" selected as best fit model.
Fit Parameters
Model
Benchmark
FM1
FM2
FM3
FM4
FM5
FM6
FM7
FM8
P
18
17
16
15
14
13
12
11
10
-21nZ
-31.467
-31.389
-30.460
-29.266
-27.873
-25.898
-22.970
-20.561
-14.980
BIC1
22.69
20.06
18.29
16.77
15.46
14.72
14.94
14.64
17.52
1 A lower value indicates a better fit.
Test with respect to
Full
Dev.1
0.078
1.007
2.201
3.594
5.569
8.497
10.906
16.487
d
1
2
3
4
5
6
7
8
Pr>3C2
0.78
0.60
0.53
0.46
0.35
0.20
0.14
0.03
1 The deviation is the difference in
the -21oglik statistics for the
nested and reference models,
respectively, per Equation 13.
Test with respect to
Previous Model
Dev.
0.929
1.194
1.393
1.975
2.928
2.409
5.581
d
1
1
1
1
1
1
1
Pr>5c2
0.34
0.27
0.24
0.16
0.09
0.12
0.02
206
-------
Table 84. Acetaldehyde (Bag 1): Coefficients and Type-Ill Tests of Effect for the Full and Best-Fit
Models.
Effect
Intercept
Ze
77
Z.Z.g(
zz
55
77
ZjZjgQ
77
f^f^er
ZZ,
e9
77
t^^ar
ZZap
ZZ,
99
ZZ
59
f
0.000000
0.000000
0.000018
0.0048
0.000000
0.00016
0.000000
0.000000
0.021
0.021
0.33
0.17
0.01738
0.004828
0.008759
0.01270
0.02718
-0.0206
0.1154
0.08743
0.01618
0.01729
0.008852
0.01503
0.01132
0.009971
898
898
898
898
898
898
1.07
0.28
0.99
0.84
2.49
-2.07
0.28
0.78
0.32
0.40
0.013
0.039
Best-Fit Model (FM7)
Estimate
-5.2323
0.8145
0.03484
-0.04170
0.08670
0.03801
-0.1669
0.06665
0.01840
0.02194
Std.Err.
0.08785
0.01020
0.008249
0.008833
0.01063
0.007764
0.007849
0.007993
0.007777
0.007845
d.f.
15
898
898
898
898
898
898
898
898
898
f-value
-59.6
79.9
4.22
-4.72
8.16
4.90
-21.3
8.34
2.37
2.80
Pr>f
0.000000
0.000000
0.000027
0.000003
0.000000
0.000001
0.000000
0.000000
0.018
0.0053
0.03959
0.1149
0.08850
0.008256
898
4.80
0.000002
8.7.4 Tobit regression
For compounds and bags with high levels of censoring, we fit "censored normal regression," or
"Tobit" models, a technique commonly used for left-censored data19'20. We fit the models using
the LIFEREG procedure in SAS 9.2, as applied for left censoring.
As with the mixed models, the procedure solves for the model parameters using maximum
likelihood estimation. However, the Tobit approach does not attempt to estimate the missing
values. Rather, the formulation of the maximum likelihood function (L) is modified so as to
compensate for the absence of the censored values and to estimate values for the model
coefficients accordingly. In the Tobit model, each measurement is represented by its probability
density (standard normal), given an assumed set of parameters, and each censored value is
represented by the cumulative probability that the value would be less than the effective
censoring level.
As the LIFEREG procedure is not able to handle random factors, it was necessary to enter each
vehicle as a fixed factor, represented as a dummy variable. Thus, the model outputs an intercept
for each vehicle, and an estimate of random error variance. It does not estimate a component of
variance representing the between vehicle variability (the variance of the random intercepts).
This step must be performed manually, as described below.
207
-------
In cases when the EDV applied, the model incorporated weights estimated from the preliminary
model, as described above, except that the preliminary model was run as a Tobit model, rather
than as a least-squares model. In all other respects, estimation of the weights was identical.
The procedure outputs the log likelihood (L) as a goodness-of-fit parameter. It does not output
an estimate of the BIC, but the BIC is readily calculated from L, the number of model terms/?
and the total number of (non-missing) observations «, as
n Equation 27
Using these parameters, the backwards elimination model-fitting was performed as with the
mixed models, as previously described, with the exception that we did not use the Type-Ill tests
of effect output by the procedure, because vehicles were not appropriately treated as random
factors. To compensate, we estimated standard errors numerically, by applying jackknife
repeated replication (JRR), using vehicle as the sampling unit. The procedure is simple. With
«veh vehicles included in the sample, we ran the model «Veh times, excluding one vehicle in each
run, and saving the coefficients from each run. In each replicate, the remaining vehicles as
assigned as weight Wb00t, calculated as
wboot = ^ Equation 28
Note that in cases when the EDV was incorporated, the entire process was repeated for each
replicate model, including the fitting of the preliminary model, and estimation of variance-based
weights. After fitting the preliminary model, a final "jackknife weight" w-^ was calculated as the
product of the variance-based and sampling weights wt and Wboot (Equation 29).
wjk = w,wboot Equation 29
After fitting all replicates, the variance of estimation for each of the parameter estimates was
estimated by calculating sample variances on the set of replicate coefficients,
v-A)2
Equation 30
where /?; is the parameter estimate for replicate y and J30 is the parameter estimate from a model
fit using all vehicles. The square roots of these variances gave standard errors that served to
develop tests of effect used to guide the backwards elimination process. Test statistics were
calculated in the typical fashion as
208
-------
, _p
'test ~ Equation 31
with corresponding two-tailed ^-values estimated by
p = 2(l- Pr{r > tactual |}) Equation 32
where the ^-statistics are taken from a t distribution with wveh degrees of freedom. After
calculating the tests of effect, the model fitting procedure proceeded as with the mixed models.
Below, we present the results of a Tobit model fitting process, using Bag 1 acrolein as an
example. Table 85 shows the models fit for Bag 1 acrolein. The best-fit model, FM8, has nine
terms, five linear effects plus one quadratic term and three interactions. As with acetaldehyde,
only one term outside the 11 design parameters (T50xT90) remains in the best-fit model. For this
compound, we fit a model containing only the five linear effects, to examine the adequacy of a
model without any 2nd-order terms.
The model fitting process is illustrated in Table 86. The format is identical to that for
acetaldehyde, using parameters calculated for the Tobit model as described above. Again, the
sequence shows the improvement in fit from the full model to the best-fit (FM8), through a
steady decline in the BIC. What is striking is that the linear-effects model gives a dramatic loss
of fit in relation to any of the other models, including the full model. Table 87 shows the
coefficients and tests of effect for the full and best-fit models, with the standard errors and t-
statistics based on the jackknife replication process described previously.
As mentioned, because the vehicles were entered into the Tobit model as fixed effects, the
procedure could not calculate a grand intercept for all vehicles, as does the mixed model
procedure. It was thus necessary and straightforward to calculate the grand intercept as the mean
of intercepts for all vehicles. As is typical with dummy variables, the intercept reported by the
model represented the intercept for the last vehicle entered, and the intercepts for the remaining
vehicles represent differences between their intercepts and those for the reference vehicle,
Accordingly, the grand intercept was calculated by adding the intercept differences for all
vehicles (A/?o,/) to the reference vehicle intercept (/?o,ref), and averaging (Equation 33).
Equation 33
Similarly, to calculate a variance to represent the random covariance parameter fit by the mixed
model for vehicle, we substituted a simple variance of the vehicle intercepts, calculated as
209
-------
Equation 34
This result is recorded as cr*eh in the table, along with the random error variance
-------
Table 86. Acrolein (Bag 1): Model Fitting History (FM8 selected as best-fit model).
Fit Parameters
Model
Full
FM4
FM7
FM8
Linear Effects
P
18
14
11
10
6
-21nZ
1105.40
1106.84
1111.16
1114.31
1264.37
BIC1
1227.52
1201.82
1185.79
1182.15
1306.08
1 A lower value indicates a better fit.
Test with respect to
Full Model
Dev.1
1.4446
5.7634
8.9122
159.9748
d
4
7
8
12
Pr>3C2
0.8364
0.5676
0.3498
0.0000
1 The deviation is the difference in
the -21nL statistics for the
nested and reference models,
respectively, per Equation 13.
Test with respect to
Previous Model
Dev.
4.3188
3.1488
151.0627
d
o
J
1
4
Pr>3C2
0.2290
0.0760
0.0000
Table 87. Acrolein (Bag 1): Coefficients and Tests of Effect for the Full and Best-Fit Models.
Effect
Intercept1
Ze
2a
zr
25
Z9
zzee
ZZ55
22ea
zzer
ZZe5
ZZep
77
t^^ar
27.a5
7-7.ag
ZZpp
ZZ.5P
ZZrp
f
0.000000
0.00012
0.00027
0.000091
0.000000
0.000026
0.00019
0.00091
1 Not fit by the model, but manually recalculated from intercepts for individual vehicles.
0.05985
0.3629
0.3213
0.01271
15
4.71
0.00028
211
-------
Figure 96. Acrolein (Bag 1): Cumulative Distributions of Coefficients for the five Linear Effects (Each
data point represents a single jack-knife replicate).
LL
U
O
o
3
o
o
n
O
o
o
n
O
o
o
0.2300 0.2350 0.2400 0.2450 0.2500 0.2550 0.2600
ze
u
<;
O
,
o
O
O
o
<>
O
^^
£
0.1500 0.1600 0.1700 0.1SOO 0.1900 0.2000
ft
c
cr
LJ.
u
D
n
n
n
n
n
n
D
D
n
n
D
0.0800 0.0900 0.1000 0.1100 0.1200
z,
to
£
>
u
A
ti
L
£
A
A
A
A
A
A
A
0.2200 0.2300 0.2400 0.2500 0.2600 0.2700
+
-|-
+
-|-
-h
I
-)-
-I-
+
+
-i_
+
+
-0.0700 -0.0650 -0.0600 -0.0550 -0.0
212
-------
9 Summary and Conclusions
9.1 Regulated Emissions, Total Hydrocarbons and Methane
9.1.1 Modeling Results
To aid in gaining a broad view of the results of this project, the sets of coefficients presented
above in tabular form are also presented in this section in the form of bar charts, in which the
bars represent both the magnitude and sign of the coefficients. It is important to note that the
coefficients presented are those for one- or two-stage standardized terms. As a reminder of this
critical step, we have retained the "Z^" notation for linear terms and the "ZZX>," notation for
quadratic and linear terms in all tables and figures throughout. Thus, all fuel properties are
centered on means of 0.0 and expressed in terms of their own standard deviations (which take the
value of 1.0). While more abstract, this approach has the advantage of keeping coefficients for
different effects comparable in terms of magnitude, allowing direct comparison of effects in
terms of importance. Such direct comparisons would not be possible were the terms scaled in
terms of each of the properties' units (vol.%, psi, °F).
Reduced Models: As in the tables, results for reduced models fit starting with the 11-term full
model and the 16-term design model are presented. For convenience, these reduced models will
be referred to throughout as the reduced^ and reduced^ models, respectively.
9.1.1.1 Hydrocarbons (THC, NMOG, NMHCandCH4)
Coefficients for reduced models for the hydrocarbon species are presented graphically in Figure
97 to Figure 100 for THC, NMOG, NMHC and CH4, respectively. The plots include results for
Bags 1-3 and constitute a graphic presentation of the tabular results previously shown in 7.2,
(page 137 ff). In addition, Figure 101 presents coefficients for the three species juxtaposed on
single graphs for each bag, to facilitate comparisons among species, as described below.
Cold-Start Emissions. For Bag 1 results, the sets of coefficients are broadly similar for THC,
NMOG and NMHC . For the linear effects, all coefficients are positive, except for RVP. In
terms of absolute magnitude, the most important linear effect is T50, followed by aromatics.
Ethanol is ranked third, and RVP fourth, except for NMHC, for which RVP is third and ethanol
fourth. T90 is consistently the smallest term. NMOG has the largest etOH effect, and NMHC the
smallest, whereas NMHC has the largest aromatic and T50 effects, and THC the smallest. Both
quadratic terms are included in all three models and have similar magnitudes. As the signs of the
etOHxetOH and T50xT50 quadratic terms are both positive, they impart upwards curvature to
their respective linear trends. In addition, all three models include the same three interaction
terms retained from the design model (etOHxarom, etOHxTSO and etOHxT90); the magnitudes
of the interactions are also similar, but with NMOG and NMHC coefficients slightly larger than
those for THC, although the differences do not appear statistically significant. As the signs of
these interactions are all positive, as are all three linear terms, these interactions qualify as
213
-------
reinforcement effects. All three models also retain the same three interaction terms from among
the five additional terms in the extended model; all three effects are positive and similar in size
in all three models.
In Bag 1, a different pattern is evident for CH/i. The coefficient for aromatics differs in size from
those for the other three species, as well as in sign, with a strong negative aromatics coefficient
as the most striking feature of the CH4 model. The ethanol effect is similar to that for NMOG,
whereas the T50 and RVP effects are just over half the size of their counterparts. The CH4
model does not retain a linear effect for T90. The CH4 model retains both quadratics terms, with
magnitudes equal in sign but about half the size of those for the other species. The pattern for the
interactions in the reducedn model is similar, except that CILj retains the etOH xRVP interaction
rather than the etOHxT90 interaction. The reducedie model retains four of the five additional
interactions; all effects are positive and similar in magnitude.
Hot Running Emissions. For Bag 2 results, the pattern is not as consistent among THC, NMOG
and NMHC, although similarities remain. The effects describing bulk properties (RVP, T50,
T90) have larger effects than those describing fuel composition (etOH, Arom), although
differences are not always large. T90 is the largest effect for all three species, with T50 and RVP
ranked second and third for THC and NMOG. NMHC differs slightly, in that this ranking is
reversed. Ethanol and aromatics effects are ranked fourth or fifth. Again, NMHC differs in that
an ethanol effect is not retained in the reducedn model. In the reducedie model, the ethanol linear
effect is not significant but is retained on the strength of two interactions involving ethanol.
With respect to aromatics, the THC model differs in that the aromatics coefficient is negative,
rather than positive, as for the other two species. The T50xT50 quadratic term is retained in all
three models, but differs in size, with the THC coefficient smaller than those for NMOG and
NMHC. The model for NMOG retains two interactions from among the terms in the design
model, that are not included in the THC or NMHC models. When considering all interactions,
including those from the extended model, the set of interactions retained is similar. All three
models retain the etOHx Arom, etOHxRVP and aromxTSO interactions, which are positive,
negative and positive in sign, respectively. Additionally, the NMOG and NMHC models retain
the etOHxTSO interaction. However, this term is marginally significant.
In Bag 2, the pattern again differs for the CH4 model. The fuel composition terms are largest,
followed by those for the bulk properties. Aromatics has the largest coefficient, although
negative in sign, followed by ethanol, which is positive. The distillation parameters follow in
third and fourth places, with RVP in fifth. The strength of the aromatics term may help explain
the negative coefficient for THC which stands out in contrast to the positive coefficients for
NMOG and NMHC. The CH4 model contains one quadratic term (T502), although its size is
small. The CH4 model includes one interaction from the design model, and two from the
extended model, although all three are small in size.
214
-------
Hot-Start Emissions. For the Bag 3 results, the overall pattern is more similar to Bag 2, rather
than to Bag 1, although less consistent. Only CH4 has a significant linear effect for ethanol. The
effect for THC is quite small but insignificant, whereas those for NMOG and NMHC are larger
but also insignificant. For these species, the ethanol effects are retained only to maintain
hierarchy with respect to two interactions involving ethanol. For aromatics, THC and CH4 have
strong negative aromatics effects; the effect for THC may be explained by the presence of its
CH4 component. In contrast, neither NMOG nor NMHC retain aromatics effects in the Bag 3
model, which contrasts with both models for Bags 1 and 2. None of the reduced models include
significant RVP terms; again the RVP effect is retained on the strength of the etOHxRVP
interaction, which is large and significant for NMOG and NMHC, small and marginally
significant for THC, and absent for CH4. For the distillation parameters, the relatively small
positive effects for THC may be explainable by the contrast between the behavior in its non-
methane and methane components, which show large and small coefficients for these terms,
respectively. None of these four models include either quadratics term.
Several additional observations can be made. One is that the magnitudes of corresponding
coefficients are generally larger for Bag 1 than for Bag 2 emissions, suggesting that the effects of
fuel properties are more pronounced for "cold start" than for "hot running" emissions. This point
also generally applies to Bag 3 relative to Bag 1, with some exceptions. One exception is that
the coefficients for distillation parameters in the NMHC models are as large as those in the Bag 1
model. Another interesting example is that the negative coefficients for aromatics in the CH4
models are very close in size in all three models.
Secondly, it is apparent that relative patterns among coefficients are more similar between Bags
1 and 2 than between Bag 3 and the other two Bags. The reasons for this difference may be
related to the relative importance of measurement error in Bag 3 relative to the other two Bags.
In the charts, this result may be illustrated by the fact that the confidence intervals for
coefficients in Bag 3 are generally larger than their counterparts in the Bag 1 or Bag 2 models.
Again CH4 is an exception, having been measured with roughly similar precision in all three
bags.
Finally, the similarities between the NMOG and NMHC models are expected, given the close
correlation of these two species to NMHCpio, as described above in 3.2 (page35 ff), and the fact
that a majority of values for Bags 2 and 3 were imputed from NMHC FID. However, the
similarities are also highly pronounced for Bag 1, for which only a small fraction of
measurements were imputed.
215
-------
Figure 97. THC: Model Coefficients for Reduced Models, based on 11-term and 16-term full models.
Tabular results shown in Table 56 and Table 57 (error bars represent 90% confidence intervals).
Standardized Coefficient
01 £
01 n
-0.05
_n 1 n .
Cold-Start (Bag 1)
1
reduced [11-term)
a
y
I'1
~ IT
£ &
a
1,
a a
H E-
1
i
s
CM
1
O
in
H
reduced (16-term)
i i
i
i
i
o
1
a
\
N 21
'- -
« £
X X
~ X
Q o
, ,
o% i- in a-'
s M H
pips
Q 5 a 5
un i-
| |
U.Z.U
riU
Crt 1 C
Q
U
a3 °-lu "
o
u
2 u.uo
N
-, o.oo
TJ
S n nK
% -0.05 ;
_n m -
i 1! i
iiisiyii
B s & o o a N 5
« i |=|
Hot-Running (Bag 2)
reduced (11-term)
1
Illlliii
&-ooA-ooo°
i>wa*>in^ai1^
'J^^'gfe^^^
i S i 1 = = s &
0.10
Hot-Start (Bag 3)
reduced (11-term)
reduced (16-term)
a a J K I 5
= = a a a a
.
2 2 2 £ £ E [2 S
» » » a S S
216
-------
Figure 98. NMOG: Model Coefficients for Reduced Models, based on 16-term and 11-term full models.
Tabular results shown in Table 58 and Table 59 (error bars represent 90% confidence intervals).
s
-
'3
S
N
I
1
(/>
0.20
0.15
0.10
0.00
0.05
0.10
Cold-Start (Bag 1)
reduced (11-term)
reduced (16-term)
c
in ^ os
t9 3 S
M M M
O
S
g i a
o
u
c
01
N
0.20
0.15
0.10
0.05
0.00
-0.05
-0.10
Q
Hot-Running (Bag 2)
reduced (11-term)
reduced (16-term)
th
R I I
o
S
I &
3 S
idardized Coefficient
~> o o o o o e
M
"-0
f\
1 c
1 n
nt
OS -
10 -
1C-
Hot-Start
1
i
i
_ ^
1
I
reduced
0
^O*i U Ift fQ ^
a a £ K 1 N
e- f- g g E £
0 in t, CC
g t- a -
a o
2 *
reduced
1
o
f-
|
§ 1
S &
H B
= S
§ 1
LO
0
in
H
S
(Bag 3)
(11-term)
(16-term)
|
0
E-
< IUO.K
in ^
0 0
H Cf
217
-------
Figure 99. NMHC: Model Coefficients for Reduced Models, based on 16-term and 11-term full models.
Tabular results shown in Table 60 and Table 61 (error bars represent 90% confidence intervals).
0.20
c 0.15
I
£
o
u
TJ
01
N
n
j_»
W)
0.10
0.05
0.00
-0.05
-0.10
Cold-Start (Bag 1)
reduced [11-term)
reduced [16-term)
fld
I!
U.£ir xos>iini^.>in^Q%as
5 w SQSF-E-CCHHHE-.
n n cc
U.iO
C
8
y 0.15
£
aj n 1 n
g U.iU
U
^ n fid
J U.US
N
n :
"e -0 05 -
S
^ n m "
_i» 1 K :
T]
J' ^ 'U ' ^ '
t3 * S I
1 e * ;
1
t
^ g in ffl ^ i^
CN fv, E ^ °
zoo
l_l_l
Hot-Start (Bag 3)
reduced [11-term)
reduced [16-term)
S | S & K S
0 t 0 0 0 0
" ca E^ H £ H
5 2 S S S &
1 1 1 1 ^ §
218
-------
Figure 100. CH4: Model Coefficients for Reduced Models, based on 16-term and 11-term full models.
Tabular results shown in Table 62 and Table 63 (error bars represent 90% confidence intervals).
U.1O
t- c\ 1 n
e
u
,
S n n^
u u.us -
0
u
(U U.UU
N
13
C3
(/i -°-lu :
.n 1 1;
-
m
-
2
t '- ' -T^
H&
-
Cold-Start (Bag 1)
1
a i
o" 1*
£ £
O
a
1
in
K
EN
O
in
H
reduced [ll'termj
reduced [16-term)
tJ
i
i i i
S 5 » « 5 n
M N N M N [S3
N Ji tsi N J^ rsi
§ & s s & s
I 1 s? % i %
2 s i
3
> " a n
I i i
1 1
°* s *
§_!,__§
° °
B e &
5 in >
£ t- i*
re
U.±3
bl
e n 1 n
g °-10 -
5
o5 u-u;j
^
"O n c\f\
j u.uu
N
t, -U.Ub
13
S n m
s -°-10
J11S -
1
" 1
1
1 1
1
Bi
rill
= i " H
E
' '
J IS i
a N S
EJ N B
§ S 1
i * j
o
S
Hot-Running (Bag
reduced (11-term)
reduced (16-term]
11111
* m ^ ~ \n c^ o*
«, S S n N N S
g; g o g; o o o
OS f- E- °= F- E- f-
g g = £ E £ <=
ffi CB CO
2)
1
S
N
0
E-
ce
U.1D
^ n 1 n
'y
o
u
aj U.UU
N
M -0.05
e
n 1 R
II
I
x
'I
H u u
,'
tllillll
I h "p 1 § 1 1
g E- ra 4
B i §
2 *
a
Hot-Start (Bag 3)
reduced [11-term)
./^'^'^'^'^'^
00^000°
1 1 1 i i I s
§ 1 1 i i s i
«isMi!i
219
-------
Figure 101. Reduced Models for Hydrocarbon species, fit based on the 11-term design model
(reducedn) (error bars represent 90% confidence intervals for standardized coefficients)
Cold-Start (Bag 1)
THC
0.25
« 0.20
01
y 0.15
E
0.10
U 0.05
N
^ 0.00
a
"e -0.05
-0.10
-0.15
ill
Hot Running (Bag 2)
THC
NMOG
NMHC
CH4
s a
O
_£_
e- o
4-4-
o
a
o
S
c
01
N
1
T3
0.25
0.20
0.15
0.10
0.05
0.00
-0.05
-0.10
-0.15
Hot Start (Bag 3)
220
-------
9.1.1.2 Oxides of Nitrogen (NOX)
Coefficients for reduced models for NOX are presented graphically in Figure 102. The plots
include results for Bags 1-3 and constitute a graphic presentation of the tabular results previously
shown in 7.2, (page 137).
Cold-Start Emissions. For Bag 1 results, aromatics is the single most important effect, by a wide
margin, followed by ethanol. As described, the reduced model differs depending on whether the
16-term or 11-term models are taken as the starting point. When starting with the 16-term
model, the reducedie model retains RVP and T90 linear terms (both insignificant), plus two
interactions not included in the reducedn model: arom>
-------
Figure 102. NOX: Model Coefficients for Reduced Models, based on 16-term and 11-term full models.
Tabular results shown in Table 66 and Table 67.
U
I
O
U
TS
4>
N
1
T=
n
0.30
0.25
0.20
0.10
0.05
0.00
-0.05
-0.10
-0.15
Ireduced (11-term)
reduced (16-term)
I
»'
s
;|
o
o
5
_«
1
u
0
0
0
0
-0
-0
-0
;
i
;
1U -
rtC '
UO _
UU 7
n c ~
Uo ~
1U ~
1 C
.
1
N
o
lo -1
I
si
E
Hot-F
i i i i i i i
£ N 8 * in s*^
"HHgg S§; =
2 [2 i - ^
X Z Z
° o i
tunning (Bag 2)
i
M
O
t-
X
X
reduced (11-term)
reduced (16-term)
1
N
I
X
g
I
2
M
O
f-
X
E
g
5
N
o
E-
X
=
E
1
in
S
o
E-
X
O
t
M
O
0-
H
ce
c
o
u
E
>
E- OS
X X
I 5
o S
o £
LO
N
O
in
5
E
^
i
O
1
n
s
M
O
O
in
F
t
M
0
t
cc
222
-------
9.1.1.3 Paniculate Matter (PM)
Coefficients for reduced models for PM are presented graphically in Figure 103. The plots
include results for Bags 1-3 and constitute a graphic presentation of the tabular results previously
shown in 7.2, (page 137).
Cold-Start Emissions. For Bag 1 results, as with NOX, aromatics is the single most important
effect, by a wide margin, followed by T90 and ethanol. A relatively small effect is also fit for
T50, which differs by 50% between the reducedie and reducedn models, whereas the other three
linear effects are relatively consistent in this respect. Neither model includes an RVP term,
whereas both include a quadratic term for T50. The quadratic term is positive, as is the linear
effect, denoting an upward curvature. The reducedie model includes two interactions not
included in the reducedn model. One term, etOHxTSO, is a design-model term, and the second,
arom>
-------
Figure 103. PM: Model Coefficients for Reduced Models, based on 16-term and 11-term full models.
Tabular results shown in Table 68 and Table 69.
_ ft Aft T
C °-4U ;
;~ U.JU
o -
B
o
N n 1 n
13 :
« onn
C?
V5 -0 10 -
1-
N
-r
1
Cold-Start (Bag 1)
reduced [11-term)
[reduced [16-term)
1 1
tSS&NN o K Sfc'3!J!J51r5?g;
O Lft £c£^^C£HHt
as-nxxxxxx1;
« = = = E£ = =
iiiiiii^
t
N
O
1
s
u.ou -
" n Aft
CQ
u n_ :
g U.JU 7
O -
N A -1 A
1
_; n ftft
t/5 -U.1U '
_n ?ft
Hot-Running (Bag 2)
reduced [11-term)
reduced [16-term)
I1 .
1
a§£aalsl^^^^^§§S
2 gaiK^gg £>gg&S§g§
2f-Sxxx xx^x
= =
S S £
ffl C9 C5
rf n 4.0
« :
«£3
(D
r'R n ?n ^
O
0)
N n i n
1
R
ft ftft
1
ff\ ft 1 ft
Oon ~_
Hot-Start (Bag 3)
reduced [11-term)
.1 . 1
reduced
(16-term)
llJrllllll'lllill
o o ^E. w ^ ^7 "^ ~~
S H ffl
X
fr ^ o1 S-
a H H K
X X X X
O O O o
o o o
i.- r r
^ ^ ^
- = in
p p
0
_
X
1
224
-------
9.1.1.4 Carbon Monoxide (CO)
Coefficients for reduced models for CO are presented graphically in Figure 104. The plots
include results for Bags 1-3 and constitute a graphic presentation of the tabular results previously
shown in 7.2, (page 137). The sets of coefficients for CO appear unique in that they show
markedly different patterns for the different Bags.
Cold- Start Emissions. For Bag 1 results, a striking feature is that the linear effects for ethanol
and T90 are strongly negative, a pattern that occurs for no other emission or bag. The effect for
aromatics is also small but negative. Linear effects for RVP and T50 are relatively small, but are
unusual in that they change sign from positive to negative between the reducedie and reducedn
models. Both models include quadratic terms for ethanol and T50. Both models include
interaction terms for etOHxarom and etOHxTSO effects. Interestingly, the etOHxArom
coefficient in the reducedn model is less than a third of than in the reduced^ model, whereas the
coefficients for the etOHxTSO terms are very close in size. The reducedn model also contains an
etOHxRVP interaction lacking in the reduced^ model. The reduced^ model retains three
interactions from among the additional interactions in the extended model.
Hot-Running Emissions. The model for Bag 2 is relatively simple but presents a very different
profile than the Bag 1 model. A prominent difference is that neither reduced models retain an
ethanol effect, either positive or negative. Also in contrast to the Bag 1 model, the aromatics and
T90 terms are positive and strong, with aromatics as the most important effect. The RVP and
T50 terms are both positive but small, although significant. The reduced^ model retains a small
T5QxT90 interaction from the extended model, aside from which the two reduced models are
very similar, much more so than the Bag 1 model.
Hot-Start Emissions. The models for Bag 3 have similarities to both the Bag-1 and Bag-2
models. They are similar to Bag 1 in that the ethanol effect is strong and negative, but also
similar to Bag 2 in that the aromatics and T90 effects are strong and positive. The RVP effect is
small but positive, but the T50 effect is not present. Both reduced models have no quadratics or
interaction terms.
If we accept the results of these models, especially those for Bags 1 and 2, it could suggest that
the effects of changing fuel properties differs markedly between cold start and running
emissions, more so than for any other emission or bag.
225
-------
Figure 104. CO: Model Coefficients for Reduced Models, based on 16-term and 11-term full models.
Tabular results shown in Table 64 and Table 65.
u
£
§
u
0.20
0.15
0.10
0.05
0.00
-0.05
-0.10
-0.15
-0.20
reduced (11-terrn]
reduced (16-term)
Cold-Start (Bag 1)
I B & a §r-gr-§r-gr-g-ir4
S 2 =
O in
O
-------
9.1.1.5 Summary
The study was conducted to assess the effects of fuel properties on the emissions of vehicles
certified to Tier-2 standards, primarily the Bin-5 standards. Reviewing the results, it is clear that
such effects exist and are measurable. In this section, we will attempt to review and summarize
the results.
As mentioned above, the application of the study design supplemented by the standardization of
parameters allows assessment of fuel effects as though they were independent. These devices
provide powerful tools to aid in interpretation of the effects of changing fuel properties on
emissions. As such, the models will aid in the interpretation of the results of other studies, as
well as the results of this project.
In reviewing Table 88 and Table 89, as well as Figure 97 through Figure 104 above, we can
make some generalizations with respect to the individual fuel properties:
Ethanol: In most models, the linear-effect coefficients for ethanol are positive for both running
and start emissions, implying that increases in ethanol content would be associated with
increases in emissions, if the remaining fuel properties could be kept constant while increasing
the ethanol level. A conspicuous exception to the pattern is CO, which has a negative coefficient
for start emissions and no ethanol term for running emissions. Another exception is NMHC,
which no ethanol term for running emissions. For start emissions, the terms are largest for PM,
NO* and NMOG, whereas for running emissions, the terms are largest for PM, NO* and CH/j,
although presumably, the underlying physical processes could vary among pollutants and
processes. The linear effects for NO* are unique in that the coefficients are nearly equal in size
for both start and running emissions, implying that the relative effect of ethanol is similar in both
processes, even though start emissions are considerably higher. For start emissions, the
etOHxetOH quadratic term is present for all HC species and CO, imparting some curvature to
ethanol trends for these species. It is consistently positive. Its size is similar for THC, NMOG
and NMHC, smaller for CH4 and larger for CO. For running emissions, no models include the
quadratic term.
Aromatics: The patterns for aromatics are less consistent. Coefficients are positive for most
models, with several exceptions for both start and running emissions. One exception is CO,
which has a small negative coefficient for start emissions and a larger positive coefficient for
running emissions. A second exception is THC, for which the start coefficient is positive and the
running coefficient negative. Thirdly, coefficients for CH4 are large, negative and similar in size
for both start and running emissions, which is unique in implying that changes in ethanol have
similar relative effects on both start and running emissions. In terms of magnitude, the pattern is
similar to ethanol, with PM, NOX and NMOG showing the strongest effects. For start emissions,
the interaction between aromatics and ethanol appears in all models except PM. The start
interaction terms are consistent in size and positive in sign for all emissions except NOX. Given
227
-------
the signs of the linear terms, the interaction qualifies as a small reinforcement for all models
except CH4 and NOX, for which it qualifies as an interference. For running emissions, this
interaction appears only for NMOG and CH4, for which it acts as a reinforcement and as an
interference, respectively.
RVP'. A linear term is included for all pollutants except NOX and PM. The sign of the term is
consistently negative with a single exception for running CO, which has a positive term. For the
hydrocarbons, the size of the term is relatively consistent, although the coefficients for running
models tend to be somewhat smaller than those for start emissions. The interaction with ethanol
appears in two models for start emissions, but in no models for running emissions. In both start
models, the terms are small and positive.
T50: Coefficients for this property consistently positive, with the single exception of start CO,
and appears in all models except running NO* and PM. For start emissions, the effects are largest
for THC, NMOG and NMHC, and smaller for CH4, NOX and PM; for CO, the term is negative
and relatively small. For running emissions, coefficients are positive but smaller than for start
emissions; for the hydrocarbons except CH4, T50 is the largest single term. For the hydrocarbon
species, T50 shows a consistent reinforcement interaction with ethanol for start emissions, for
running emissions, the interaction applies only to NMOG. For start CO, the interaction is present
but acts as an interference, in that both linear terms are negative and the interaction is positive.
T90: This term is unique in that it appears more frequently in models for running than for start
emissions, and in that it is sometimes larger in running models than start models. In the start
models, the term is large and positive for PM, small and positive for THC and NMHC, large and
negative for CO, and absent for the remaining models. In the running models, the term is large
and positive for the hydrocarbons except methane, small and positive for PM and CO, and absent
for NOX. The interaction between T90 and ethanol is retained in only two models for start
emissions, THC and NMHC, in which it is positive and similar in size. In both models, the linear
and interaction terms are all positive, qualifying this effect as a reinforcement interaction. The
T90 coefficient is largest for start PM, where it has a reinforcement interaction with the even
stronger aromatics effect.
In addition, it is possible to make some general points about the responses of exhaust emissions
to changing fuel properties that apply across the measured compounds and species, and for both
start and running emissions.
Other factors being equal, increasing ethanol is associated with an increase in
emissions, as indicated by the positive ethanol coefficients in most models, both for
running and start emissions.
Other factors being equal, increasing volatility is associated with reductions in
(exhaust) emissions, as indicated by generally negative coefficients for RVP (and
generally positive coefficients for T50).
228
-------
In relative terms, fuel effects are generally more pronounced for start than for running
emissions, as indicated by the fact that in most cases, the coefficients for Bag 1 models
are larger than their counterparts for Bag 2 models. If we assume that we can validly
make direct comparisons between coefficients between Bag-1 and Bag-2 models, this
result may suggest that the effects of fuel properties are more pronounced during
engine starts than during running operation. One interpretation might be that fuel
effects could be damped by efficient operation of the catalyst after the engine comes
up to temperature.
9.1.2 Models Selected for Application
Based on our review and analyses of study results to date, our intent is to make use of reduced
models fit with respect to the 11-term design model (reducedn) for purposes of description and
prediction. For application, we are retaining reduced models for Bags 1 and 2, based on concerns
that Bag 3 models may not reliably represent fuel-parameter effects on emissions.
Given the low design efficiency for the extended model, we presume that it is more probable that
terms not included in the design model may be more liable to bias or to represent artifacts of
design or measurement. Until such concerns have been ruled out, it is reasonable to retain
models fit within the space defined by the study design.
Coefficients for the reduced models, previously presented in 7.2.3 above, are summarized
below. Coefficients for Bag 1 and Bag 2 models are presented in Table 88 and Table 89,
respectively.
As noted, the results of this project directly apply to vehicles certified to Tier-2 standards. In
considering the applicability and representativeness of the results, several questions arise. Firstly,
as the vehicles employed in the study were relatively new, with less than 10,000 miles
accumulated, we can ask whether these modeling results would apply to similar vehicles having
aged, deteriorated or having accumulated high mileage. Although not as important, we can also
ask whether the results might also be applicable to vehicles certified to earlier standards, such as
NLEV or LEV-I standards.
An important characteristic of emissions, including the results of this project, is that they
typically follow multiplicative or logarithmic scaling. Given that the coefficients represent
differences in logarithms, which represent relative multiplicative differences, i.e., ratios, it is
plausible that the model coefficients, expressed as relative differences, should prove
transportable to either aged Tier-2 vehicles or even pre Tier-2 vehicles.
It is also necessary to consider the composition of the vehicle sample. As described above, it
comprises a judgment sample of high-sales models from major manufacturers in model year
2008. In terms of standards, the vehicles represent the emissions standards that are most
229
-------
prevalent for light-duty vehicles, including Bins 3 and 5 (or equivalent LEV and ULEV
standards under LEV-II), as well as a single Bin 8. The selection of makes and models does not
qualify as a random sample in a strict sense. One practical limitation is that the effort and
expense involved in measurement precluded drawing a reasonably sized random sample of
makes and models. Nonetheless, given the size of the sample, it is plausible that a well-designed
judgment sample may perform as well as a small random sample, as noted by one of the pioneers
of survey design: "... No clear rule exists for deciding exactly when probability sampling is
necessary, and what price should be paid for it. The decision involves scientific philosophy and
research strategy If a research project must be confined to a single city in the United
States, I would rather use my judgment to choose a "typical" city than select one at random.
Even for a sample of 10 cities, I would rather trust my knowledge of U.S. cities than a random
91
selection. ..." Additional clarification of the utility of the models may be provided by
subsequent validation using independent data, including the aspects of applicability and
transportability.
Table 88. Bag 1: Reduced Models, based on the 11-term Design Model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RW
etOH x T50
etOH x T90
Notation
Intercept
Ze
Za
Zr
z,
Z9
zzee
ZZ55
zzea
Zj/JQf
zze5
ZZe9
THC
-0.8664
0.0548
0.0676
-0.0445
0.1288
0.0183
0.0436
0.0736
0.0179
0.0445
0.0214
CH4
-3.0074
0.06994
-0.1053
-0.03275
0.07554
0.02844
0.05170
0.02088
0.01082
0.03048
NMOG
-0.95209
0.08019
0.08782
-0.04224
0.1345
0.04432
0.07579
0.01693
0.04653
NMHC
-1.0315
0.03094
0.09461
-0.04568
0.13689
0.02160
0.04612
0.07534
0.02045
0.04729
0.02441
CO
1.3466
-0.1049
-0.01242
-0.00762
-0.03273
-0.1571
0.07304
0.05358
0.02086
0.01596
0.1064
NO,1
-2.8594
0.06750
0.1339
0.04783
-0.02369
PM
0.6559
0.1582
0.3833
0.0550
0.2923
0.0935
Vehicle variance
Residual error
02veh
^
0.1325
0.06872
0.2855
0.03014
0.1224
0.07538
0.1266
0.07624
0.3920
0.07214
0.5925
0.1458
0.4251
1.0359
Fit excluding the Ford Focus.
230
-------
Table 89. Bag 2: Reduced Models, based on the 11-term Design Model.
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RVP
etOH x T50
etOH x T90
Notation
Intercept
Ze
zfl
Zr
Z;
Zp
zz
ZZ5J
77
t^^ea
77
f^/^QY
ZZe5
ZZe9
THC1
-4.6533
0.0327
-0.0195
-0.0355
0.0501
0.0514
0.0337
CH4
-5.7075
0.05860
-0.09836
-0.02049
0.04394
0.02575
0.01227
0.008769
NMOG1
-5.2360
0.02673
0.03634
-0.04786
0.04915
0.07252
0.05349
0.02171
0.02586
NMHC1
-5.3253
0.03987
-0.05881
0.04548
0.08202
0.04774
CO
-1.3893
0.0913
0.0299
0.0261
0.0440
N(V
-4.5692
0.06299
0.04407
PM
-1.3107
0.1126
0.1662
0.1072
Vehicle variance
Residual error
0 veh
^
0.8384
0.06717
1.1108
0.02518
0.8502
0.1310
0.9691
0.1708
1.9187
0.1256
0.4720
0.1836
0.7827
1.1337
Fit excluding the Honda Odyssey and Toyota Sienna.
2 Fit excluding the Chevrolet Cobalt.
9.1.3 Comparison to Previous Results
It is valuable to place the results of a new study in the context of the existing body of knowledge.
Results of this program generally show that Tier 2 vehicles continue to exhibit sensitivity to fuel
parameters in many of the same ways as older vehicles certified to previous standards, with some
exceptions that will be examined here. This discussion will focus on the direction of specific
linear-effects coefficients in the models, i.e., effects of each property one at a time, as though the
other properties were held constant. Attempts to characterize interactive effects are more
complicated and are thus more difficult to interpret.
9.1.3.1 Ethanol
Over the past two decades a relatively large number of programs have studied the effects of
ethanol. At the same time, since blending ethanol into gasoline also affects many other fuel
properties, and given that ethanol is blended in into gasolines in different ways that affect the
collateral property changes differently, it is difficult to interpret trends across the body of
literature without more information on multiple fuel property changes. A recent summary of the
literature cites several studies from the late 1980s through the early 2000s (covering Tier 1 and
99
earlier vehicles) that consistently show ethanol blends as having increased NOX emissions.
More recently, in the package for the 2009 Renewable Fuels Standard, EPA also summarized a
231
-------
9^
number of data sources and found the same trend . The results of the present study are
consistent with the published results, in that the coefficient for ethanol is positive for both start
and running emissions. However, it is important to note that the models also suggest that
reductions in NO* could occur with corresponding reductions in aromatics, particularly for start
emissions, for which the aromatics coefficient is larger than that for ethanol. Thus, despite much
lower overall emission levels that have been achieved in Tier 2 vehicles through improved fuel
control and catalyst efficiency, the effect of ethanol on tailpipe emissions appears to persist in
both cold-start as well as hot-running operation.
Many of the past studies have simply measured THC or NMHC by flame ionization detector and
therefore have not fully captured the impacts of ethanol on VOC or NMOG emissions/ In the
current study, we fit models to HC as THC, CH4, NMHC and NMOG, and found positive
coefficients for ethanol for both start and running emissions, although the linear-effects
coefficients for running emissions were significant for THC and CH/j, marginally significant for
NMOG and insignificant for NMHC. However, if typical collateral fuel changes (lower T50 and
aromatics) are accounted for, we might project that blending ethanol would tend to reduce THC,
NMHC and NMOG emissions (highlighting the important sensitivities to these other fuel
parameters). Potential changes for CH4 are more difficult to project, as the aromatics and T50
coefficients are opposite in sign.
9.1.3.2 Aromatics
Aromatic content of gasoline has long been understood to affect regulated and unregulated
emissions. The literature review shows consistent findings of reduction in hydrocarbon
emissions with reduced aromatics level, however the trend is more variable for NOX. The
literature summary suggests that higher aromatics content is likely to increase engine out NOX
due to the higher flame-front temperatures produced by aromatic compounds, but they also
appear to increase the efficiency of three-way catalysts that reduce NOX to nitrogen.
For NOX, this study finds positive coefficients for both cold-start and hot-running operation,
which is consistent with effects reported in some of the more recent studies described in the
review. This pattern suggests that the effect might be due primarily to the engine-out effect
during cold-start and transients during hot-running operation. Recent studies performed by
Honda show that PM emission increases are amplified with higher heavy aromatics (C9+)
content, an effect that may be reflected in the positive coefficients for aromatics (and T90)
reported in the present study, pointing to aromatics composition as an area for further
investigation.24 It is interesting to note that aromatics is the only fuel property studied for which
r The typical hydrocarbon analyzer used for emission testing contains a flame ionization detector (FID),
which is calibrated (typically using propane) to accurately count carbon atoms within H-C bonds.
Carbons bonded to oxygen, which occur in carbonyl and alcohol emissions from burning ethanol fuels,
produce a much smaller response in the FID, and thus emissions from ethanol fuels require additional
characterization methods to properly quantify as NMOG or VOC.
232
-------
an increase is associated with increases in all regulated emissions, with the exception of cold-
start CO.
9.1.3.3 Distillation parameters
Relative to ethanol and aromatics, the effects of distillation properties require more complex
study designs and fuel-blending procedures in order to properly evaluate their effects. In
addition, as bulk properties of the fuel blend, their effects on combustion are more difficult to
interpret and understand mechanistically. The literature review summarizes a number of older
studies that suggest reducing T50 and T90 (i.e., lighter, more volatile fuel) increases NOX
emissions, while the present study suggests the opposite. The literature review doesn't
distinguish cold-start vs. running emissions, but the current study suggests that reduced T50 is
associated with NOX reduction during the cold start, and has no effect after warm-up. With the
understanding that emission controls on Tier 2 vehicles are highly efficient once active, we could
surmise that a more volatile fuel combusts more readily during the first several seconds after a
cold start, and thus causes the catalyst to become active earlier than with less volatile fuels.
Once the catalyst is active, subsequent changes in combustion behavior and engine-out emissions
due to distillation properties may be much less important than in older vehicles where after-
treatment became active later in the cycle. The present study shows little or no effect of T90 on
NO*, which is reasonable considering thatNO* production and control are largely influenced by
release of energy as heat. As T90 represents the less volatile fractions composing the "higher
end" of the distillation curve, it represents to a lesser degree the volatile fractions primarily
combusted under cold-start conditions, and thus less energy during starts, than T50, which
represents the center of the distribution curve.
Consistent with the older studies described, the current study suggests that reducing T50 will
reduce both start and running HC, for all four species modeled, which can be plausibly attributed
to more rapid and complete evaporation and combustion of the bulk of the fuel mixture during all
phases of combustion. For T90, the present study is consistent with older data during hot-
running operation, where reducing T90 reduces HC emissions for all four species, although in
varying degrees. In addition, for cold-start emissions, the models for THC, NMHC and NMOG
suggest that the distillation parameters interact with ethanol so as to reinforce the effects of both
(both linear-effects and interaction coefficients are positive in all cases). This situation highlights
the fact that combustion is quite complicated on a microscopic level, involving complex
interactions of multiple fuel parameters and other dynamic phenomena occurring in rapidly
changing conditions during the first several seconds of engine operation.
9.1.3.4 Vapor Pressure
Fuel vapor pressure (as RVP or DVPE) primarily affects evaporative emissions (both direct and
permeation), but it has been studied in some exhaust programs as well. The literature reports that
a program conducted in the late 1980s found lower RVP associated with reduced exhaust
233
-------
hydrocarbon emissions, but that other studies since conducted since have found variable or non-
significant effects. For NOX, the literature shows little or no effect of RVP.
The present study confirmed the previous findings of no (linear) effect of RVP on NO*
emissions; however, the coefficients for the HC species are consistently negative for both cold-
start and hot-running emissions, suggesting increased HC emissions with reduced RVP (other
properties constant), a trend apparently opposite to that described for hydrocarbons in the older
studies, although concurrent fuel property changes in the cited studies have potential to confound
this comparison. In the current study, the direction of the RVP coefficients seems direct!onally
consistent with the positive T50 coefficients, suggesting that less volatile fuel (higher T50, lower
RVP) would tend to show higher hydrocarbon emissions.
9.2 Speciated Hydrocarbons and Air Toxics
Summary results for several compounds in Bags 1 and 2 are presented below. We refer the
reader to several tables, previously presented, for information on samples and model fitting
approaches. Table 79 shows sample sizes, including numbers of total and censored
measurements. Table 80 presents the notation used to identify each model term, and its level of
standardization (one- or two-stage). Table 81 summarizes model-fitting approaches, for all
models fit.
Sets of model coefficients for best-fit models are presented in Table 90 and Table 91. Detailed
results, including models fit, fitting histories, coefficients and tests of effect are presented in
Appendices K to Q.
In reviewing the coefficients for cold-start emissions in Bag 1 (Table 90), it is interesting to note
that the additional terms not included in the optimized design are not retained in best-fit models,
with the exception of T50xT90 for acetaldehyde, formaldehyde and acrolein. In reviewing tests
of effect for the full models (Table 84, Table 87), the additional parameters are often estimated
with low precision. This result is not unexpected, given that the matrix was optimized to
maximize the precision of estimation for those effects included in the design.
The model results reflect the study design applied to each compound as well as the underlying
physico-chemical processes. The reduced model structures are more complex for those
compounds fit with the full design, specifically the aldehydes, acrolein and ethanol.
Ethanol. The ethanol coefficients are positive and large for the aldehydes, acrolein and ethanol.
For acetaldehyde and ethanol, the ethanol effects are clearly dominant. These results are not
surprising, given the structural affinity between acetaldehyde and ethanol, and that the strongest
indicator of ethanol in the exhaust is ethanol in the fuel. For formaldehyde and acrolein, the
ethanol coefficients are important but not as dominant. Neither benzene nor 1,3-butadiene retain
ethanol coefficients in their reduced models. All compounds except formaldehyde retain large
and negative etOHxetOH quadratic terms, which are clearly required to fit the downward
234
-------
curvature in the logarithmic trends. See Figure 92 for acetaldehyde, Appendix M.I for acrolein
and Appendix N.I for ethanol.
Aromatics. In contrast to ethanol, the aromatics coefficients are small for the aldehydes, although
several times stronger for acrolein. Ethanol does not retain an aromatics term in its reduced
model. Not surprisingly, the aromatics coefficient for benzene is large, and the only effect
retained in the reduced model (Note that fuel benzene is also a strong predictor of exhaust
benzene, but was not a target study parameter). The two aldehydes retain small but significant
reinforcement interactions between aromatics and ethanol.
RVP. The sign and strength of RVP coefficients are similar for all four compounds fit under the
full design (but absent for those fit under the reduced design). As with the RVP terms in the
models for aggregated hydrocarbons (THC, NMHC and NMOG, Table 88, page 230), the sign of
the RVP linear effects are negative. The magnitudes for the individual species are also similar in
size to those for the aggregate HC (-0.04 to -0.06). The interaction between ethanol and RVP is
retained only in the acetaldehyde model, in which it is positive and small.
T50. For the four compound fit under the full design, linear-effect coefficients for T50 are
positive. However, the pattern in the size of the coefficients mirrors that for ethanol, in that the
two compounds with largest ethanol coefficients (acetaldehyde and ethanol) have smaller T50
coefficients than formaldehyde and acrolein, which have T50 coefficients about twice as large.
These results may reflect similarities in structure between the two pairs of compounds, or
similarities in formation processes during combustion. In addition to large linear coefficients,
formaldehyde and acrolein have small interference interactions between T50 and ethanol.
T90. More so than for the other properties, linear coefficients for T90 differ among the
compounds fit under the full design. The coefficients for acetaldehyde, formaldehyde and
acrolein are positive, but increasing, respectively, with the values for formaldehyde and acrolein
approximately 3 and 8 times larger than that for acetaldehyde. In contrast, the coefficient for
ethanol is negative, suggesting reduced ethanol emissions for less volatile fuels. In addition to
large linear effects for ethanol and T90, formaldehyde and acrolein have small reinforcement
interactions between these properties.
The structures for reduced models are much simpler for benzene, 1,3-butadiene and ethane,
reflecting the limits imposed by the reduced design. It is clear that in model fitting for these
compounds, that only strong effects appear significant and are hence retained in the reduced
models.
Covariance Parameters. The table also includes the two covariance parameters fit by the mixed
model, or recalculated manually for the Tobit models. The "vehicle" component reflects the
variance among vehicles, or the "between-vehicles" variance. It represents the variance of a
normal distribution with mean 0, i.e., the random scatter of individual vehicles around the mean
for all vehicles. The "residual" component represents random error, unexplained by the model
235
-------
after accounting for fuel-parameter and vehicle effects. The relations between these two
components are compound specific. For acetaldehyde, both components are relatively low and
almost equal in size. Formaldehyde has a larger vehicle variance, whereas acrolein and ethanol
show less variance between vehicles and larger residual error variances.
Corresponding sets of coefficients for the Bag 2 models are shown in Table 91. As with two of
the Bag 1 models, these models reflect the simplicity of the reduced design. As mentioned, it was
not possible to fit RVP effects for any compound, nor did we attempt to fit quadratic or
interaction terms. For acrolein, benzene and 1,3-butadiene, no model fitting was attempted,
given the high rates of censoring for these compounds (Table 79). Intercepts are much lower than
in Bag 1, showing the much lower emission levels during hot-running operation. Models for
acetaldehyde and ethanol are reduced to ethanol effects, whereas that for formaldehyde includes
a negative term for T90. The relative sizes of the ethanol effects for the aldehydes and ethanol in
Bag 2 are similar to those for Bag 1, i.e., ethanol has the largest effect, followed by acetaldehyde
and formaldehyde. Residual error variances in Bag 2 are consistently higher than in Bag 1,
showing the expected result that relative variability is considerably higher for running emissions
in Bag 2 than in Bag 1 which is dominated by the start increment.
Table 90. Coefficients for Reduced Models, by Compound, for Bag 1 (Cold-start Emissions).
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RW
etOH x T50
etOH x T90
Notation
Intercept
Ze
Za
Zr
25
7-9
ZZee
ZZ55
2Zea
z,z,er
ZZe5
ZZeg
Com]
Acetaldehyde
-5.2323
0.81449
0.03483
-0.04170
0.08670
0.03801
-0.1669
0.06665
0.01840
0.02194
Formaldehyde
-5.9771
0.2299
0.02822
-0.04718
0.1672
0.1302
0.05262
0.01651
-0.01627
0.02004
Acrolein
-7.9338
0.2476
0.1122
-0.06450
0.1880
0.2489
-0.08310
-0.1186
0.04617
sound
Ethanol
-4.9080
1.4627
-0.06054
0.07029
-0.09923
-0.4970
0.1108
Benzene
-4.1074
0.4032
1,3-
Butadiene
-5.8365
Ethane
-4.3412
0.05222
-0.1925
0.1334
0.09828
0.1830
Arom x RVP
Arom x T50
Arom x T90
T90 x T90
T50 x T90
RVP x T90
77
t^^ar
7-.ZaS
2Zag
ZZpp
ZZ.5P
ZZrp
0.03959
0.03489
0.05986
Vehicle
residual
0 veh
02s
0.1149
0.0885
0.3358
0.1407
0.1032
0.3629
0.1283
0.5730
0.2739
0.1896
0.0616
0.0690
0.2454
0.0296
236
-------
Table 91. Coefficients for Reduced Models, by Compound, for Bag 2.
Model term
Intercept
etOH
Arom
T50
T90
Notation
Intercept
zs
Za
zs
Z9
Com]
Acetaldehyde
-9.4192
0.1910
Formaldehyde
-8.6574
0.07804
-0.08322
Acrolein
NO
MODEL
sound
Ethanol
-9.5634
0.8163
Benzene
NO
MODEL
1,3-
Butadiene
NO
MODEL
Ethane
-7.7241
-0.1092
0.1452
0.1270
Vehicle
residual
0 veh
02s
0.05372
0.4153
0.08239
0.3776
0.4634
1.1682
2.6669
0.1517
9.2.1 Model Fitting under the Reduced Design
In section 8.2 above, we described the evaluation of the G-efficiency for the reduced design.
The conclusion was that the efficiency of the reduced design was adequate to allow modeling
based on a 4-term full model (Model l.c, Table 78, page 189). During analysis, it is possible to
follow the preliminary evaluation of design efficiency with a more direct assessment of the
adequacy of the reduced design in model fitting. Specifically, for selected compounds measured
under the full design, it is possible to compare model coefficients for models fit under both full
and reduced designs.
The secondary evaluation is useful because in Bag 1, three compounds were measured a variant
of the reduced design. Benzene, 1,3-butadiene and ethane were measured on the reduced fuel set
(11 fuels), but on the entire set of fifteen vehicles.
We performed these analyses for NMOG (Bag 1) and (Bag 2), as well as Ethane (Bag 1),
Acetaldehyde (Bag 1) and Formaldehyde (Bag 1).
9.2.1.1 NMOG (Bag 1)
To more directly evaluate the utility of the reduced design in estimating fuel property effects, we
fit models for NMOG (Bag 1) using the full design and the reduced design as applied to benzene,
1,3-butadiene and ethane.
The results, shown in Table 92 and Figure 105, suggest that the fuel effects estimated from the
reduced design (with 15 vehicles) are in agreement with those estimated from the full design.
For the four fuel-property terms, the mean coefficients for the reduced design fall within the 90%
confidence intervals for the full-design values. Results for both levels of the design also suggest
that were model fitting performed, the T90 term would be dropped from the reduced model,
while the remaining three linear effects would be retained. Not unexpectedly, the figure shows
that the uncertainty of coefficients is larger for the reduced design than for the full design,
reflecting the higher G-efficiency of the full design.
237
-------
Table 92. NMOG (Bag 1): Coefficients for Four-term models fit using Full and Reduced designs.
Model Term
Intercept
etOH
Arom
T50
T90
Vehicle
residual
Notation
Intercept
Ze
Za
z,
Z9
O veh
02s
Full design1
Full3
-0.9548
0.08786
0.08325
0.1611
0.009286
0.1222
0.07989
Reduced Design
(15 veh.)2
Full4
-0.8943
0.1040
0.09435
0.1527
0.02127
0.1091
0.08907
Best fit
-0.8939
0.1020
0.09193
0.1518
0.1090
0.08948
Reduced Design
(5 veh.)3
Full"
-0.9987
-0.00148
0.04899
0.07441
-0.09081
0.07728
0.05717
1 Includes 1 5 vehicles and 27 fuels, with replication.
Includes fifteen vehicles measured on 1 1 fuels.
3 Includes five vehicles measured over 1 1 fuels.
4 All terms highly significant, except for T90.
5 All terms significant, except for ethanol.
Figure 105. NMOG (Bag 1): Coefficients for fuel-property effects for model fits using full and reduced
designs (as defined in Table 92) (Error bars represent 90% confidence intervals).
0.20
0.15
Cold-Start (Bag 1)
full design
reduced design (15 veh.]
reduced design (5 veh)
9.2.1.2 NMOG (Bag 2)
We repeated these steps for NMOG (Bag 2). However, in this case we applied the variant of the
reduced design applied in the Bag 2 measurements. Accordingly, these models were fit using
measurements for 11 fuels on five vehicles. For the "full design" the results represent the
measurements on 13 vehicles on 27 fuels, with replication. The total included 13, rather than 15
238
-------
vehicles, because we did not include the Odyssey or Sienna, as previously described (see 6.1.3,
page 122).
For the full dataset, coefficients for all four terms are highly significant. Thus in this case, the
"full model" coincides with the "best-fit" or reduced model. However, model fitting with the
reduced design gives a different picture. The lower efficiency of the reduced design is apparent
in margins of error roughly twice as large as for the full design (Figure 106). If model fitting is
performed for the reduced design, using the same criteria applied for the full design, a single
term, T90 is retained in the best-fit model. It is clear that a reduced model under the reduced
design does not give estimates of fuel effects similar to the corresponding reduced model under
the full design.
Table 93. NMOG (Bag 2): Coefficients for Four-term models fit using Full and Reduced designs.
Model Term
Intercept
etOH
Arom
T50
T90
Vehicle
residual
Notation
Intercept
Ze
Za
25
Z9
02veh
02s
Full design1
Full3
-5.2388
0.04792
0.03600
0.08332
0.06919
0.8458
0.1348
Reduced
Design2
Full
-4.7775
0.01778
0.03320
0.04258
0.09051
1.1405
0.1026
Best fit
-4.7768
0.08393
1.1448
0.1052
1 Includes 1 3 vehicles and 1 1 fuels, with replication.
2 Includes five vehicles and 1 1 fuels.
3 All terms highly significant; full model also qualifies as "best fit."
239
-------
Figure 106. NMOG (Bag 2): Coefficients for fuel-property effects for model fits using full and reduced
designs (as defined in Table 92) (Error bars represent 90% confidence intervals).
0.20
0.15
-0.05
-0.10
Hot-Running (Bag 2)
full design [best fit]
reduced design (fullmodel)
9.2.1.3 Aldehydes
We performed similar analyses for the two aldehydes measured in the program. Because Bag 1
emissions were measured under the full design, they also provided an opportunity to compare
modeling results obtained under the reduced design to corresponding results under the full
design.
For acetaldehyde (Bag 1) we included a fifth term (etOHxetOH) in the full model (see Table 94).
We made this exception because of the marked curvature in the In(acetaldehyde) vs. ethanol
trend, which requires inclusion of the quadratic term to get satisfactory model fits (see Figure
93). For the full design, the full five-term model also qualifies as a "reduced" model as all terms
are significant. For the reduced design, the reduced model contains two fewer terms than the full
model. The T50 linear effect is marginally significant at the 90% confidence level and the T90
linear effect is much smaller than its standard error (see Figure 107).
Relative to the full design, the reduced design estimates the dominant effects to within margins
of 14% for the ethanol linear term, and -3% for the ethanol quadratic term. Differences are
larger for the remaining effects, ranging from -50% for T50 to +280% for aromatics.
240
-------
Table 94. Acetaldehyde (Bag 1): Coefficients for Five-term models fit using Full and Reduced Designs.
Model Term
Intercept
etOH
Arom
T50
T90
etOHxetOH
Vehicle
residual
Notation
Intercept
Zs
za
Z5
Z9
zzee
0 veh
028
Full design1
Full3
-5.1667
0.7976
0.01667
0.1181
0.04180
-0.1823
0.1140
0.1160
Reduced Design2
Full
-4.9682
0.6862
0.06360
0.05774
-0.00816
-0.1882
0.05896
0.1027
Best fit
-4.9694
0.6397
0.08083
-0.2289
0.05915
0.1052
Includes 1 5 vehicles and 27 fuels, with replication.
2 Includes five vehicles and 1 1 fuels.
3 All terms significant; full model also qualifies as "best fit."
Figure 107. Acetaldehyde (Bag 1): Coefficients for fuel-property effects for model fits using full and
reduced designs (as defined in Table 92) (Error bars represent 90% confidence intervals).
Similar results for formaldehyde are shown in Table 95 and Figure 108. The full design has
significant effects for all properties except aromatics, which is marginally significant at the 90%
confidence level. For the reduced design, all four effects appear significant. All effects agree in
sign between the two designs, but differ in size. The reduced-design effect is larger for
aromatics, but smaller for the remaining three effects. Coefficients for ethanol, T50 and T90
241
-------
from the reduced design are 32%, 41% and 68% smaller than their counterparts from the full
design, respectively.
Table 95. Formaldehyde (Bag 1): Coefficients for models fit using Full and Reduced Designs.
Model Term
Intercept
etOH
Arom
T50
T90
Vehicle
residual
Notation
Intercept
Ze
za
Z5
Z9
0 veh
028
Full design1
Full
-5.9671
0.2524
0.01231
0.2039
0.1315
0.3351
0.1690
Best fit
-5.9671
0.2507
0.2018
0.1313
0.3353
0.1695
Reduced Design2
Full3
-5.7087
0.1720
0.04935
0.1199
0.04251
0.2196
0.1216
Best fit
Includes 1 5 vehicles and 27 fuels, with replication.
2 Includes five vehicles and 1 1 fuels, without replication.
3 All terms significant; full model also qualifies as "best fit."
Figure 108. Formaldehyde (Bag 1): Coefficients for fuel-property effects for model fits using full and
reduced designs (as defined in Table 92) (Error bars represent 90% confidence intervals).
-0.10
242
-------
9.2.2 Models selected for Application
Results presented the sub-section 9.2.1 above raise questions about the adequacy of the reduced
design to support model fitting, depending in some degree on the number of vehicles included in
the subsamples of data.
In Bag 1, where data for all 15 vehicles is available for the reduced subset of fuels, the results for
NMOG suggest that the reduced design with all vehicles can estimate fuel property effects that
are comparable to those estimated under the full design (Figure 105).
However, results also suggest that the reduced design with only five vehicles is not adequate to
generate estimates fuel property effects comparable to those from the full design. The main
reason for this conclusion is that reduced, or "best fit" models lack terms that the full design
indicates should be present. This point is illustrated by the results for NMOG (Bag 2) shown
above in Table 93 and Figure 106. The best-fit model under the reduced design lacks terms for
ethanol, aromatics and T50, retaining only a single term for T90. However, in the corresponding
models fit under the full design, these terms are present and significant.
Results for aldehydes in Bag 1, in which models fit under the full and reduced designs can be
compared, give a similar impression. The best fit model for acetaldehyde (Figure 107) lacks
terms for T50 and T90 that are present in the full-design model. By contrast, the reduced model
for formaldehyde (Figure 108) includes an aromatics term not present in the best-fit model under
the full design. For the bag-1 aldehydes, the models under the reduced design may be better off
than corresponding models in Bag 2, in that they contain replicate measurements lacking in Bag
2. In Bag 1, the presence of the replicates roughly doubles the number of measurements
available, which may improve the models' precision and power.
Overall, it appears that model fitting under the reduced design is prone to errors in which terms
that retained under the full design are rejected as insignificant. From these results we conclude
that best-fit models under the reduced design are not adequate to represent the behavior of
emissions in response to changing fuel properties.
An additional consideration is that the fuel effects estimated under the reduced design with five
vehicles can differ from those estimated under the full design by margins ranging from 30% to
300%. This result suggests that coefficients estimated from the reduced design are subject to
error in terms of magnitude. However, for compounds measured only under the reduced design,
the available measurements comprise the best information available relating emissions to fuel
properties.
Thus, for purposes of application, we have elected to use full rather than reduced models for the
compounds included in this analysis and measured under the reduced design, including
acetaldehyde, formaldehyde, acrolein, benzene, 1,3-butadiene and ethane. Coefficients for Bag-
243
-------
1 models, representing cold-start emissions, and Bag-2 models, representing hot-running
emissions, are presented below in Table 96 and Table 97, respectively.
For compounds in Bag 1 measured under the full design, reduced models are retained for
application. Coefficients for these models, previously presented in Table 90 (page 236), are
summarized in Table 98.
Table 96. Coefficients for Full Models, for Bag 1, for Compounds measured under the reduced Design1.
Model term
Intercept
etOH
Arom
T50
T90
Notation
Intercept
Ze
Za
Z5
Z9
Model
Benzene
-4.1029
-0.00468
0.4056
0.04242
0.01133
1,3-Butadiene
-5.8371
-0.01729
0.02673
0.1247
0.1004
Ethane
-4.3079
0.1204
-0.1728
0.2169
0.09531
Vehicle
residual
0 veh
028
0.2741
0.1873
0.2192
0.1089
0.1407
0.04970
For these models, "reduced design" signifies 1 5 vehicles measured on 1 1
fuels.
Table 97. Coefficients for Full Models, for Bag 2, for Compounds measured under the Reduced Design1
Model term
Intercept
etOH
Arom
T50
T90
Notation
Intercept
Ze
Za
zs
Z9
Com]
Acetaldehyde
-9.4189
0.1520
0.07991
-0.02997
-0.07836
Formaldehyde
-8.6574
0.08456
0.01575
0.01863
-0.08138
Acrolein
NO
MODEL
sound
Ethanol
-9.3072
0.9233
-0.3772
-0.01910
-0.3017
Benzene
NO
MODEL
1,3-
Butadiene
NO
MODEL
Ethane
-7.7241
0.07345
-0.1260
0.1815
0.1322
Vehicle
residual
0 veh
02s
0.05654
0.3814
0.08205
0.3762
0.3707
1.0889
2.6785
0.1458
1 For these models, "reduced design" signifies 5 vehicles measured on 1 1 fuels.
244
-------
Table 98. Coefficients for Reduced Models, for Bag 1, for Compounds measured under the full Design1
Model term
Intercept
etOH
Arom
RVP
T50
T90
etOH x etOH
T50 x T50
etOH x Arom
etOH x RW
etOH x T50
etOH x T90
Notation
Intercept
Ze
Za
Zr
25
7-9
ZZee
ZZ55
77
^^ea
zzer
ZZe5
ZZep
Compound
Acetaldehyde
-5.2323
0.81449
0.03483
-0.04170
0.08670
0.03801
-0.1669
0.06665
0.01840
0.02194
Formaldehyde
-5.9771
0.2299
0.02822
-0.04718
0.1672
0.1302
0.05262
0.01651
-0.01627
0.02004
Acrolein
-7.9338
0.2476
0.1122
-0.06450
0.1880
0.2489
-0.08310
-0.1186
0.04617
Ethanol
-4.9080
1.4627
-0.06054
0.07029
-0.09923
-0.4970
0.1108
Arom x RVP
Arom x T50
Arom x T90
T90 x T90
T50 x T90
RVP x T90
77
t^^ar
ZZ.a5
7-Zag
ZZpp
ZZ.5P
ZZrp
0.03959
0.03489
0.05986
Vehicle
residual
O veh
O's
0.1149
0.0885
0.3358
0.1407
0.1032
0.3629
0.1283
0.5730
1 For these models, "full design" signifies 15 vehicles measured on 27 fuels.
245
-------
10 References
1 USEPA Office of Mobile Sources. Final Regulatory Impact Analysis for Reformulated Gasoline. EPA-
420-R-93-017. Assessment and Standards Division, Ann Arbor, MI. December, 1993.
2 Technical Support Document: Analysis of California's Request for Waiver of the Reformulated Gasoline
Oxygen Content Requirement for California Covered Areas. EPA420-R-01-016 (see also 40 CFR
80.45).
3 Auto/Oil Air Quality Improvement Research Program. AQIRP Program Final Report. January 1997.
4 USEPA Office of Transportation and Air Quality. Assessing the Effect of Five Gasoline Properties on
Exhaust Emissions from Light-Duty Vehicles certified to Tier-2 Standards: Pilot Studies (EPAct Phases 1
and 2, and FTP Pilot). Memorandum to Docket EPA-HQ-OAR-2011-0135. Assessment and Standards
Division, Ann Arbor, MI. March, 2013.
5 USEPA Office of Transportation and Air Quality. EPAct/V2/E-89: Assessing the Effect of Five Gasoline
Properties on Exhaust Emissions from Light-Duty Vehicles certified to Tier-2 Standards : Final Report
on Program Design and Data Collection. EPA-420-R-13-004. Assessment and Standards Division, Ann
Arbor, MI; National Renewable Energy Laboratory, U.S. Department of Energy, Golden, CO;
Coordinating Research Council, Alpharetta, GA. March, 2013.
6 CRC report No. 648 (2006 CRC Hot-Fuel-Handling Program), Coordinating Research Council,
Alpharetta, GA.
7 Kuhfeld, Warren F. Efficient Experimental Designs Using Computerized Searches. Sawtooth Software
Research Paper Series. Sawtooth Software, Inc. Sequim, WA. 1997.
8 Neter, J.; Kutner, M.H.; Nachtsheim, C.J.; Wasserman, W. 1996. Applied Linear Statistical Models.
Fourth Ed., Irwin, Chicago.
9 Gunst, Richard. F. Statistical Analysis of the Phase 3 Emissions Data Collected in the EPAct/V2/E89
Program: Final Report. Subcontract LGC-0-40441-01. National Renewable Energy Laboratory,
Department of Energy, Golden, CO. July 2011.
10 Sluder, C. Scott; West, Brian H. 2011. NMOG Emissions Characterizations and Estimation for
Vehicles using Ethanol-blendedFuels. ORNL/TM-2011/461. Fuels, Engines and Emissions Research
Center, Oak Ridge National Laboratory, Oak Ridge, TN. October 15, 2011.
11 Clarke, Joan U. 1998. Evaluation of censored data methods to allow statistical comparisons among
very small samples with below detection limit observations. Environmental Science and Technology. 32:
117-183.
12 Painter, L.J.; J. A. Rutherford. 1992. Statistical Design and Analysis Methods for the Auto/Oil Air
Quality Research Program. SAE 920319. Society of Automotive Engineers, Warrendale, PA.
13 Zhao, Yuchao; Frey, H. Christopher. 2004. Quantification of variability and uncertainty for censored
data sets and application to air toxic emission factors. Risk Analysis 24(4): 1019-1034.
14 Maddala; G.S. Introduction to Econometrics. Macmillan, New York. 1998. (p. 283 ff.).
15 SAS Institute, Inc. SAS/STAT User's Guide, Version 6, Fourth Edition: Volume 2. Gary, NC. 1990.
(PROC LIFEREG).
16 Schabenberger, O. Mixed Model Influence Diagnostics. Paper 189-29 In: SAS Institute, Inc.
Proceedings, Twenty-Ninth Annual SAS Users Group International Conference (SUGI 29), Montreal,
Canada May 9-12, 2004.
246
-------
17 Mason, Robert L.; Buckingham, Janet P. Re-design of Fuel Matrices for Epact Program.
Memorandum for Contract EP-C-07-028. Southwest Research Institute, Austin, TX. September, 2008.
(Appendix A to Testing Report, citation 5).
18 Lewis, Jeffrey B.; Linzer, Drew A. 2005. Estimating regression models in which the dependent
variable is based on estimates. Political Analysis 13(4): 345-364.
19 Maddala; G.S. Introduction to Econometrics. Macmillan, New York. 1998. (p. 283 ff.).
20 SAS Institute, Inc. SAS/STAT User's Guide, Version 6, Fourth Edition: Volume 2. Gary, NC. 1990.
(PROC LIFEREG).
21 Kish, Leslie. Survey Sampling. John Wiley & Sons, New York. 1965. (page 29).
22 Hochhauser, A.M. (2008). Review of Prior Studies of Fuel Effects on Vehicle Emissions. Coordinating
Research Council, Alpharetta, Georgia. CRC Report No. E-84 (www.crcao.org).
23 USEPA Office of Transportation and Air Quality. Renewable Fuel Standard Program (RFS2):
Regulatory Impact Analysis. EPA-420-R-10-006. Assessment and Standards Division, Ann Arbor, MI.
February, 2010. (Chapter 3).
24 Aikawa, K.; Sakurai, T.; Jetter, J. 2010. Development of a predictive model for gasoline vehicle
particulate matter emissions. SAEInt JFuels Lubr. Vol. 3, Paper No. 2010-01-2115.
247
------- |