A STUDY OF THE EPA MOTOR VEHICLE
~
CERTIFICATION PROCESS
T. Doyle
R. Farrell
A. Frazier
W. Kinley
G. Miller
26 October 1977
Submitted to:
Environmental Protection Agency
Office of Mobile Source Air Pollution Control
Contract Number 68-03-2466
REPORT NUMBER
Vector Research, Incorporated
VRI EPA-2
FR 77-1
Ann Arbor, Michigan
-------
EXECUTIVE SUMMARY
A preliminary analysis of the procedures used by the Environmental
Protection Agency (EPA) to certify motor vehicle compliance with the Clean
Air Act revealed a number of activities which might be made more cost effec-
tive. A subsequent investigation, reported on here, focused on three of
these activities: (1) EPA's classification of the manufacturer's model line
into engine families; (2) EPA's identification of "worst case" prototype
vehicle configurations from each of these families* and (3) EPA's decisions
regarding whether to test certain vehicles. The analysis of these activities
concentrated on EPA procedures designed to assess the initial emission levels
of vehicles and did not examine the components of these activities concerned
with estimating the deterioration in these emissions with time or vehicle
usage. The following paragraphs provide a brief description of each of these
activities and the results of the analysis performed on them.
Engine Family Clasaifioation
EPA classifies each manufacturer's model line into engine families to sepa-
rate vehicles with one set of engine and control system parameters from those
with another. The vehicle parameters used 1n this sorting process are selected
under the assumption that they provide the significant determinants of a
vehicle's lifetime emissions. Thus, vehicle configurations within one engine
family should be more similar in emission characteristics than configurations
classed in different families. Assuming the classification scheme performs
-------
i i
the appropriate segregation, EPA's certification of emission level com-
pliance is then performed on an engine-family by engine-family basis.
That is, all vehicles in a family are certified if selected configura-
tions from the family comply with the standards.
If the classification scheme produces too many engine families, then
the certification process is unnecessarily costly. Conversely, a scheme
which results in too few engine family classes increases the likelihood
that EPA will incorrectly certify a vehicle configuration whose lifetime
emissions exceed the standards. Therefore, an analysis was performed to
determine whether the current classification scheme might be revised to
one which appeared to be more cost effective than the current scheme, yet
feasible to implement.
An exploration of the initial (4,000-mile) emission levels of 1977
certification vehicles revealed a greater difference among emission levels
of vehicles from different engine families than among those of vehicles
within the same family. Thus, the current engine family classification
procedure does group vehicles with similar emissions and segregate those
that are dissimilar. Further examination of the engine family classes
indicated that EPA may be able to reduce the number of engine families by
about ten percent without grouping vehicles with dissimilar initial emission
levels. This reduction would be achieved by reducing the number of para-
meters defining families from the ten or more parameters generally used
to distinguish between engine families. Further analysis of the relation-
ship between the emission deterioration and the engine family classifica-
tion procedure is necessary, however, before recommending such a reduction.
-------
iii
Prototype Vehicle Identification
Given that the classification of vehicles into families provides a
reasonable stratification of the manufacturer's vehicle fleet, EPA then
must identify the configurations from each family to be used to determine
whether all of the vehicle configurations in the family meet federal
standards. Two types of vehicles are selected: configurations with
anticipated high sales are chosen first and then configurations expected
to have the greatest chance of exceeding emission standards are selected
from the remaining members of the family. Identifying the first type 1s
relatively straightforward, requiring limited EPA effort.
The identification of potential "high emitters", however, is a diffi-
cult and time consuming task. The rationale for identifying these vehicles
is that all vehicle configurations within an engine family will by defini-
tion comply with the standards if it can be demonstrated that the "highest
emitters" within that family meet the standards. If, however, the process
of identifying these vehicles does not in fact produce the "high emitters",
then the EPA effort dedicated to this task should be re-evaluated.
An extensive analysis was conducted to compare the emission levels of
vehicles selected as potentially high emitters to those selected on the basis
of a high sales criterion. Although there 1s a clear difference among the
emission levels of Individual vehicles within an engine family, no signifi-
cant difference could be found between emissions of the vehicles 1n the high
sales category and those of vehicles in the "high emitter" category. The
likelihood of exceeding the standards, as well as the average and vari-
ability of the emission levels of vehicles 1n one category were not dis-
tinguishable from those in the other.
-------
iv
Given the evidence that vehicles with high sales configurations are
not necessarily high emitters, the above results suggest that EPA should
re-evaluate and alter current procedures for identifying prototype configu-
rations. For example, vehicles could be selected at random from each family,
or the process of selecting any remaining high emitter vehicles could be
refined, or both. A random selection scheme could be fully random, or it
could employ a form of bias (such as selecting high sales vehicles with a
greater probability than others). The judgmental procedures could be im-
proved by augmenting the knowledge used to relate emissions to vehicle
specification with data collected during certification. That is, a greater
reliance could be placed on a continued analysis of the current model year
data (e.g., low mileage deterioration vehicle results) and historical
manufacturer and EPA data collected on the emission levels of similar vehicles
from previous years.
The most promising alternative appears to be one which would be expected
to improve judgmental procedures (such as an increased analysis of historical
data) and utilize a random sampling mechanism in areas where use of judgment
appears to provide little advantage. For example, the decision of whether
judgment or random sampling is to be used in a particular engine family
could be based on the likelihood that the family will have low or high emis-
sion levels, with judgment being reserved for the families falling in the
latter category. That is, judgment should be at least as good as random
sampling and may, in some instances, be clearly superior. Therefore, when
used, it should be applied to selecting vehicles from engine families which
are likely to contain failing configurations, i.e., families with a previous
history, early deterioration results, or other evidence that suggests they
-------
V
contain such configurations. Alternatively, both selection techniques could
be used on most families, permitting EPA to evaluate engineering judgment
further as a method for prototype vehicle identification.
Testing Strategies and Certification Criteria
Once EPA selects a vehicle for testing, it then must determine
(1) whether to test the vehicle, (2) if tested, how many times to test
it, and (3) how to use the test results to determine whether to certify
its engine family. Under the current procedure, EPA tests all prototype
vehicles selected from an engine family, and certifies the family if
every vehicle tested in the family complies with all standards. If a
particular vehicle fails the test, the manufacturer may request a retest,
in which case the results of the second test are used in the certification
decision.
EPA's current decision to test a vehicle is generally independent of
the test results of other vehicles, thus reducing the organization's
flexibility to adapt its testing strategy with knowledge about the emissions
performance of similar vehicles tested previously. In the current procedure,
EPA tests all prototype vehicles in an engine family irrespective of the
test results from the first vehicle tested in that family. A more adaptive
approach to EPA's testing program would be a sequential testing strategy.
Using such an approach, EPA would determine the desirability of subsequent
tests based on initial vehicle test results. If tests on one or two pro-
totypes in a family resulted in relatively low emissions, it might be
desirable to certify the family without further testing.
-------
vi
A broad class of such sequential testing strategies has been identified
which would reduce EPA's testing workload without significantly increasing
the risk of incorrectly certifying an engine family in violation of the
standards. These strategies could save EPA between 10 and 25 percent of
its 4000-mile certification tests per year by using the test results from
one or two vehicles to determine whether further testing is necessary in a
family. In addition, if the families which are most likely to contain failing
vehicles could be identified in advance of testing, this identification could
be used either instead of or in conduction with a sequential testing strategy
to identify those families on which testing should be concentrated.
EPA's current practice of using the results of a second test for a
vehicle which fails a single test has the effect of basing certification
decisions on biased estimates of the true emissions of these vehicles. For
this reason, EPA may wish to consider a policy of averaging test results when
a manufacturer requests a retest. This policy would decrease the chances of
certifying engine families containing vehicles which exceed the standards.
However, such a policy would also cause an increase in the risk of erroneously
rejecting families. Combining this averaging with a sequential testing strategy
would allow a reduction in the risk of erroneous rejection. An alternative
policy would have EPA test every vehicle twice and average the results. However
since nearly all vehicles which pass the first test would also pass the second
one, such a policy would produce results which are essentially identical to
those produced by the policy of retesting only failed vehicles, while requiring
EPA to conduct considerably more tests.
-------
vii
Running Change Testing
In addition to selecting prototype vehicles for certification, EPA
must also decide whether changes to previously certified configurations
(i.e., running changes) may have a significant effect on emissions and must
therefore be tested to insure continued compliance with federal standards.
Since the review and testing of running changes constitute a significant
amount of the certification workload, an investigation of running change
emissions appeared to be warranted.
An analysis of the test results from running changes revealed that
the time spent making the decision to test these vehicles was for the most
part nonproductive. That is, the chance that a running change would fail
to meet the standards appeared to be independent of EPA's decision to test
(or not test) the running change. The running changes tested by EPA had
emission levels which were symmetric about and very near to the levels of
their corresponding certified configurations. Furthermore, the difference
in emissions between the certified vehicle and the running change was 1n
general less than the difference In emissions between any two vehicles
chosen at random from within an engine family. Since the EPA decision to
test a running change should reflect the likelihood the change will raise
emission levels above the standards, EPA tests should primarily be per-
formed on changes to families for which the emissions of previously-tested
vehicles exceed prespeclfied thresholds. The results of manufacturer
testing should determine whether or not other running changes are certified,
with only random review and testing by EPA.
-------
vi i i
Resource Implications
The above recommendations may require additional investigation to
determine whether operational, political, or engineering constraints would
make their implementation infeasible. Thus, the net effects of the above
recommendations, in terms of changes in resources required to operate the
certification process, depend on which recommendations are considered feasible
and, of these, which are adopted by EPA. For example, a ten percent reduction
in the number of engine families combined with a random selection of proto-
type vehicles which then used a sequential testing strategy to determine
which vehicles were tested could reduce the EPA engineering workload by about
two person years per year (approximately one year in family reduction, less
than one year in vehicle identification and approximately one-quarter of a
year using sequential testing). An additional savings of one to four engineer-
ing person years could be realized if EPA were to reduce the amount of time
reviewing running change applications as is suggested by the analysis of
running change vehicle test results. The net implications on the EPA testing
workload of adopting these recommendations would be a reduction of over 1,000
tests per year. About two-thirds of this test savings would be realized from
a reduction in the number of tests on running change vehicles and the remainder
on initial 4,000-mile certification vehicles.
-------
IX
CONTENTS
Page
1.0 INTRODUCTION 1
1.1 Organization of the Report 1
1.2 Purpose and Scope of the Study 2
2.0 ENGINE FAMILY CLASSIFICATION 7
2.1 Analysis of Family Groupings 8
2.2 Regression-Based Family Definitions 10
2.3 Heuristic Family Definitions . 11
3.0 PROTOTYPE VEHICLE IDENTIFICATION 19
3.1 Current Methods of Vehicle Identification 19
3.2 Comparison of Type A and Type B Vehicle Emissions ... 21
3.2.1 Vehicle Rejection Rate 22
3.2.2 Correlation between Pollutants . 23
3.2.3 Average Emission Levels. . . 25
3.2.4 Emission Level Variability 28
3.3 Interpretation of Results .... 29
3.4 Alternative Methods of Vehicle Identification 30
4.0 ALTERNATIVE TESTING STRATEGIES AND CERTIFICATION CRITERIA. . 37
4.1 Analysis of Emission Data Vehicle Testing Strategies
and Compliance Criteria 38
4.1.1 Current Procedure and Manufacturer Behavior. . . 40
4.1.2 Alternative Compliance Criteria 41
4.1.3 Alternative Testing Strategies 44
4.2 Analysis of Running Change Testing Strategies 50
4.2.1 Effectiveness of Current Strategy 51
4.2.2 Alternative Running Change Testing Strategies. . 54
APPENDIX A - DESCRIPTION OF VRI DATA BASE 61
A.l Data Base Development 61
A. 2 Data Base Content 63
APPENDIX B - ANALYSIS OF EMISSION TEST VARIABILITY 83
B.l Approach 84
B.2 Results 89
-------
X
CONTENTS
(concluded)
Page
APPENDIX C - ANALYSIS OF ENGINE FAMILY CLASSIFICATION 105
C.l Analyses of Variance 105
C.2 Regression Analysis 107
C.3 Heuristic Analysis 108
APPENDIX D - ANALYSIS OF ENGINEERING JUDGMENT 125
D.l Rejection Rates for Type A and Type B Vehicles 125
D.2 Correlation Among Pollutants 126
D.3 Emission Level Magnitudes for Type A and Type B
Vehicles 133
D.4 Emission Level Variability for Type A and Type B
Vehicles 140
D.5 Running Change Vehicle Emission Levels. ... 143
APPENDIX E - ANALYSIS OF ALTERNATIVE TESTING STRATEGIES 169
E.l Approach and Summary 169
E.2 Current Procedure Effectiveness 173
E.3 Alternative Compliance Criteria 175
E.4 Sequential Testing Strategies 176
E.5 Strategies Involving Modified Use of Engineering
Judgment , , . . 178
APPENDIX F - METHODOLOGY FOR EFFECTIVENESS ASSESSMENT 189
F.l The Manufacturers' Fleet 190
F.2 EPA's Prototype Vehicle Identification 192
F.3 EPA's Emissions Testing and Certification Process ... 192
F.4 Effectiveness Computations 199
APPENDIX G - DESCRIPTION OF EPA FUEL ECONOMY TESTING PROGRAM. . . 207
G.l Background 209
G.2 The Fuel Economy Testing Process. . . 211
APPENDIX H - REFERENCES 227
-------
xi
EXHIBITS
Number Title
1 Statistical Tests of Identity of Emissions Character-
istics of Families Proposed for Grouping
2 Rejection Rates for Domestic Vehicles
3 Comparison of Some Alternative Testing Strategies
A-l Numbers of Emission Data Vehicles in VRI Data Base
A-2 Selected Data from the Data Base for 1977 Vehicles
A-3 Data Base Vehicle Identification
B-l Sample Size and Distribution by Manufacturer of Vehicles
with Multiple Tests under "Identical" Conditions
B-2 Square Root of the Mean Sums of Squares for Each Model
B-3 Estimates of a Constant Test-to-Test Standard Deviation
(Model 1)
B-4 Estimates of the Coefficient of Variation of Test Results
for Each Pollutant
B-5 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for HC Using Data on 73 Domestic and
Foreign Manufactured Vehicles with Catalysts
B-6 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for CO Using Data on 73 Domestic and
Foreign Manufactured Vehicles with Catalysts
B-7 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for NO* Using Data on 73 Domestic and
Foreign Manufactured Vehicles with Catalysts
B-8 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for HC Using Data on 8 Foreign Manufac-
tured Vehicles without Catalysts
B-9 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for CO Using Data on 8 Foreign Manufac-
tured Vehicles without Catalysts
-------
xi i
EXHIBITS
(continued)
Number Title Page
B-10 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for N0X Using Data on 8 Foreign Manufac-
tured Vehicles without Catalysts 100
B-ll Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for HC Using Data on 83 Domestic and
Foreign Manufactured Vehicles with and without Catalysts 101
B-12 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for CO Using Data on 83 Domestic and
Foreign Manufactured Vehicles with and without Catalysts 102
B-13 Plot of Test-to-Test Standard Deviation Versus the Mean
Emission Level for N0X Using Data on 83 Domestic and
Foreign Manufactured Vehicles with and without Catalysts 103
C-l Number of Emission Data Vehicles and Engine Families per
Manufacturer 110
C-2 Analysis of Variance Results for Test of Family Differ-
ences under Constant Test Variability Assumption
(Model 1) 1,1
C-3 Analysis of Variance Results for Test of Family Differ-
ences tinder the Assumption that Test Variability Is
Proportional to the Mean (Model 4) m
C-4 Analysis of Variance Results for Test of Family Differ-
ences Using Test Variability from [Juneja et al.t 1977]
and Assuming Constant Variability (Model 1) 112
C-5 Histograms of Mean Engine Family Emission Levels by
Vehicle Manufacturer (with and without Application of
Deterioration Factor, df) 113
C-6 Histograms of Mean Engine Family Emission Levels for
Total of Three Manufacturers (with and without Applica-
tion of Deterioration Factor, df) 114
C-7 Mean of Engine Parameters for Chrysler 115
C-8 Mean of Engine Parameters for Ford 116
C-9 Mean of Engine Parameters for General Motors H?
-------
xiii
EXHIBITS
(continued)
Number Title Page
C-10 Summary of Stepwise Multiple Linear Regression on Engine
Family Characteristics 118
G-ll Plot of Family Means of NQX Versus Means of CO for
Chrysler Vehicles with Catalysts 119
C-12 Plot of Family Means of N0X Versus Means of CO for Ford
Vehicles with Catalysts 120
C-13 Plot of Family Means of N0X Versus Means of CO for General
General Motors Vehicles with Catalysts 121
C-14 Plot of Family Means of N0X Versus Means of CO for
Domestic Manufactured Vehicles with Catalysts 122
C-1E Grouping of Current Families Using Heuristic Classifi-
cation Criteria 123
D-l Actual and Expected Certification Disposition of 1977
Model Year Vehicles In VRI Data Base by Test Selection
Criteria 1^8
D-2 Effect of Engineering Judgment on Distribution of
Selected Vehicles if Emissions Are Negatively Correlated 149
D-3 Effect of Air Fuel Ratio on Emissions 150
0-4 Number of Engine Families with Negative Estimates of the
Correlation Coefficient 151
0-5 95 Percent Confidence Intervals for the Correlation
Coefficients 152
D-6 Correlation among Pollutants 153
D-7 Correlation among Pollutants by Inertia Weight (Domestic
Vehicles with Catalysts) 153
D-8 Correlation among Pollutants by Inertia Weight (Foreign
Vehicles With Catalysts) 154
D-9 Correlation among Pollutants by Inertia Weight (Foreign
Vehicles without Catalysts) 154
-------
XIV
EXHIBITS
(continued)
Number Title Page
D-10 Correlation among Pollutants by Inertia Weight (Domestic
and Foreign Vehicles with Catalysts) 155
D-ll Correlation among Pollutants by Displacement (Domestic
Vehicles) 156
D-12 Correlation among Pollutants by Displacement Group
(Domestic Vehicles) 157
D-13 Analysis of Variance Results Including Type A Versus B
Differences under Constant Test Variability Assumption
(Model 1) 158
D-14 Analysis of Variance Results Including Type A Versus B
Differences under the Assumption that Test Variability
Is Proportional to the Mean (Model 4) 159
D-15 Analysis of Variance Results Including Type A Versus B
Differences from Using [Juneja et al.3 19/7] Test Vari-
ability Estimates under Model Assumption 160
D-16 Normalized Sum of Mann-Whitney Test Statistic to Compare
Type A to Type B Vehicle Emissions 161
D-17 Kolmogorov-Smirnoff Test of the Sample Cumulative Dis-
tribution 161
D-18 95 Percent Confidence Interval on the Standard Deviation
of Type B Vehicles Divided by the Standard Deviation of
Type A Vehicles 162
D-19 Vehicle Emission Data Used in Analysis of Running
Changes 163
E-l Family Mean Emissions and Deterioration Factors Used in
Effectiveness Analysis 180
E-2 Test-to-Test and Design-to-Design Variability Assumed in
Effectiveness Analysis 182
E-3 Parameters of Distribution of Vehicle Emission Levels 183
E-4 Summary of Results of Effectiveness Analysis 184
E-5 Frequency Distribution of Rejection Probabilities 186
-------
XV
EXHIBITS
(concluded)
Number Title Page
E-6 Results for Several Sequential Testing Strategies 187
E-7 Representation of Hypothetical Capability to Identify
Families with High Rejection Rates 188
F-l Structure of Effectiveness Methodology 204
F-2 Expression for the Probability a Vehicle Is Not
Rejected under a Policy of Averaging on Failure 205
6-1 Fuel Economy Process and Test Procedures 206
-------
xvi
-------
1
1.0 INTRODUCTION
This document is a final report of work performed on Contract No. 68-03-
2466 by Vector Research, Incorporated, (VRI) for the Environmental Protection
Agency (EPA). The report describes the second part of a two-part study de-
signed to analyze the cost-effectiveness of EPA activities used to certify
motor vehicle compliance with federal emission standards. The first study
[VRI, 1976] was designed to assist EPA with decisions regarding efficient
utilization of its resources used to certify motor vehicle emissions in
fiscal year 1977. The second study, reported on here, examines those
activities identified in the first study which might be made more cost
effective over a longer time frame.
1*1 Organization of the Report
The material in the report is presented at three levels of detail.
The executive summary which precedes this introductory chapter contains
an overview of the study. It is intended to provide the time-constrained
reader with a description of the EPA procedures Investigated during the
study and a synopsis of the major results of this investigation. The main
body of the report amplifies and explains many of the statements found in
the executive summary. It is organized into four chapters, each of which
corresponds to a major research task of the study. A sufficient level of
detail is provided in these chapters to permit an understanding of the
general approach used in the analyses and the conclusions derived from this
work. The appendices contain further detail about the data, analytic
-------
2
methods, and results used to develop the material in the main body of the
text. The hierarchical organization of detail in the executive summary,
main body, and appendices required some repetition to facilitate presenta-
tion. Wherever possible, however, such redundant information has been
kept to a minimum.
The remainder of this introductory chapter describes the purpose and
scope of the study. The remaining three chapters describe the investigation
of three EPA certification procedures--(l) engine family classification;
(2) prototype vehicle identification; and (3) testing and certification
decision making. The techniques used in each of these investigations are
then described in the seven technical appendices, with the bibliography of
reference material provided at the end of the report.
1.2 Purpose and Scope of the Study
The purpose of the study was to analyze and make recommendations for the
conduct of EPA's motor vehicle emission certification program in order to
enhance the program's cost-effectiveness. The recommendations produced by
the study were those which appeared to provide a favorable tradeoff between:
(1) the certification resources expended by EPA (and others); and (2) the
error inherent in the certification process. The implementation of the
recommendations developed could require changes or additions to EPA regula-
tions and advisory circulars, as well as changes to current certification
practices and procedures.
The types of recommendations considered for analysis in this project
were those which would:
-------
3
(1) revise EPA's method of sorting vehicle configurations
into engine families to improve the degree to which this
classification scheme groups vehicles with similar emissions
characteristics;
(2) alter EPA's procedures for identifying which prototype
vehicles should be built by manufacturers in order to
provide a sufficient representation of the manufacturer's
fleet;
(3) result in an overall reduction in the number of EPA tests
without significantly affecting certification performance;
and
(4) institute a test averaging procedure to provide an unbiased
estimate of whether vehicles which failed and then passed an
emissions test were in compliance with the standards.
The study recommendations are based primarily upon a statistical
analysis of the exhaust emission test results collected by EPA during
its certification of 1976 and 1977 model year vehicles. As such, the
recommendations may require additional Investigation to determine whether
operational, political, or engineering constraints would make their imple-
mentation infeasible. Although some of the analyses (particularly the
investigation of alternative EPA testing strategies) considered certain
elements of these constraints, most of the results do not, in general,
reflect a detailed investigation of their impact. Therefore, adoption of
the study recommendations should only occur subsequent to an investigation
of their implications on these practical and political constraints.
-------
4
Test results from 583 vehicles manufactured by three domestic and six
foreign manufacturers were statistically analyzed to provide insight into the
effectiveness of existing EPA procedures, in terms of the tradeoff between the
resources required to certify vehicles and the risks of certifying vehicles
which do not meet standards and not certifying those that do. Most of these
analyses concentrated on the 4,000-mile test results from 335 vehicles from
the 1977 model year certification program.1 These vehicles represented 88
percent2 of the fleet of 4,000-mile light duty vehicles required by EPA to
certify three domestic (Chrysler, Ford, and General Motors) and six foreign
(BMW, Datsun, Fiat, Honda, Mazda, and Volvo) manufacturers. The analyses of
running changes examined results of tests performed by those 1976 and 1977
domestic vehicles which were used both to certify a family and determine appro-
val of a running change request. Although the 66 vehicles used iin these
analyses appeared to be representative, they comprise only a small fraction
(about ten percent) of the total number of vehicles tested in this environment.3
xThe primary reasons for concentrating on the 1977 model-year test results
were the difficulties in obtaining a consistent and comprehensive automated
data base for 1976 model-year vehicles (see appendix A for discussion).
2The remainder, twelve percent) were additions to the manufacturer's fleet
of configurations after initial certification (e.g., the Ford Fiesta). The
absence of these configurations should not, however, bias any of the study
conclusions.
3The reason for this small sample was that this was the only set of running
changes for which the before and after test results could be identified
from the data base. The inherent bias in this data set was that it included
only those running change tests performed on vehicles used in original certi-
fication. Consequently, the analysis may be biased if these running changes
were generally less extensive in nature and were less likely to influence
emission levels. However, there is limited evidence or rationale to suggest
that such a difference exists.
-------
5
In this investigation, only emission test results on emission data
vehicles (i.e., 4,000-mile vehicles) were examined. The study was not
designed to include an analysis of the emission tests used by EPA to deter-
mine the deterioration of vehicle emissions with time and vehicle usage.
Consequently, further analysis may be necessary to determine whether the
recommended changes still hold after consideration of this deterioration data.
-------
6
-------
7
2.0 ENGINE FAMILY CLASSIFICATION
Manufacturer model lines are divided into engine families to separate
vehicles with one set of engine amd control system parameters from those with
another. The parameters used in this sorting process provide a description
of the characteristics of configurations in a family and are generally
selected to provide an indication of the lifetime emission levels of vehicles
within the family. Thus, vehicle configurations within one engine family
should be more similar in emission characteristics than configurations in
different families. If this is not the case, then the engine family classi-
fication scheme may be inappropriate and more costly to EPA, either in terms
of miscertifying families or in terms of requiring an unnecessary workload,1
than sotife alternative.
Since the engine family classification 1s an important determinant of
the workload and effectiveness of EPA's program, it was considered worthwhile
to examine the degree to which vehicle configurations 1n different families
did have different emission characteristics. One approach to investigating
this question is to compare the emission levels of emission data vehicles
from different engine families.2 Although the emission data vehicles (EDVs)
are a subset of the configurations contained 1n each family, the results of
the analysis of prototype vehicle selection (see chapter 3.0) Indicate they
are representative. That is, the analysis shows that EDVs within the same
^or example, if the emission characteristics of different vehicle configurations
within an engine family are not similar, then EPA may miscertify configurations
in that family which are assumed to be similar to those tested in the certifi-
cation process. Similarly, if the emission characteristics of vehicle configura-
tions from separate families are not different, then the EPA expends additional
effort reviewing and testing more families than is necessary.
2Since the current study does not consider the deterioration of vehicle emissions,
results of this investigation apply only to 4,000-mile vehicle emission levels.
-------
8
family have very similar emission levels, even though they were selected on
two rather divergent criteria--high sales and high emissions.1 The examina-
tion of family-to-family differences in emission levels was therefore con-
ducted using the test results from the 1977 model year certification of
emission data vehicles.2
Three types of analyses were performed. The first was a statistical
comparison of the emission level variability of EDVs in the same family with
variability in emission levels across families. The second was an investi-
gation of the degree to which the values of specific vehicle parameters might
be used to predict the average emission levels of an engine family, thereby
indicating possible alternatives to the current method of grouping vehicles
into families. The last was a manual review of engine family characteristics
and vehicle emission levels within each family, to see if any regrouping or
clustering of current families might be available. The results-of these
three analyses are presented in the following sections.
2.1 Analysis of Family Groupings
To address whether current engine family classifications group vehicles
with similar emission levels, and segregate those that are dissimilar,
engine families were differentiated by domestic and foreign manufacture and
by catalyst and non-catalyst categories. The emission levels in each of these
categories were then analyzed using an analysis of variance technique.3
Although these criteria are rather divergent, they may result in the same
vehicle being chosen. However, an analysis of high sales vehicle emission
levels did not indicate that these vehicles would, in general, also be
"high emitters".
2If the EDVs cannot be considered a random sample of vehicles from an engine
family, then the results of this analysis will tend to be conservative (i.e.,
if high sales vehicles have above average emission levels, then the analysis
will tend to find similar emission levels among vehicles within a family and
dissimilar levels when comparing vehicles from different families).
3See section C.l of appendix C for details.
-------
9
The results confirmed the hypothesis that there was a significant difference
in the average emission levels between engine families. That is, the differ-
ence in emission levels between families is significantly greater than the
difference in the emission levels between vehicles in the same family.
Since only emission data vehicle tests were analyzed, it was conjectured
that this difference in the emission levels from one family to the next might
be explained by differences in the family deterioration factor (df). Since
the df is known to the manufacturer in advance, each vehicle may be calibrated
or designed so that the product of its 4,000-mile emission level and df would
achieve some pretargeted value. If this were the case, then a difference in
average family emission levels might disappear with the application of the
deterioration factor. This possibility, albeit remote, was examined by
observing the differences in the average family emission levels with and
without application of the df.1 The results (see section C.l of appendix C)
indicate that the differences in engine family emission levels cannot be
solely explained by differences in the deterioration factor.
The results of the above analyses therefore led to the conclusion that
the current method of segregating vehicles into engine families does, in
the aggregate, perform its function. Vehicles with dissimilar emission
levels are generally found in separate families. The results do not,
however, imply that the engine family classification performs the best
method of grouping.
xSince an investigation of deterioration Was outside the scope of the current
study, the purpose of this examination was only to investigate whether
differences in family means were primarily attributable to differences in
deterioration factors.
-------
10
2.2 Regression-Based Family Definitions
One approach to investigating alternative engine family classification
schemes was to analyze the relationship between the vehicle parameters of a
family and its average emission level. For each domestic manufacturer,1 a
multiple stepwise regression (see appendix C, section C.2 for details) was
performed to determine whether vehicle parameters can be used to predict the
mean emission levels for an engine family. The vehicle parameters considered
were: inertia weight, displacement, bore, stroke, rated horsepower, number
of cylinders, compression ratio, axle ratio, and timing RPM. Also considered
were the following ratios: bore divided by stroke, displacement per cylinder,
and inertia weight divided by displacement.2
The results showed that the vehicle parameters do not consistently
predict the mean emission level of engine families. Although one parameter
(timing RPM) was selected as a significant predictor of the mean emission
levels of Chrysler engine families, this parameter did not demonstrate a
similar effect on the emission levels in other manufacturer engine families.
Further regressions performed on the combined manufacturer data did not select
any parameter as a consistent predictor of pollutant levels. Thus, it was not
1Since engine families were analyzed by manufacturer, the domestic manufac-
turers were chosen because they had more engine families than did the
foreign manufacturers, as shown in appendix C. Furthermore, data from only
one test per vehicle was used to ensure that all data points within a family
had the same distribution.
2These parameters were those suggested in [Patterson, 1972] and were easily
obtained from the information available.
-------
11
possible from the data used in this analysis to develop new engine family
definitions from an analysis of the relationship between vehicle parameter
values and emission levels.
2. 3 Heuristic Family definitions
Since the above regression-based family definitions did not show any
potential for changes in family definitions, two heuristic clustering approaches
were examined. The first of these was to plot one pollutant against another to
see if any two-way clustering was evident. This analysis was performed for
three domestic manufacturers to determine if separate engine families had
visibly similar emission levels. That is, would scatter plots of the average
family emission levels for one pollutant versus another reveal clusters of
engine families? A review of such plots did not reveal any clustering with
the possible exception of CO - N0X pairings for Chrysler families.1
The second heuristic analysis was a manual review of 1977 Summary Data
Sheets. The purpose of this review was to examine subjective criteria which
grouped several existing engine families Into one family. In this analysis,
a subset of the 1977 engine families from a single manufacturer (Chrysler)
was used to develop the criteria, with the families from the other manufacturers
reserved as an independent data base on which to analyze the performance of
the new criteria 1n grouping vehicles with similar emission levels.
In the initial phase, the Chrysler families were stratified, by subjective
clustering, into three groups on the basis of three different levels of 4,000-
mile emissions. The groupings were then analyzed to identify those vehicle
xThis slight degree of clustering can be seen in the scatter plots provided
in section C.3 of appendix C.
-------
12
parameters which had similar values for families within a single grouping,
and different values for families in different groupings. The analysis
was conducted independently by three investigators, in order to ensure
both comprehensive and accurate analysis of the potential clustering
methods. No clustering of California with 49-state families was considered,
so any reduction in the number of families was limited to 49-state families.
From these analyses, only one potential method for generating broader
family definitions was identified. Specifically, the newly defined families
were those in which: (1) the displacements were within 15 percent of 50^
cubic inches of one another; (2) the types of emission control systems used
were identical; and (3) the numbers of cylinders, the numbers of carbur-
etors, and the numbers of barrels did not vary.1 Such a regrouping would
lead, for 1977 Chrysler families, to a reduction from 15 to 12 families.2
The 12 redefined families contained nine families which were identical to the
2The actual aggregation rule used initially was qualitatively phrased: the
families to be aggregated should have identical types of control systems
and identical engine configurations and carburetion and should be "similar"
in displacement. During the analysis, it was found that the 15 percent or
50 cubic inch standard which is used by EPA in family classification could
be used to define "similar" displacements. The revised criteria differ from
the present classification rules in that fewer criteria were used to distin-
guish one family from another (the Federal Register, Vol. 40, No. 126,
June 30, 1976 specifies approximately fifteen mandatory parameters to be
used in classifying configurations into engine families) and no judgment
was incorporated into the sorting process. The application of these rules
also differs from that experienced by EPA since the number of engine
families per manufacturer in this regrouping is much smaller than that
originally submitted to EPA.
2To determine which parameters defined the 15 but not the 12 families
requires investigation of the criteria used by the engineers who made the
original classification. The identification of the Chrysler engine families
aggregated in this analysis is provided in exhibit C-15, section C.3 of
appendix C.
-------
13
original set and three families which contained three pairs of six families
under the original definition.
The three pairs of combined engine families were examined quantita-
tively to analyze the emission level differences between each pair of families.
This analysis did not reveal any statistically significant difference in
either the average or the variability of emission levels for each pair of
families.1 That is, the emission levels found in each set of family pairs
were highly similar.
Thus, for the initial Chrysler sample, it was found that the 15 families
presently defined actually constituted only 12 groups for which family-to-
family differences in emission levels could be found, and that the clustering
of the 15 families into the 12 groups could be accomplished by aggregating
families using essentially the same distinguishing characteristics for families
that are now used by EPA. No other clustering mechanism which could accomplish
such grouping was found. Given this result, the next step of the analysis was
to examine the use of the clustering scheme on other manufacturers' fleets in
order to ascertain its general utility.
Exhibit 1 summarizes the results of this analysis for 90 engine families2
for 9 manufacturers. As can be seen from the table, the results for all manu-
facturers examined bore out the tentative results from the Chrysler sample-
there are clusters of families which cannot be distinguished by emission
^he statistical tests used were standard t-tests and F-tests at the one
percent level. When tested at the five percent level, one of the 18 tests
was statistically significant: if the families being combined were identical,
an average of one out of twenty such tests should be significant.
2Family count includes California families although no California aggregation
or California 49-state aggregation was performed.
-------
u
EXHIBIT 1: STATISTICAL TESTS OF IDENTITY OF EMISSION
CHARACTERISTICS OF FAMILIES PROPOSED FOR
GROUPING
Manufacturer
Original
Aggregate
Di fferences
Significant
at 1% Level
Differences
Signifi cant
at 5% Level
Number of
Stati stical
Tests
CHRYSLER
15
12
0
1
18
FORD1
22
18
0
0
24
GENERAL MOTORS2
25
22
0
0
18
BMW
3
3
-
-
-
FIAT
4
4
-
-
-
HONDA
3
3
-
-
-
DATS UN
9
7
0
0
12
VOLVO
4
4
MAZDA
5
5
-
-
-
TOTAL
90
78
0
1
72
1Some Ford engine families in the 1977 data base are identical with subsets of
others, so that uniting them would literally combine the same vehicle designs.
These cases exist because the differences in the families only affect the
vehicle after 4,000 miles (e.g., scheduled maintenance); therefore, the same EDV
test is used to certify both families. There were two sets of two families
each of this kind. They were not proposed for union in this analysis, since
they had clearly been deliberately separated.
2No useful aggregation of GM families across divisions—whether on the basis
of the present criterion or any other—was found, supporting the engineering
judgment that the divisional lines are actually quite distinct.
-------
15
characteristics and which can be defined by the engine/control system simi-
larities described earlier. Because this study did not include a statistical
analysis of the accuracy of deterioration factor measurement, it is impossible
to state whether the aggregations which were identified here involved statisti
cally significant differences in deterioration. They did involve the
aggregation of a few family pairs with what appeared to be numerically
different deterioration factors, but the variances may have been within
the statistically expected variability.
If analysis of the deterioration1 factor measurement process shows that
an additional family aggregation of the kind analyzed here does preserve
the family characteristics (within the expected statistical variability),
it appears that a reduction of about 10 to 20 percent of the number of
families could be expected.
It is difficult to determine precisely what cost savings would be
realized by eliminating 10 to 20 percent of the families. The concomitant
reduction in EPA workload would primarily beJn two areas—engineering
review and testing. Using EPA's estimate2 of the amount of engineering
manpower devoted to individual certification activities, a 20 percent
reduction in the number of families represents about a 20 percent reduction
in workload. Such an estimate is clearly tenuous since 1t presumes that
certification of every engine family takes approximately the same effort.
However, even if only a ten percent reduction in families could be achieved
by combining families, and these take approximately one-half the "normal"
^his study has been restricted to the analysis of emission data testing.
However, the final definitions of families must not only cluster the
initial emission levels (4,000-mile) of the designs involved but also
the deterioration characteristics.
2See [EPA, 1976-j].
-------
16
effort to certify, EPA could reduce its certification burden by over a
person-year of engineering effort.
Estimating the reduction in testing workload with a reduction in the
number of families is even more difficult since two original families each
having three to four emission data vehicles could be combined into a single
family having three to eight EDVs. Thus, the percent reduction in EDV testing
workload could vary from zero to the percentage reduction in the number of
families.1 Test workload savings (per durability vehicle) would be nearly
proportional to the reduction in families, since each family typically has
only one durability vehicle. A reduction in the number of families should
not influence the number of running changes proposed by the manufacturer,
hence no test savings are likely to be realized in this area. Therefore, if
analysis of vehicle deterioration permits the merging of families suggested
above, then the majority of EPA's workload savings realized will be in the
area of engineering review and analysis.
In addition to EPA's potential savings, the reduction in the require-
ment for durability vehicles and emission data vehicles would provide a
substantial savings to the manufacturers.2 It is estimated that development
xAny savings in EDV testing at EPA could be partially negated by EPA's
current policy that 25 percent of all fuel economy data vehicles (FEDVs)
be tested at the EPA facility. That is, FEDVs are essentially EDVs which
are developed solely for the fuel economy testing program. The removal
of one EDV could augment the number of FEDVs by one. Thus, for every four
EDVs removed the fuel economy program would, at most, require EPA to test
one of these as an FEDV.
2In spite of these savings, some manufacturer representatives have expressed
concern about reducing the number of engine families, probably because
certification of a larger portion of a manufacturer's fleet would depend on
test results from a smaller number of vehicles.
-------
17
of each durability vehicle costs the manufacturer between $85 thousand and
$125 thousand, while the typical cost for development of an EDV1 is between
$17 thousand and $35 thousand ([Ford, 1976] and [EPA, 1977]).
Savings to manufacturers by reducing EDVs may be negligible, since many EDVs
may have to be replaced by equivalent FEDVs to comply with fuel economy
program requirements. A rough estimate 1s that one out of every three EDVs
removed would be replaced by an FEDV.
-------
18
-------
19
3.0 PROTOTYPE VEHICLE IDENTIFICATION
The previous chapter indicated that current engine family classifica-
tion performs a reasonable job of grouping vehicles with similar emission
characteristics into the same family. This chapter investigates the effec-
tiveness of current procedures for using engineering judgment to identify
the "high emitter" vehicle configurations in each engine family. The
results suggest ways in which these procedures might be modified. The
chapter includes a general description of engineering judgment in the
context of the above activity, a description of and rationale for the
technical approach taken to analyze the outcome of decisions using such
judgment, and a presentation of the results and observations derived from
them.
3.1 Current Methods of Vehicle Identification
The emissions from an automobile are determined by a complex process
which is not completely understood. For this reason, estimates of antici-
pated emissions cannot be made via a fixed set of rules but must instead
involve professional judgment. Such judgment is used at several points in
the certification process. One of its most difficult and time-consuming
applications involves the identification of configurations which are likely
to be high emitters of pollutants and which, therefore, should be built in
advance (as prototype vehicles) for analysis prior to certification by EPA.
The rationale for selecting these vehicles is that all configurations within
an engine family will by definition meet the standards, if it can be demon-
strated that the vehicles representing configurations with the highest
emissions in the family meet the standard.
-------
20
The problem of selecting configurations which appear to have a low prob-
ability of meeting emissions standards is a complex and difficult one. Many
factors affect vehicle emissions, and a manufacturer is likely to balance
these factors against one another in designing automobiles to achieve ap-
proximately the same average emission levels for all vehicles.1 This complex
trading off of vehicle characteristics, plus the fact that vehicles with
similar engine and control system parameters have already been grouped by
engine family, makes the identification of "high emitter" configurations
(represented by type B vehicles) within each family very difficult.
In their examination of proposed vehicle specifications, the EPA
engineers review several vehicle parameters to determine if the combination
of proposed values is likely to lead to high emissions. Although a number
of parameters2 are suggested for review [Marzen, 1976], the final selection
of high emitters is left largely to the judgment of the individual engineers.
The complexity of this task is further compounded by requirements which
tend to emphasize that the type B vehicles be representative of certain seg-
ments of the family as well as the "higher emitters". The fact that this
condition exists is evidenced by EPA's requirement that each engine system
combination within a family must be represented by an emission data vehicle
and also by EPA's attempt to select EDVs so that most of the base levels in
EPA's fuel economy testing program are also represented.3 Since a large
XVRI believes that these target levels are generally approximately 60 percent o^
the emissions standards, in order to assure that the vehicles will have a high
probability of passing certification tests. Conversations with manufacturers
indicated that they concurred that a design target below standards existed;
however, no information was provided concerning its level.
2These parameters include: inertia weight class, transmission options, N/V
ratio (a combination of axle ratio and tire size), engine code (specifying
characteristics of the engine), body type, characteristics of the catalyst,
starting and shifting procedures, and road load horsepower requirements.
3See [Marzen, 1976],
-------
21
number of individuals are involved in exercising their judgment to select
a combination of "representative" and/or "high emitter" EDVs, the final
selection may depend on which engineers were tasked to identify the EDVs
from which family. Therefore, the question arises: does the effort
expended selecting the type B vehicles produce an effective result, i.e.,
are the emission levels of selected vehicles, in general, higher than the
emissions of other vehicles in the engine family?
Preliminary analyses of this question were conducted on a limited
amount of data in a previous study [VRI, 1976-a]. A test was performed
to determine whether vehicles chosen as high emitters (type B emission
data vehicles) had higher emissions than the corresponding high sales
vehicles (type A emission data vehicles) for each engine family. The
results from this analysis Indicated that the average differences in
emission levels between type A and type B emission data vehicles were
small and possibly non-existent.
3.2 Comparison of Type A and Type B Vehicle Emissions
One purpose of the current study was to provide an extension of the
previous work by analyzing in greater detail the degree to which judgment
has been used to identify vehicles with potentially high emission levels.
In this analysis, a number of statistical techniques were used to compare
the emission levels of vehicles selected as high emission vehicle (type B)
with the levels of vehicles selected as high sales vehicles (type A).
Three different sets of analyses were performed. In the first set,
the fraction of type B vehicles rejected by the certification process was
compared to that for type A vehicles in model year 1977. The second set
-------
22
of analyses compared the relative amounts of pollutant measured by the
certification tests performed on these vehicles. The last set compared
the variability of the pollutant from type A vehicles to that measured
for the type B vehicles.
3.2.1 Vehicle Rejection Rate
If the type B vehicle configurations consistently represent the higher
emitters in each engine family and the type A configurations may or may not
have emissions which fall into this higher range, then the type B vehicles
would be expected to be rejected for non-compliance more often. To deter-
mine whether this was true, the frequency of type B rejections was compared
to that for type A vehicles. Specifically, exhibit 2 shows that 8.3 percent
of 1977 emissions data vehicles from Chrysler, Ford, and General Motors did
not comply with the standards. If this percent was the same for both types
of vehicles, then 6 of the type A vehicles and 15 of the type B vehicles
would be expected to be rejected during certification. The exhibit shows
that in actuality, 5 type A vehicles and 16 type B vehicles were rejected.
EXHIBIT 2: REJECTION RATES FOR DOMESTIC VEHICLES
- . L— 1
Vehicles Tested
T
Rejected for
Non-Compliance
Percent
Rejected
All EDVs
252
21
8.3%
Type As
77
5
6.5%
Type Bs
175
16
9.1%
-------
23
A test on this data indicated that the probability of not complying
with the standards is nearly the same for both type A and type B vehicles.1
Similar tests on vehicles from individual domestic manufacturers, or on
those manufactured by foreign companies, were not possible due to the small
number of observed failures. However, the results of the combined domestic
manufacturer tests strongly indicate that vehicles selected as high emitters
are as likely to be rejected during certification for non-compliance as are
vehicles selected on the basis of projected high sales.
3.2.2 Correlation between Pollutants
Given that the rejection rates appear to be similar, the type B vehicles
may still have slightly higher emissions than the type A vehicles. To
examine this question it is necessary to determine, in general, whether
all three pollutants of a type B vehicle might be higher than those for a
type A vehicle, or whether one might be higher and another lower.
Thus, to examine the potential differences between type A and type B
vehicle emissions, two possible situations must be treated. In the first,
the emission levels of different pollutants (measured under standard test
conditions) are generally independent or show a positive association
(correlation) as vehicle designs vary. In this situation engineering
lA Chi-squared test was performed to examine whether the probability of
rejection was dependent on the type (A or B) of vehicle. The hypothesis
that this probability was identical was accepted at the 5 percent significant
level for these three domestic manufacturers. (See appendix D, section
D.l for details.)
-------
24
judgment should be expected to select type B vehicles whose three pollutant
levels were, on the average, greater than those for type A vehicles. The
second situation is one in which the emission levels of different pollutants
show a negative association (correlation) as vehicle designs vary. In this
case, vehicle designs which have a high level of emissions in one pollutant
would generally have a low amount in another. Engineering judgment in this
situation might select some type B vehicles based on one pollutant and some
vehicles based on another pollutant (which is negatively associated with the
first). Under these circumstances a comparison of the average emission levels
for type B to those of type A could show no difference, even though there is
in fact a difference in emissions between type A and type B vehicles.
Therefore, it was necessary to examine the correlation between each
pair of pollutants to determine which situation holds. Based on the assumption
that the correlation among pollutants within engine families is a constant,1
tests show no significant negative correlation between HC and CO or between
HC and NO . A negative correlation between CO and NO was, however, found to
X A
exist2 for domestic vehicles in engine families containing catalyst equipped
vehicles. The range of values for the correlation coefficient between CO and
NO for vehicle designs in the domestic engine families was found to be -.03
A
to -.35, with a median value of -.19,3 a relatively small correlation.
2The mean and standard deviation of emission levels are not assumed to be
constant within engine families (see appendix D, section D.2 for details).
2At the .05 level of significance,
3No other negative correlation between pollutants was found when vehicles
were stratified by a number of vehicle parameters (see appendix D, section
D.2).
-------
25
Since a negative correlation appears to exist between NO and CO and
since it appears to be only slightly negative in magnitude, two types of
analyses were used to investigate the degree to which type B vehicles
have higher emission levels than type A vehicles. The first, which examined
type A versus type B average emission levels, assumed that the correlation
between all pollutant pairs is non-negative. The second, which examined
the emission level variability of type A versus type B vehicles, assumed a
negative correlation between CO and NO . The results of these two analyses
A
are presented in the following two subsections.
3.2.3 Average Emission Levels
The relative magnitude of emissions from type B vehicles and those from
type A vehicles1 were compared using test results from 335 emission data
vehicles from the 1977 model year. The vehicles were stratified by domestic
versus foreign manufacturer and by catalyst versus non-catalyst equipped
vehicles.2 Three hypotheses were examined:3
(1) for vehicles of the same type (A or B), belonging to the
same engine family, the average emission level of one vehicle
is not different from the average emission level of another
vehicle;
^he details of these three analyses are presented in appendix D, section D.3.
^hese strata were selected since it was felt that the complexity of identi-
fying high emitters may differ for vehicles with and without catalysts and
for domestic versus foreign vehicles.
3A nested analysis of variance model was used to explain the effects of engine
family, type of selection, vehicle configuration and test-to-test differences
in emission levels (see section 0.3.1 of appendix D).
-------
26
(2) for vehicles within the same engine family, the average emission
levels of type B vehicles are not different from the average
emission levels of type A vehicles; and
(3) the average emission levels of vehicles in a family do not differ
from those in another family.
Each hypothesis above was tested on a pollutant-by-pollutant basis
under a number of assumptions concerning the magnitude of the test-to-test
variability1 and its relationship to average vehicle emissions. In all,
the above hypotheses were tested a total of 45 times (three pollutants, five
Vehicle categories and three test-to-test variability assumptions) without
once rejecting the second hypothesis (i.e., emission levels of type B and
type A vehicles are equal). Conversely the first and third hypotheses were
accepted a total of 39 and 42 times, respectively.
These results indicate that:
(1) the difference between the emission levels of vehicles within a
family is greater than can be attributed to test-to-test varia-
bility;
(2) the difference between the average emission levels of the type B
vehicles and those of the type A vehicles in the same family is
not significantly greater than the difference between the emission
levels of any two vehicles selected at random in a family; and
xIf the estimated value of test-to-test variability used in this analysis
were larger than in actuality then the test might erroneously accept the
hypothesis of no difference when in fact a difference does exist. There-
fore, to reduce the chance of adopting such a conclusion, three estimates
of test-to-test variability were used (two from data collected in this
study and a recent estimate made by General Motors investigators). All
three estimates were in approximately the same range (e.g., when expressed
as a percent of average emissions; 10 percent-13 percent for HC, 13 percervt-
16 percent for CO, and 4 percent-9 percent for N0X) , and the results of the
analyses using any of these estimates were essentially identical.
-------
27
(3) the difference between the average emission levels of vehicles
from different families is greater than the difference between
the emission levels of vehicles within the same family.
The above conclusions require assumptions concerning the distribution
of the emission levels of vehicles in a particular family or class about
the average of that class1 and the degree to which variability of emissions
in one family equals that of all other families.1 Since these assumptions
may not be precisely correct, two additional analyses were performed, which
do not necessarily rely on these assumptions.
The first compared the number of times that the emission levels of a
type B vehicle exceed those for a type A vehicle (within an engine family)
with the number of times this condition would be expected if the emission
levels were equal.2 The second examined the degree to which the difference
between type B vehicle and type A vehicle emission levels appeared to vary
evenly about zero.3 The results of these analyses supported the above
finding, i.e., type B emission levels are not distinguishable from type A
emission levels.
Specifically, the analyses of variance model assumes: (1) that each of the
effects (i.e., engine family, selection criteria, and configuration) is a
independently-normal random variable with zero mean and known variance,
and (2) that the variability of emission levels within an engine family is
a constant from one family to the next.
2A Mann-Wh1tney test was used to provide a non-parametric method of comparing
the relative rank of type B vehicles to type A vehicles (see section D.3.2
of appendix D).
3The special purpose test examined the distribution of t-stat1stics as discussed
in section D.3.3 of appendix D.
-------
28
3.2.4 Emission Level Variability
Under the assumption that the correlation between pollutants is negative,
the analysis of whether type B vehicles could have been identified appropri-
ately (i.e.» as high emitters in at least one pollutant) becomes a comparisono
type A and type B emission level variability. Such an analysis presumes that
the type B vehicles within a particular family were alternatively selected
on the basis of anticipated high emission levels of CO and NO . Under this
X
presumption, the above finding that the average emission of type A and type
B vehicles were approximately equal would be expected even if engineering
judgment had indeed selected those vehicles with high CO and high N0x in a
particular family.1
The variability of type A vehicle emission was found to be indistinguish-
able from the variability of the type B vehicle emissions. That is, ratios
of type B vehicle variance to type A vehicle variance could not be shown to
differ significantly from 1.0. However, the general range of the values for
these ratios demonstrated a general bias, albeit insignificant, favoring the
hypothesis that type A variability is greater than that for type B.2 These
results appear to refute the situation presumed above, i.e., that type B
vehicles were appropriately selected on the basis of expected high emission
levels for either N0x or CO (but not both).
Thus, the comparison of type B vehicle to type A vehicle emission levels
reaches the same conclusion irrespective of the correlation among pollutants.
A difference between the emission levels of vehicles in these two categories
does not exist.
iSee appendix D, section D.2.
2Details of this analysis are provided in section D.4 of appendix D.
-------
29
3. 3 Interpretation of Results
The results of the previous analyses demonstrate the similarity
between the emission levels of two types of prototype vehicles selected
on rather different criteria—high sales and high emissions. Consequently,
the results imply either that, in general, the collection of high sales
vehicles are also high emitters or that the vehicles selected on the basis
of emissions do not necessarily have higher emission levels. It is diffi-
cult to determine which case holds without test results on vehicles
representing all configurations within a family. However, an analysis of
the relationship between emission levels and engine family size1 (a proxy)
for sales level) found no significant correlation, indicating that high
sales vehicles selected for testing should not be expected also to have
high emission levels. Therefore, the vehicles selected as high emitters
do not appear to represent the configurations with the highest emissions
within a family.
The interpretation of the above result is dependent on the degree to
which EPA engineers are making the tradeoff between selecting vehicles with
the anticipated highest emission levels and selecting these vehicles on
other criteria (e.g., a fuel economy criterion). The extent that the type
B selection is based on a "representativeness" criterion rather than simply
a "high emission" criterion, will affect the degree to which type B vehicle
emission levels are the same as those for type A vehicles. However, if
the higher emitters from a small set of representative sub-family groupings2
Engine family size was measured by the number of EDVs used to certify the family.
2A review of the vehicle parameter values for the 1977 emission data fleet re-
vealed a relatively wide array of values, Indicating that individual engineers
may be attempting to select vehicles which represent the various subclasses
within a family.
-------
30
were selected as type B vehicles, the type A vehicles would have lower
expected emission levels, in general. Since this was not reflected by
the results in the above analysis, one must conclude that judgmental
selection tends to favor representativeness over high emissions or that
EPA1s ability to identify high emitters is not as effective as it could
be (under the assumption and evidence that high sales vehicles are not
abnormally high emitters).
If emission data vehicles are selected to represent a segment of an
engine family, then failure of an EDV increases the likelihood that other
vehicles in that segment fail to comply with the standards. A modifica-
tion of the EDV by the manufacturer in a subsequent submission of
that family then requires a second and different kind of engineering
judgment to determine which of the configurations in the family must be
likewise modified. EPA's current practice is to examine this resubmission
as though it were an original application and choose the highest emitter(s)
in the family to replace the failed vehicle(s). The knowledge about
characteristics of the failed configurations should improve the engineers'
ability to identify potential high emitters and, hence, relate configura-
tions that are similar within families.
However, without a detailed examination of the history of a number of
families which failed, were modified, and then passed, it is difficult to
determine the ability of EPA's engineers to relate configurations in this
manner.
J. 4 Alternative Methods of Vehicle Identification
The conclusion drawn from the preceding analysis is that engineering
judgment has not been effective in selecting the high-emission vehicles
-------
31
within engine families for the 1977 model year. There are at least five
possible explanations for this conclusion.
(1) A single engine family is defined to consist of vehicles which
have similar emissions. The degree to which this similarity
is achieved will determine the degree to which certain vehicles
within a family will have higher emission levels than the rest.
(2) If the manufacturer attempts to design all vehicles in a family
to some "target" emission levels, he does so by making complex
tradeoffs among the values of many parameters of the vehicle
designs. It is then very difficult even for the manufacturers,1
let alone EPA's engineers, to identify, In advance of testing,
which vehicles will tend to deviate from these target levels.
(3) EPA's engineers have insufficient time, experience, or data to
perform the task of identifying high emitters.
(4) Current procedures for selecting type B vehicles place too
great an emphasis on identifying a representative set of
vehicles from each engine family to allow the flexibility
necessary to identify high emitters.
(.5) EPA is currently selecting too many type B vehicles. In attempting
to fill its quota of prototype vehicles, EPA has included some
vehicles with low emissions along with one or two high emitters.
Conversations with Ford and GM personnel concerned with certification
revealed that they would have difficulty Identifying in advance which
vehicles in a family would have the highest emission levels.
-------
32
Although any of the above reasons could explain why the vehicles
selected as high emitters do not have, in general, the higher emission
levels within a family, a combination of three of these explanations
appears to be most likely. First, the differences in the emission
levels of tested vehicles are relatively small (often being statisti-
cally insignificant for NO in many families1). Second, given this
A
small difference, the complexity of selecting the high emission designs
within each family is increased by the numerous adjustments which the
manufacturers may use to vary the emission levels from one design
to the next. Finally, any incentive to select a representative sample
of test vehicles (such as ones designed to represent subclasses of
engine families or cover fuel economy testing requirements) dilutes
EPA's ability to focus solely on a high emitter criterion.
The relative contribution of each of these three explanations
to the resultant selection cannot be ascertained without an examina-
tion of the vehicle selection process on a case-by-case basis. If the
primary cause is the fact that engine families are so small that vehicles
within them have very similar emission levels, then the solution is to
restructure the family classification or simply select vehicles within
each family at random. If, however, the problem is created by the com-
plexity of the relationship between emissions and combinations of
vehicle parameters or by an interest in providing more representative
vehicles, then a restructuring of the selection process (possibly in-
cluding some random selection) seems appropriate. Such a restructuring
!See appendix C, exhibits C-2 and C-3.
-------
33
could alter the amount of time spent selecting the high emitter vehicles or
improve EPA's ability to identify these vehicles, or both. Three possible
alternatives are:
(1) to rely totally on mechanized selection methods;
(2) to enhance judgment criteria by increasing the use of data;
feedback from other engineers, etc.; and/or
(3) to combine new judgment procedures with mechanized selection
procedures.
The first of the above alternatives would essentially replace the time
currently spent by the engineering teams selecting type B vehicles with an
automatic selection methodology. Such a methodology should have a random
component to avoid situations in which the knowledge about how the vehicles
were selected would promote potentially deceptive practices by the manu-
facturers. The scheme could be totally random or contain some form of bias
(e.g.. selecting vehicles with certain parameter values with a greater
probability).
The potential annual savings in engineering analysis time used to select
these vehicles would be approximately a man year or less.1 The gain would
free some of EPA's engineering talent to perform other tasks; however, it
would require other resources, primarily in the data processing area. Thus,
^PA's annual estimates of the number of families submitted for review vary
between 300 and 400. EPA further estimates that the selection of EDVs from
a typical family takes between three and five hours. Thus, a rough average
of EPA workload to select emission data vehicles is about 1400 man hours per
year, or about three-fourths of a man year. (This estimate probably does
not include the time necessary for supervising personnel to approve the
selection nor the time necessary for EPA to deal with manufacturer problems
concerning this selection or with approval of carry-over or carry-across
requests [EPA, undated].)
-------
34
EPA would realize little net savings in total resources by adopting a
totally mechanized selection process.1
The second of the above alternatives (alter current judgment practices)
appears to offer limited improvement without a more detailed examination of
the performance of individual engineers. That is, one must investigate the
extent to which the application of engineering judgment could be improved
by increasing the time to make the decision, improving the understanding of
the emission production process, greater reliance on routinized procedures,
etc.
Finally, a combination of mechanized solutions and improved judgmental
procedures appears to offer the greatest promise. Under this alternative,
EPA could vary the number of selections made mechanically and judgmentally
as circumstances dictate. For example, current procedures could be used to
select the type A vehicles and one type B vehicle while the remaining
vehicles would be selected at random.2 This would provide a mechanism for
the engineers to monitor their performance against a random sample and
possibly stimulate an improvement in the ability to identify vehicle char-
acteristics which indicate potential high emissions. Further, a combined
procedure would provide a method to compare the performance of a variety
of mechanical schemes to engineering judgment and vice versa. This approach
LThis conclusion is clearly erroneous if mechanized selection would permit
EPA to eliminate much of the time consumed reviewing the manufacturers'
applications. However, since this review is necessary to perform other EPA
tasks (e.g., designation of engine families, selection of durability vehicles#
review of required disclosures, identification of defeat devices, etc.), eli-
mination of the EDV selections would not alone preclude this activity.
2This random selection could be designed to account for current selection cri-
teria which stipulate coverage of engine-system combinations, high altitude,
and fuel economy base-level configurations.
-------
35
could be implemented almost immediately without incurring additional workload
and without degrading certification program performance.1
A more extensive change in current procedures would adopt some new
judgment methods and combine them with mechanized sampling. Such a procedure
could use data from previous years, low mileage durability vehicle tests and/or
manufacturer development data to determine where judgment versus random
sampling techniques would be used to select vehicles. For example, judgment
might be used to select vehicles from families suspected of having a greater
likelihood of having vehicle configurations whose emissions exceed standards,
and random sampling used on the remainder. Similarly, characteristics of
vehicles which failed one or more emission tests2 could be compiled and
analyzed continuously to provide a mechanism to update judgment procedures
or modify mechanized sampling routines which reflect the knowledge gained
in this monitoring process.
It should be noted that none of the actions suggested above would
significantly reduce EPA's engineering or testing workload.3 Rather, the
primary gain from any of the above actions other than completely random
selection would be to improve the capability of the certification process
to identify engine families (and vehicles within these families) which do
not comply with the Clean Air Act.
1The results of the analyses in the first part of this chapter show current
judgment is very similar to a random selection of vehicles.
2These tests could include those from the fuel economy, durability running
change, and emission data testing programs.
3As noted previously, EPA's estimate of the workload to select emission data
vehicles is less than a man-year [EPA, undated].
-------
36
-------
37
4.0 VEHICLE TESTING STRATEGIES
AND COMPLIANCE CRITERIA
After a set of emission data vehicles has been selected and built
for a given engine family, EPA's current procedures require that EPA
test all of these vehicles and certify the engine family only if, for
each of the vehicles, the measured emission levels (after application of a
deterioration factor) are within the standards on at least one of two tests.1
Section 4.1 of this chapter compares this testing strategy (i.e., the
requirement that all EDVs be tested by EPA) and this criterion for vehicle
compliance (i.e., the requirement that a vehicle pass one of two possible
tests) with certain alternative strategies and criteria. These alternatives
may allow EPA to save resources without substantially increasing either the
risk of erroneously passing vehicles as complying with the standards or the
risk of erroneously falling them.
Alternative strategies for testing "running changes" are the subject of
section 4.2 of this chapter. Once an engine family has been certified, the
manufacturer is likely to want to make modifications to designs within
the family, for various reasons relating to the manufacturing process or the
performance of the vehicles. Although these changes may not be Intended to
improve (or degrade) emission levels, the complexity and interrelatedness of
the total automobile power train system will cause the changes to have potential
effects on these levels. For this reason, EPA must approve all proposed running
changes affecting emission-related parameters before they may be implemented.
xThe second test is conducted only in the case of a failure on the first test
and a subsequent request for a retest by the manufacturer.
-------
38
Each is examined by EPA engineers, who may authorize the change directly,
may require manufacturer tests of the change, or may require EPA tests of
the change. Testing may involve tests of the changed version of the vehicle
only, or may consist of "back-to-back" testing in which the original test
vehicle is tested first in the initial configuration and then as modified.
4.1 Analysis of Emission Data Vehicle Testing Strategies and Compliance
Criteria
An analysis of alternative testing strategies and of alternative criteria
for determining compliance of a single EDV was conducted on a representative
fleet of 1977 model year vehicles.1 Emission data for this fleet were used
to develop probability distributions of true emission levels for prototype
vehicles. These distributions were used in conjunction with statistical
estimates of test variability to predict the effectiveness of each of several
sets of procedures for conducting the certification process. For each
set of procedures evaluated, several effectiveness measures were computed,
including
(1) the fraction of EDVs failed by the certification process;
(2) the fraction of EDVs failed by the process when at least one
2
of their true emission rates exceeded the standards; and
(3) the fraction of EDVs failed by the process when all their true
3
emission rates were within the standards.
Appendix E includes a detailed description of this data base, and of the results
of the analysis. The analytical procedures used are described in appendix F.
20ne minus this fraction is referred to as the probability of erroneously
passing a vehicle.
3This is referred to as the probability of erroneously failing a vehicle.
-------
39
All these measures are expressed as failure rates of EDVs rather than as
rates of failing to certify engine families. It is VRI's understanding
that the usual effect of failure of an EDV is that the manufacturer modifies
the specifications of the vehicles in the portion of the engine family
associated with the failure so that the family eventually is able to be
certified. These EDV failure rates reflect the frequency at which this
activity occurs and whether or not the activity should have been required
by EPA. A change in the first of these effectiveness measures would repre-
sent a change in the cost of the certification process to the manufacturer;
a substantial change in this value might therefore be expected to produce a
change in manufacturer behavior.1 The second of these measures represents
the effectiveness of the certification process in terms of its ability to
detect vehicles which exceed the standards. The third measure reflects the
frequency with which EPA makes the error of failing vehicles which meet the
standards. Failure to keep this frequency at a reasonably low level would
impose an unfair burden on the manufacturer which he (and society) might be
unwilling to bear.2 In addition to these three measures of effectiveness,
differences in pollution emitted by vehicles certified under the alternative
strategies were predicted, and differences in the resources required to im-
plement the various strategies were estimated. The following three sections
discuss the results of this analysis.
xFor most strategies analyzed in this study, the effects of a change in
manufacturer behavior were not explicitly included in the analysis. However,
a discussion of the impact of such a change is provided later in this chapter.
2Note that there is an inherent tradeoff between the second and third of these
measures—an increase in one tends to lead to an increase in the other.
-------
40
4.1.1 Current Procedure and Manufacturer Behavior
In order to develop estimates of changes in effectiveness resulting from
alternative strategies, the effectiveness of EPA1s current process was first
estimated by analyzing EDV test results. The result1 was that approximately
ten percent of EDVs were predicted to fail under current procedures. Of
all vehicles whose true emissions are within all three exhaust emission
standards, approximately two percent are erroneously failed by the process,
while 62 percent of all vehicles which actually exceed the standards are
failed by the process.
The relatively low failure rate predicted for the current process (which
is consistent with EPA experience) is the result of the fact that most vehicles
(approximately 87 percent in the domestic portion of the fleet) which are
submitted for certification meet all exhaust emission standards, presumably
to avoid the cost, of failures by the certification process. As a result,
the direct effect of EPA's failure of vehicles on the total emissions of
vehicles in certified families is relatively small, and thus none of the
strategies analyzed in this study produces a large change in pollution emitted
by vehicles in certified families.2 The primary benefit of the current certi-
fication program is that it has helped to compel manufacturers to design
vehicles which comply with the Clean Air Act.3 Any recommended change
See appendix E for a more detailed description of these results, and appendix
F for a description of the methodology used to generate all results presented
in this chapter.
2
The largest percentage saving realized by any strategy considered is only 1.5
percent of the corresponding emissions under EPA's current strategy. (This
savings occurs for CO in the case in which only one test is allowed for every
EDV, so that retests are not allowed. See section 4.1.2.)
3
Even failed vehicles generally do not exceed the standards by much. The
typical failure can be expected to be about eight percent above the standard
on either CO or N0X-
-------
41
to EPA's current procedures must therefore be evaluated for its likely
effect on this indirect benefit, and should be implemented in such a way
that it can be monitored for any resulting undesirable changes in manufacturer
behavior.
The impact of the certification process on manufacturers is felt primarily
via the rate at which vehicles are failed by the process. It is reasonable
to assume the manufacturers respond to changes in the certification process
by adjusting the overall emissions of their fleet so as to maintain an accep-
table fraction of EDVs failed by the process.1 Analysis of this type of
behavior2 reveals that an overall increase (or decrease) in total emissions
of one percent will cause an increase (or decrease) 1n the failure rate of
approximately one percentage point. Thus, assuming the current failure rate
of ten percent is acceptable to the manufacturers, a change in EPA's procedures
causing this rate to decrease to eight percent might be expected eventually
to cause the manufacturers to allow the overall emissions of their fleets to
increase by two percent. This hypothesized effect of EPA's procedures on
manufacturer behavior is not explicitly included 1n the predicted effectiveness
of alternative EPA policies considered below, but the possibility of some such
effect should be borne in mind in evaluating the overall impact of proposed
changes.
4.1.2 Alternative Compliance Criteria
Two alternative criteria for determining whether a single EDV is
passed by the certification process emphasize two differences from EPA's
lThis assumes that the technology to perform this adjustment is available to
the manufacturers, and that the cost or savings of doing so 1s justified by
the corresponding savings (or cost) associated with a decrease (or increase)
in the rejection rate.
2See section E.2 of appendix E.
-------
42
current procedures: averaging the results of multiple tests on a single EDV,
and changing the total number of tests conducted on a single EDV.
In VRI's preliminary analysis of EPA's certification procedures [VRI, 1976]
it was suggested the EPA consider a policy of averaging the results of
the two tests conducted whenever a manufacturer requests a retest of a failed
EDV. The suggestion was based on the belief that an averaging policy would
be more consistent with other EPA policy and with the intent of the Clean
Air Act than is the current policy of certifying the EDV if the second test
is passed.1 An averaging policy would impact on the effectiveness of the
certification process as follows:2 adoption of such a policy would cause
an increase in the overall failure rate from the current ten percent to
approximately 14 percent. A major portion of this increase would be associated
with the failure of additional vehicles whose true emissions exceed the
standards—approximately 85 percent of such vehicles would fail, compared
to the current 62 percent. Overall emissions of certified vehicles would be
reduced, but the reduction would be small, for reasons given in the previous
section. A few additional test procedures (fewer than 100 per year3) would
be required to allow for the testing of the increased number of resubmitted
vehicles as a result of the higher failure rate. However, the greatest
potential disadvantage to this policy is that it would more than double
the rate of erroneously failing vehicles, increasing it from two percent to
about five percent. In summary, EPA may wish to consider a test averaging poli^
since it allows detection of more vehicles which exceed the standards than does
Statistically, the averaging policy would produce an unbiased estimate of the
true emissions of the vehicle, while the current policy does not.
2See section E.3 of appendix E.
3A11 estimates of changes in testing requirements are based on EPA's projection
that 938 EDVs will be submitted for model year 1978 [EPA, undated].
-------
43
EPA1s current policy. However, such a policy also causes an increase in the
risk of erroneously failing vehicles which EPA might consider excessive.
As the number of tests on a single EDV increases, the variability of the
estimate of the true emissions of the vehicle (taken as an average of the test
results) will decrease. As a result, the rates of both erroneously passing
and erroneously failing vehicles should decrease with an increase in the
number of tests conducted on each vehicle. To investigate this effect, an
analysis was conducted of the case in which only one test is conducted on every
EDV (so that retests are not allowed), and of the case in which EPA tests
every EDV twice and averages the results. As expected, both types of errors
(erroneous passing and erroneous failure) were reduced somewhat in switching
from a one-test policy to a two-test policy. The one-test case produces
a rate of erroneously failing vehicles (seven percent) which Is almost certainly
unacceptably high, while saving EPA relatively little in resources (fewer than
100 tests per year). The two-test case produces results (in terms of the
three types of failure rates) which are essentially identical to those of the
previously-analyzed policy of retesting only failed vehicles and averaging
the outcomes. This results from the fact that the emissions of most passing
vehicles tend to be well below the standards, so that a vehicle which passes
one test is highly unlikely to fall a retest. However, the two-test policy
would be significantly more costly to EPA than a policy of retesting and
averaging only in the event of a failure. Approximately 1,300 more tests
per year than are currently conducted would be required to implement a two-
test policy. Thus, EPA's current policy of retesting only failed vehicles
appears to be an efficient one. A three-test policy would result in a further
decrease in the rates of both types of errors, but the increase in the number
of tests required to implement such a policy would be prohibitive.
-------
44
4.1>3 Alternative Testing Strategies
If the engine family classification scheme groups vehicles into fami-
lies whose emissions are similar, EPA should be able to use this grouping
to its advantage in deciding which EDVs to test. This section investigates
two general kinds of such testing strategies: sequential testing strategies
and judgmental selection of potentially high-emission families.
A sequential testing strategy would base a decision as to how many
EDVs to test in a family on the results of earlier tests on a small number
of vehicles in the same family.1 If these preliminary tests indicate that
the vehicles in the family are likely to produce emissions close to or in
excess of the standards, all EDVs in the family would be tested. Otherwise,
only a small number of EDVs would be tested at random.2 Such a sequential
testing strategy would still require the manufacturers to build the same
numbers of prototype emission data vehicles. The major difference would be
the number of these vehicles which EPA would test at its facility and the
number that manufacturers would test to determine certification outcome.
2It should be stressed that EPA must have control over which vehicles are
tested first within a given family, in order to avoid basing the testing
decision on EDVs effectively selected by the manufacturer through his con-
trol of the order in which EDVs are tested. This requirement may produce
test scheduling problems which would have to be overcome before a sequential
testing strategy could be adopted.
2This random testing would allow EPA to monitor manufacturer behavior and
to adjust the parameters of the sequential strategy if a change in behavior
resulted in a change in the characteristics of the fleet submitted to EPA.
-------
45
To assess the effectiveness of adoption of a sequential strategy,
several such strategies have been analyzed, varying the criteria for
deciding whether to test all the EDVs in a family.1 A broad set of
sequential testing strategies was identified which would save certifi-
cation and testing resources with no increase in the risk of erroneously
failing vehicles, and with no increase to a moderate increase in the
risk of erroneously passing vehicles, compared with the risks under EPA's
current testing procedures.2
As an example, one of the more conservative of these strategies
requires two vehicles to be tested in each engine family. If the meas-
ured emissions on both vehicles are less than 60 percent of the standard
for HC and CO, and less than 70 percent of the standard for NO , the
family is certified.3 Otherwise all EDVs in the family are tested before
a certification decision is made. This strategy produces results (in
terms of the three types of failure rates and total lifetime emissions)
which are essentially identical to the results under EPA's current pro-
cedures, while testing 125 fewer EDVs per year (for a saving of approxi-
mately 158 tests, counting voided tests, retests, and tests of resubmitted
Results for all sequential strategies evaluated can be found 1n section
E.4 of appendix E,
2A11 of the strategies considered are somewhat conservative in that they
require no use of manufacturer data 1n the process of deciding which
vehicles to test.
3These percentages were selected on the basis of the statistical charac-
teristics of the emissions levels involved. See section E.l.2 (and
especially exhibit E-3) of appendix E for a description of these char-
acteristics.
-------
46
vehicles after a failure).1 An associated saving of Certification Division
personnel time can also be expected, due to a reduction in observation of
tests and administrative support of the testing process.2 Further analysis
has indicated that these savings can be more than doubled through adoption
of a less conservative sequential strategy (one requiring only one vehicle
to be tested in each family) if EPA is willing to accept a moderate increase
in the risk of erroneously passing vehicles. Such a strategy would have a
negligible direct effect on emissions, but it would decrease the overall
failure rate to .08, so it might affect manufacturer behavior somewhat.
The above results are based on the assumption that EPA's current com-
pliance criteria would be employed on those vehicles which would be tested
under a sequential strategy. An analysis was also performed of a sequential
strategy coupled with a policy of averaging the results of the two tests
whenever a retest is conducted on a failed EDV. The particular testing
strategy involved would require the testing of only one vehicle in each
family unless one or more of the thresholds listed earlier was exceeded.
This strategy produces failure rates comparable to those of EPA's current
LThe removal of an EDV could "uncover" a fuel econoiny base level (see appendix
G for discussion of the fuel economy testing process). Consequently another
vehicle would have to be tested in its place to provide the necessary fuel
econoiny data. The likelihood of such an occurrence is dependent on the prob-
ability that a base level has more than one EDV. Data from the current pro-
gram indicate that about half of the base levels are covered by more than
one EDV. Therefore, the chance that the removal of an EDV would "uncover" a
base level is at most about 33 percent. As noted previously, EPA's current
policy of testing 25 percent of all fuel economy data vehicles (FEDVs) would
then require EPA to test one out of every four EDVs (or equivalent) whose
removal "uncovered" a base level. Therefore, the fuel econoiw program would
require EPA to test at most eight percent (or.one of every 12) EDVs removed.
2Exact quantification of this saving is difficult, but a rough approximation
is between two and three hours per EDV not tested.
-------
47
procedures (11 percent of all vehicles fail, 61 percent of vehicles with
true emissions above the standards fail, and three percent of vehicles with
true emissions below the standards are erroneously failed), while saving
EPA roughly 380 tests per year.
In summary, sequential testing strategies are available which would save
resources and result in acceptable risks. A comparison of the effectiveness
of the alternative sequential strategies described above is summarized in
exhibit 3.
A possible alternative to using a small number of tests to identify
those families in which all EDVs are to be tested would be to employ the
judgment of certification engineers to identify high-emission families.
While it is not known how effectively engineers could perform this function,
ft may be less difficult than the task of selecting high-emitting vehicles
within each engine family.1 Under EPA's current certification procedures,
more than 90 percent of the failed vehicles are contained in only 50 percent
of the engine families. If engineers were able to preselect these families
perfectly and test all EDVs in them (testing only a few randomly-selected
vehicles from other families), EPA could test slightly more than half the EDVs
currently tested and suffer only a small increase in the risk of erroneously
passing vehicles. It is unlikely that engineers have such a high capability
to Identify these families, however.
lLow mileage durability vehicle tests and/or manufacturer development data
might assist in this selection process.
-------
EXHIBIT 3: COMPARISON OF SOME ALTERNATIVE TESTING STRATEGIES
Strategy1
Fraction of EDVs Failed
Reduction
Overall
When True Emissions
Exceed Standards
When True Emissions
Are Within Standards
in Tests
Required
Current strategy
.10
.62
.02
0
Two-vehicle sequential
strategy
.09
.62
.02
158
One-vehicle sequential
strategy
.08
.46
.02
392
One-vehicle sequential
strategy with averag-
ing on failure
.11
.61
.03
380
Judgmental family
selection
.08
.50
.02
418
Judgmental family
selection and
sequential testing
.09
.59
.02
228
^hese strategies are defined precisely in appendix E.
-------
49
Using a more realistic (although still arbitrary) representation of the
engineers' ability to identify families with high emissions,1 analysis indi-
cates that a 30 percent reduction could be achieved in the number of EDVs tested
(a savings of approximately 418 tests on 281 vehicles) with a moderate increase
in the risk of miscertification. In this case, the overall failing rate
would be about eight percent, the failure rate of vehicles whose true emissions
exceed the standards would be 50 percent and the rate of erroneous failure
would remain at about two percent.
A more conservative use of this same assumed capability to identify
high-emission families would be to combine the capability with sequential
testing in the following way: if engineering judgment indicates a family is
likely to contain high-emitting vehicles, apply a conservative sequential
strategy to it (e.g., require that two vehicles have emissions below the thres-
holds in order to certify the family without further testing). Otherwise,
apply a less conservative strategy to the family (such as one based on the
test of a single vehicle in the family). Analysis of such a mixed strategy
reveals that the resulting risks are only slightly greater than under EPA's
current procedures (nine percent of vehicles are failed, 59 percent of vehicles
whose true emissions exceed the standards are failed, and two percent of vehicles
are erroneously failed), while saving EPA approximately 228 tests on 167 EDVs.
It thus appears to be worthwhile for EPA to Investigate engineers'
ability to select high-emitting families. If such a capability exists, it
alone could be used to reduce significantly EPA's testing workload with a
^he exact nature of this representation 1s described in section E.5 of
appendix E.
-------
50
moderate increase in the risk of erroneously passing vehicles; or it could
be Used in conjunction with sequential testing to increase the benefits and/or
reduce the risks resulting from implementation of a sequential strategy alone.
The potential effectiveness of each of these alternative uses of engineering
judgment is summarized in exhibit 4-1.
4.2 Analysis of Running Change Testing Strategies
Running change vehicles result from changes to certified vehicle designs
proposed by the manufacturer which are additions or modifications to the current
product line. EPA engineers determine whether these proposed designs belong
in a certified engine family and therefore represent "running changes" to
previously certified configurations.1 Proposed vehicles which are running
changes must then be evaluated to determine whether EPA and/or manufacturer
testing is required for approval. This testing decision is based on engineering
judgment that the effect on emissions of the proposed change is an increase
in pollutant levels above the standard, or is unknown. This section is divided
into two parts. The first provides an analysis of the emission levels of the
running change vehicles tested by EPA. Then, based on the results of this
analysis, the second suggests alternative ways of determining whether or not
EPA should test these vehicles.
Closely related to running changes are "additions of vehicles." These addition5
are changes to a certified vehicle proposed to permit a manufacturer to market
a new product. These changes are also reviewed by EPA to determine whether
testing is required, however, they are much less frequent and were therefore
not analyzed in this study.
-------
51
4.2.1 Effectiveness of Current Strategy
In order to examine the potentials for reduction in either the amount
of engineering analysis or the number of tests performed, it was first neces-
sary to analyze and understand the performance of the present process.1 This
investigation examined EPA test data on 1977 running changes (RCs) proposed by
Chrysler, Ford, and General Motors. The running change tests used in this
analysis were those which were conducted on modifications to original emission
data vehicle (EDV) configurations so that the emission levels before and after
the changes could be examined. A total of 66 such test data sets were identified.
For each of these sets, the results from the certified vehicle tests were
coupled with one or more running change tests.2
Because EPA tests were conducted on changes when engineering analysis
suggested that they were needed (presumably because of a potential for
increased emissions of at least one of the pollutants), it seemed reasonable
to expect that the EPA running changes in the 66 test data sets would, in
general, have higher emission levels than those
-------
52
running change tests could show an increase in at least one pollutant,
even though no single pollutant showed an average increase in emission
levels.
Statistical tests were performed for both these hypotheses. The first
test compared the average emission levels of running changes to those of
their previously certified vehicle. The results of this examination revealed
no significant difference at the five percent level of significance.1 The second
test examined the frequency with which one or more of the pollutant levels
from the running changes were above or below that for the certified EDV.2 A
total of 109 EDV-RC comparisons revealed that the RC emission levels were
symmetrically distributed above and below the EDV emission levels. Since it was
possible that some of these running change test results represented the unchang^
portion of a back-to-back test, a check was made to see if this possible
ambiguity in the test results would influence the conclusions of this analysis.
The examination revealed that the results of the analysis were indifferent to
the interpretation of these test data and that running changes which showed
an increase on at least one pollutant were not more frequent than would be
expected from random effects.
JThe statistical test used in this investigation was a one-way analysis of
variance to determine if the mean of the EDVs was significantly different
from the mean of the running changes (see section D.5 of appendix D).
2A Chi-square test (like that used in comparing the failure rates of type B
vehicles to type A vehicles) was used to compare the number of times that one
or more RC pollutant levels exceeded those of the EDV with the number of times
expected from random effects only (see section D.5 of appendix D).
-------
53
The above results suggest that, given the present manufacturer behavior
in the design and modification process, the running changes selected for
testing show no bias toward higher emissions than the originally certified
configurations. These analyses only indicate, however, that the running change
emissions vary randomly above and below the original certification levels and
do not indicate the magnitude of this variation. Since the decision to test
a running change may be based on knowledge that the proposed change can signi-
ficantly affect emissions but the direction of this impact is uncertain, it
was necessary to examine the relative variability of running change emissions.
An examination was therefore conducted to measure the variability in
the emissions of the original vehicles, as compared to that of their running
change counterparts. Since test-to-test errors would Introduce some variation
even if the running change designs were Identical to their original counter-
parts, the estimate of variability was computed 1n two steps. First, an esti-
mate of the total variability for each pollutant within each EDV-RC group
was computed to get an estimate of the percent of variation around the group
averages. This estimate contained both a test-to-test and running change
variability components. In the second step, an estimate of test-to-test var-
iability (see appendix B) was subtracted from the total estimate for each pol-
lutant to approximate the amount of variability which could be associated with
running changes. The resultant values range from eight percent to 12 percent
for HC, 22 percent to 32 percent for CO, and five percent to seven percent for
»)«¦
The above RC variability ranges are less than the configuration-to-
configuration variabilities found within an engine family.1 Thus, on
xThe precise estimate of the configuration-to-configuration coefficient of
variation within a family varies depending on the statistical model used,
but all estinates run about 20 percent for HC, 30 percent for CO, and 15
percent for N0X.
-------
54
the average, the probability that an EPA test will detect an RC whose true
emission levels are above the standards (and therefore reject the proposed
change) is less than the probability of rejecting a configuration selected at
random within the family. In fact, using both the data from the analysis of.
the EDV emissions in chapter 3.0, and that from this analysis of running
changes, one can show that the probability of failing a design on an RC test
is between 4 and 40 times smaller than that of failing a randomly selected
confi guration.1
The above findings were further supported by a review of a small sample
of proposed changes (five from Ford and five from GM) for which EPA decided
to rely solely on the manufacturer's test results rather than to require
additional testing by EPA. An analysis of these vehicles revealed that they
were as likely to fail certification as those selected for testing by EPA.
Thus, EPA's decision to test or not to test a proposed running change is
statistically independent of the estimated probability that the change would
cause the vehicle to fail the test.
4.2.2 Alternative Running Change Testing Strategies
The primary conclusion of the previous analysis is that EPA reviews
and tests a large number of running change vehicles whose emission levels
cannot be differentiated from those of the original configurations which are
to be modified. The cost to EPA to perform this task is significant. EPA
estimates it will take between one and four person years of effort to review
xThe relatively high variability in the estimated ratio of failure proba-
bilities is due to the data-oriented and statistical uncertainties in all
the estimates.
-------
55
the 1800 running change applications anticipated for the 1978 model year.1
Further, if the historical RC test rate continues, about half of all RC
applications will be tested by EPA, and this testing will consume approxi-
mately 50 percent of the exhaust emission testing workload. This suggests
that EPA will concentrate a significant amount of time and effort approving
many running changes which the above analysis shows are more likely to com-
ply with standards than a randomly selected configuration within the same
family.
The small difference in the emissions between the original EDV and
the RCs to this configuration (and the generally low probabilities of
rejecting a running change2) suggests strongly that a great deal of
selectivity should be exercised to control the amount of EPA attention
and resources which are given to the running changes. While some
randomly-selected changes should be reviewed to determine the need for
testing, it would appear that most of the selection of changes for
testing could be made on a formal basis similar to that suggested 1n
the analysis of sequential testing strategies for emission data vehicles
(see section 4.1). In fact, the same policies suggested 1n section 4.1,
in which the choice of whether to test or not is based on the past tests
of vehicles in the same family, could be applied directly. That 1s, If
the sequential strategy used 1n the testing of EDVs suggested that EDVs
l
EPA's FY77 Program Plan states that the review of a running change can
take from one hour to several days with most changes requiring about one
to four hours for review [EPA, undated].
Conversations with EPA engineers indicate that failure of a running change
is an extremely rare event. Manufacturers do, however, withdraw some
applications for running changes which may indicate their avoidance of an
anticipated failure. The data supplied to VRI on running changes Indicate
all of the running changes examined 1n this study were accepted by EPA.
-------
56
in a family should not be tested further, running changes within that
family would also not be tested (except on a limited random basis). In
addition, a choice based on the certification results of the original
EDV, if it was tested, would provide a slight improvement over the use
of only family data.
-------
57
APPENDICES
-------
58
-------
59
The purpose of the appendices which follow 1s to provide the Interested
reader with further details about the data, analytic methods and results
used to develop the conclusions and recommendations presented 1n the body of
the report. For ease of reference, the methods described in the appendices
are presented 1n approximately the same order as they appear 1n the main
text. All figures and tables 1n the appendices are referred to as exhibits
and appear at the end of each appendix.
There are seven technical appendices and one appendix which provides
the bibliographic references used 1n the study. The title of each appendix
is given below.
Appendix A: Description of VRI Data Base
Appendix B: Analysis of Emission Test Variability
Appendix C: Analysis of Engine Family Classification
Appendix D: Analysis of Engineering Judgment
Appendix E: Analysis of Alternative Testing Strategies
Appendix F: Methodology for Effectiveness Assessment
Appendix G: Description of EPA Fuel Economy Testing Program
Appendix H: References
-------
60
-------
61
APPENDIX A
DESCRIPTION OF VRI DATA BASE
One of the major tasks of the study was the identification, collection
and review of emission data vehicle (EDV) test results and associated
information. The majority of information contained 1n VRI's data base
was that provided by the Data Branch, Program Management Division, in
machine readable form. This data was augmented with emissions data vehicle
information maintained by three certification teams within the Light Duty
Certification Branch, Certification Division. The following subsections
provide a description of the development of this data base and a summary
of Its content.
A. 1 Data Base Development
At project outset, the Intent was to analyze EDV test data from model
years 1975-1977 for all manufacturers to study the effectiveness of engi-
neering judgment, the engine family classification scheme, and the cost
and risk implications of alternative testing strategies. In order to follow
such a plan within the project resources, 1t was necessary to acquire a
complete and comprehensive set of machine readable EDV test data. Because
portions of the necessary data were not 1n that form, two operational deci-
sions were made to ensure that the data collection activities of the pro-
ject would not consume a disproportionate amount of available resources.
These decisions were:
(1) to use EDV data from three domestic (Chrysler, Ford, and General
Motors) and six foreign (Honda, BMW, Fiat, Datsun, Mazda, and
Volvo) manufacturers and
-------
62
(2) to concentrate on 1977 model year data, using 1976 model year
data to provide a larger vehicle sample for certain analyses
and to support conclusions derived from the more recent data.
The emissions data vehicles included in the study data base were iden-
tified by reviewing manufacturer Part I and Part II applications and comparing
this information with the data found in the light duty vehicle summary sheets.
This comparison permitted the identification of those EDVs which were tested
by EPA to certify the original vehicle fleet of each manufacturer (i.e.,
vehicles which were withdrawn prior to EPA testing, vehicles tested for pur-
poses other than certification, and vehicles which were "carryovers" from
previous years were excluded from the data base). In addition, the review
of material maintained by the EPA certification teams provided information
necessary to determine whether the high sales or high emissions criterion
was used to select each EDV.1 The description of the engineering specifica-
tions and test results2 for each of these vehicles was then extracted from
data contained in the light Duty Vehicle Data System, Phase 2, maintained
by the Data Branch.
Information was extracted from four data files in this system. These
files contain information detailing vehicle specifications (EPA's 1000D-
CURMAS file), the tests performed by EPA (EPA's 1200D file) and manufac-
turers (EPA's 1202D file) on each EDV, and a locator file to connect Informal
*The criterion for selection is an essential element for the analysis of
engineering judgment and is not currently available on the EPA automated
data base.
20nly those test results which were collected using the 1975 federal test
procedure and considered valid by EPA (i.e., not voids) were included in
the data base.
-------
63
in these files (EPA's 1200D-MASL0C file). Since these files are constantly
being updated, VRI's information was that found in EPA's files on the date
of transfer, November 18, 1976, for domestic manufacturers and February 15,
1977, for foreign manufacturers.
In order to analyze test information for model year 1976,1 the above
files were augmented using information maintained in the previous data system
(EPA's file 10300). Data in this file is much less comprehensive and not
compatible with that 1n the Phase 2 system; thus vehicle Information for 1976
EDVs tested prior to January 1977 was less complete than that found for
vehicles tested after that date.
A. 2 Data Base Content
The study data base consisted of emission data and vehicle specifica-
tions collected on 583 emission data vehicles. These vehicles were distri-
buted by year and by manufacturer as shown 1n exhibit A-l. As noted pre-
viously and as can be seen In the exhibit, only 1977 model year vehicles
are Included for foreign manufacturers.
A review of test data for the 1977 model year vehicles 1n this data
base revealed a number of situations which appeared to contradict VRI's
understanding of certification procedures, required further Interpretation,
or contained potentially erroneous Information. The majority of these
situations were encountered 1n an attempt to determine whether vehicles
with identical data base specifications were Indeed Identical and whether
multiple tests on such vehicles were conducted under the same procedures.
^ince the recoding of pre-Phase 2 information was not completed at the
time of transfer, the files do not contain Information on EDVs tested
prior to the beginning of the Phase 2 system (approximately January 1976).
-------
64
Examples of the types of problems encountered in this examination included:
(1) the identification of a failed vehicle versus its "fixed"
version (i.e., the modification of a failed vehicle to permit
retesting of it under the configuration specified for the pro-
posed replacement),
(2) the determination that manufacturers had not requested retests
for vehicles with one failed test (this was necessary to ensure
that the single failed test was not a coding error or that there
did not exist an additional failed test which was missed),
(3) the identification of low altitude tests performed on high
altitude vehicles (the latter set of vehicles being excluded
from our analysis),
(4) the identification of detectable coding errors (the most problem-
atic of these being errors in coding test procedure and certifi-
cation disposition), and
(5) the identification of whether a series of tests on a running
change to a certified vehicle were multiple tests on the same
running change, back-to-back tests on a proposed change before
and after its implementation, or sequential changes with each
tested once.
Resolution of the above and similar problems required a reasonable amount
of effort by the VRI study team and the participation of the certification
team responsible for encoding much of this information since much of the
necessary information was not readily available without searching the manuals
and personal memories of these groups. The amount of work required by this
-------
65
activity, plus the knowledge that ferreting out the similar information for
1976 vehicles would be a much more demanding task, led the study team to
decide to begin by performing most analyses on 1977 vehicle information
and test results.1 The two analyses which used both the 1976 and 1977
vehicle data were the investigations of the correlation between pollutants
(described in section 0.2.2 of appendix 0) and the running change emissions
(discussed in section D.5 of appendix D).
All analyses using only 1977 data were either made on the entire
collection of 1977 emission data vehicles or on the subset of these vehicles
which had more than one test conducted under the same test procedure (i.e.,
the 1975 Constant Volume Sample, Federal Test Procedure). Where dictated
by the design of the statistical analysis, only one test per vehicle2 was
used. The Identification of which data set was used 1s provided in the
description of each analysis.
A review of data collected on vehicles for the 1977 model year is
provided 1n exhibit D-2. Of the 335 vehicles in this exhibit, a total of
313 vehicles passed, and 22 vehicles failed certification. Although more
type B vehicles than type A vehicles failed certification, the proportion
of type B vehicles that failed 1s not significantly different than the pro-
protlon of type A vehicles.3 There are a number of retests performed for
reasons other than failure. A review of codes for retest and discussions
with the Certification Division personnel revefeled that most of these retests
Shat is, they were performed on the data associated with the 335 vehicles
for model year 1977 whose distribution across manufacturers 1s shown in
exhibit A-l. These vehicles represent 88 percent of the 1977 emission data
vehicles used to certify light duty vehicle engine families for the nine
manufacturers shown (or approximately 60 percent of all such vehicles for
all manufacturers).
2
In order to avoid potential bias introduced by systematic changes to the
Vehicle between tests, the first test in each series was selected for
analysis.
3
See section D.l of appendix D for analysis.
-------
66
are either the consequence of a manufacturer's request1 or of the need to
perform additional fuel economy tests.
Exhibit A-3 provides a listing of the set of vehicles in VRI's data
base. To facilitate analysis, each emission data vehicle in the study data
base was assigned a unique mnemonic code which permitted rapid segregation
of the vehicles by manufacturer, model year, and engine family. The VRI
codes shown in exhibit A-3 consist of three separate fields. The first
field contains two characters, an alphabetic character for the manufacturer
followed by the last digit of the vehicle model year. The remaining two
fields provide sequence numbers for the engine family and emission data
vehicle, respectively. Alphabetic codes for the manufacturers are given
below and at the top of each page of exhibit A-3.
MANUFACTURER
CODE
Chrysler
Crt—~*-*
Ford
F*-**-*
GM
G*-**-*
BMW
B*-**-*
Oatsun
Honda
H*-**-*
Fiat
I*-**-*
Mazda
Volvo
^ne reason for permitting such retests is to permit vehicles to pass
California standards.
-------
67
EXHIBIT A-l: NUMBERS OF EMISSION DATA VEHICLES
IN VRI DATA BASE
MANUFACTURER
MODEL YEAR
1976
1977
1976 + 1977
Chrysler
70
1 72
142
General Motors
146
116
262
Ford
32
64
96
BMW
0
8
8
Datsun
0
25
25
Flat
0
14
14
Honda
0
12
12
Mazda
0
12
12
Volvo
0
12
12
All Manufacturers
00
CM
335
583
-------
EXHIBIT A-2: SELECTED DATA FROM THE DATA BASE FOR 1977 VEHICLES
Item
Selection
Type
Chrysler
Ford
General
Motors
BMW
Datsun
Honda
Fiat
Mazda
Volvo
Certified
Vehicles
A
B
15
51
22
32
35
76
4
4
13
11
4
8
6
8
6
6
7
5
Certified
Vehicles
That Failed
One Test
A
B
2
3
1
4
4
3
0
0
0
0
0
1
0
0
0
0
2
0
Failed
Vehicles
A
B
0
6
4
6
1
4
0
0
0
1
0
0
0
0
0
0
0
0
Vehicles
With One
Or More
Retests
A
B
5
22
6
12
7
13
1
1
2
1
1
1
1
3
0
1
3
3
Average
Number
A
1.53
1.23
1.19
1.25
1.15
1.25
1.17
1.00
1.43
Of Tests
Per Vehicle
B
1.40
1.32
1.16
1.25
1.08
1.13
1.38
1.17
1.60
Total
Number
A
15
26
36
4
13
4
6
6
7
Of
Vehicles
B
57
38
80
4
12
8
8
6
5
-------
69
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION (C = CHRYSLER)
Manufacturer Codes
VRI ID Code Engine Family Vehicle
C 5 - 1-1 FA-360-4-P 2069
C 5 - 1-2 FA-360-4-P 2074
C 6- 1-1 CB-440-4HP-P7S 4171
C 6- 1-2 CB-440-4HP-P7S 4180
C 6- 1-3 CB-440-4HP-P7S 4188
C 6- 2-1 CC-440-4ST-P5L 4169
C 6- 2-2 CC-440-4ST-P5L 4170
C6- 2-3 CC-440-4ST-P5L 4172
C 6 - 2-4 CC-440-4ST-P5L 4225
C 6- 3-1 CD-225-1-P5S 4151
C6- 3-2 CD-225-1-P5S 4152
C 6 - 3-3 CD-225-1-P5S 4154
C 6- 3-4 CD-225-1-P5S 4155
C6- 3-5 CD-225-1-P5S 4175
C 6 - 3-6 CD-225-1-P5S 4182
C 6 - 4-1 CD-318-2-P5S 4156
C6- 4-2 CD-318-2-P5S 4159
C6- 4-3 CD-318-2-P5S 4176
C6- 4-4 CD-318-2-P5S 4177
C6- 4-5 CD-318-2-P5S 4178
C 6- 5-1 CD-360-4-P5S 4158
C6- 5-2 CD-360-4-P5S 4160
C6- 5-3 CD-360-4-P5S 4161
C 6- 5-4 CD-360-4-P5S 4162
C6- 5-5 CD-360-4-P5S 4163
C6- 5-6 CD-360-4-P5S 4173
C6- 5-7 CD-360-4-P5S 4174
C6- 6-1 CD-400-4-P5S 4157
C6- 6-2 CD-400-4-P5S 4166
C6- 6-3 CD-400-4-P5S 4168
C 6- 6-4 CD-400-4-P5S 4179
C6- 7-1 FA-318-2-P 4057
C6- 7-2 FA-318-2-P 4094
C6- 7-3 FA-318-2-P 4095
C6- 7-4 FA-318-2-P 4096
C6- 7-5 FA-318-2-P 4097
C6- 7-6 FA-318-2-P 4109
C6- 8-1 FB-400-4-P 4065
C6- 9-1 FB-440-4HP-7S 4075
C6- 9-2 FB-440-4HP-7S 4076
C6- 9-3 FB-440-4HP-7S 4103
C6- 9-4 FB-440-4HP-7S 4107
C 6-10-1 FC-440-4ST-5L 4063
C6-10-2 FC-440-4ST-5L 4064
C6-10-3 FC-440-4ST-5L 4066
C 6-10-4 FC-440-4ST-5L 4074
C 6-11-1 FD-225-1-5S 4084
Cfr-11-2 FD-225-1-5S 4085
-------
70
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION (C = CHRYSLER)
(Continued)
Manufacturer
Codes
VRI ID Code
Engine Family
Vehicle
C6-11-3
FD-225-1-5S
4093
C6-12-1
FD-225-1-5SS
4053
C6-12-2
FD-225-1-5SS
4055
C 6-12-3
FD-225-1-5SS
4060
C 6-12-4
FD-225-1-5SS
4067
C6-12-5
FD-225-1-5SS
4068
C6-13-1
FD-318-2-5SS
4058
C6-13-2
FD-318-2-5SS
4061
C6-13-3
FD-318-2-5S S
4062
C6-13-4
FD-318-2-5SS
4069
C6-13-6
FD-318-2-5SS
4091
C6-13-7
FD-318-2-5SS
4092
C6-13-8
FD-318-2-5SS
4100
C 6-14-1
FD-360-2-5S
1035
C6-14-2
FD-360-2-5S
4071
C 6-14-3
FD-360-2-5S
4072
C 6-14-4
FD-360-2-5S
4073
C6-15-1
FD-400-2-5S
4079
G 6-15-2
FD-400-2-5S
4089
C6-16-1
FE-400-4-EM
4080
C6-16-2
FE-400-4-EM
4081
C6-16-3
FE-400-4-EM
4082
C6-16-4
FE-400-4-EM
4083
C6-16-5
FE-400-4-EM
4106
C 7 - 1-1
CD-225-1-EP
9101
C 7- 1-2
CD-225-1-EP
9102
C 7- 1-3
CD-225-1-EP
9103
C 7 - 1-4
CD-225-1-EP
9104
C 7 - 1-5
CD-2 25-1-EP
9105
C7- 2-1
CD-318-2-GP
9106
C 7- 2-2
CD-318-2-GP
9107
C 7 - 2-3
CD-3 18-2 -GP
9108
C7- 2-4
CD-318-2-GP
9109
C7- 2-5
CD-3 18-2 -GP
9116
C 7- 3-1
CD-440-4HP-FP
9114
C 7- 3-2
CD-440-4HP-FP
9115
C 7- 4-1
CD-440-4ST-GEP
9128
C 7- 4-2
CD-440-4ST-GEP
9129
C 7- 4-3
CD-440-4S T-GEP
9130
C 7- 5-1
CD-360-4-GP
91 10
C 7- 5-2
CD-360-4-GP
91 12
C 7- 5-3
CD-3 60-4-G P
9113
C 7- 5-4
CD-360-4-GP
9132
C 7- 5-5
CD-3 60-4-GP
9117
C 7- 7-1
FA-400-4-NE
9123
C 7- 7-2
FA-400-4-NE
9124
C 7- 7-3
FA-400-4-NE
9126
C 7- 7-4
FA-400-4-NE
9161
-------
71
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION
-------
72
EXHIBIT A-3:
DATA BASE VEHICLE IDENTIFICATION
(F = FORD)
(Continued)
Manufacturer
Codes
VRI ID Code
Engine Family
Vehicle
F6- 1-1
20 OA(1CEF)
5Q1-200-4-7G
F6- 1-2
20 OA(1CEF)
5Q2-200-4-1 1G
F6- 2-1
250(1CEF)
5K1-250-4-4C
F6- 2-2
25 0( 1CEF)
6B 1 -250-4-4F
F6- 2-3
250(1CEF)
6D2-250-4-1F
F6- 2-4
250(1CEF)
6H1 -250-4-6F
F6- 2-5
250(1CEF)
6K2-2 50-4-5F
F6- 3-1
302(2CMF)
5Z 1 -3 0 2-4-2 7 3C
F6- 3-2
302(2CMF)
6Z1-302-4-1F
F6- 3-3
302(2CMF)
6Z1-302-4-3F
F6- 3-4
302(2CMF)
6Z 1-302-4-9C
F6- 4-1
30 2"A"(1CEF)
5Q1-302-4-266G
F6- 5-1
302"A"(1CET)
5Q1-302-4-265D
F6- 6-1
351M/400(2CET)
6A1-351M-4-4R
F6- 6-2
351M/400(2CET)
6A1-400-4-1R
F6- 6-3
351M/400(2CET)
6A1-400-4-17R
F6- 6-4
351M/400(2CET)
6A1-400-4-2R
F6- 6-5
351M/4 00(2C ET)
6A1-400-4-28R
F6- 6-6
351M/400(2CET)
6C1-351M-4-24R
F6- 6-7
351M/400(2C ET)
60 1 -3 51M-4 -1 4R
F6- 6-8
351M/400(2CET)
601-400-4-21R
F6- 6-9
35 1M/400(2CET)
601-400-4-3R
F6- 7-1
351W(1CEF)
6H1-351W-4-7F
F6- 8-1
460"A"(2CMT)
6A1-460-4-2D
F6- 8-2
460"A"(2CMT)
6L1-460-4-12G
F6- 8-3
460"A"(2CMT)
6L1-460-4-4D
F6- 8-4
46 0"A"(2CMT)
6L1-460-4-9D
F6- 8-5
46 0"A"(2CMT)
601-460-4-1OG
F6- 8-6
460"A"(2CMT)
60 1 -460-4-2 9D
F6- 8-7
460"A"(2CMT)
6S1-460-4-22G
F6- 8-8
460"A"(2CMT)
6S 1-460-4-2 4D
F6- 8-9
46 0"A"(2CMT)
6S1-460-4-3D
F 7- 1-1
F2.3A(1CV1)
7E1-2. 3-C-35
F 7- 1-2
F2.3A(1CV1)
7E2-2. 3-C-23
F 7- 1-3
F2.3A(1CV1)
7E2-2.3-C-29
F7- 1-4
F2.3A(1CV1)
7Y1-2. 3-C-3 4
F 7 - 2-1
F2.3A(1CV5)
7E1-2. 3-F-2 5
F 7- 2-2
F2.3A( 1CV5)
7E2-2. 3-F-33
F 7- 2-3
F2.3A(1CV5)
7Y1-2. 3-F-4 3
F 7- 2-4
F2.3A(1CV5)
7Z2-2. 3-F-2 2
F 7- 2-5
F 2.3A(1CV5)
7Z1-2. 3-F-26
F 7- 4-1
F2.8B(1CV5)
7E1-2. 8-F-23
F 7- 4-2
F2.8B (1CV5)
7E1-2. 8-F-33
F 7 - 4-3
F2.8B(1CV5)
7Z2-2. 8-F-25
F 7- 5-1
F2.8BV(1CV1)
7E1-2. 8-C-35
F7- 5-2
F2.8BV(1CV1)
7Y1-2.8-C-27
F7- 6-1
F200A(1CV5)
7K1-200-F-1 1
F 7- 6-2
F200A(1CV5)
7K2-200-F-10
-------
73
EXHIBIT A-3:
DATA BASE VEHICLE IDENTIFICATION (F
(Continued)
FORD)
VRI ID Code
F 7-
F7-
F 7-
F7-
F 7 -
F7-
F 7-
F 7 -
F7-
F7-
F 7 -
6-3
7-1
7-2
8-1
8-2
8-3
8-4
8-5
8-6
9-1
9-2
F7-10-1
F7-10-2
F 7-10-3
F7-11-1
F 7-12-1
F 7-12-2
F 7-12-3
F 7-13-1
F 7-14-1
F 7-14-2
F 7-14-3
F7-14-4
F7-14-5
F 7-14-6
F7-14-7
F7-14-8
F7-15-1
F 7-15-2
F7-15-3
F 7-15-4
F 7-15-5
F 7-15-6
F7-15-7
F7-16-1
F7-16-2
F7-16-3
F 7-16-4
F 7-17-1
F 7-1 7-2
F7-17-3
F 7-18-1
F 7-1 9-1
F 7-19-2
F 7-19-3
F 7-2*0-1
F 7-2 0-2
F 7-20-3
Manufacturer
Engine Family
F200A(1CV5)
F250A(1CV1)
F250A(1CV1)
F250A(1CV5)
F25 0A(1CV5)
F250A(1CV5)
F250A(1CV5)
F250A(1CV5)
F250A(1CV5)
F302A(1CV5)
F302A(1CV5)
F302AV(ICVI)
F302AV(1CVI)
F302AV(1CV1)
F302C(2CV4)
F302DCICV5)
F302D (1CV5)
F302D(1CV5)
F351MA(2CV1
F351MB(2CV1
F351MB(2CV1
F351MB(2CV1
F351MB(2CVl
F351MB(2CV1
F351MB(2CV1
F351MB(2CV1
F3 51MB(2CV1
F351MD(2CV4
F351MD(2CV4
F351MD(2CV4
F3 51MD(2CV4
F351MD(2CV4
F351MDC2CV4
F351MD(2CV4
F351WC(ICVl
F35 tWC(IC Vi
F351WC(1CV1
F351WC(1CV1
F351WC(1CV3
F351WC(1CV3
F351WCC1CV3
F351WC(2CV4
F460A(2CVI>
F460A{2CVI)
F460A(2CV1)
F460B(2CV4)
F460B(2CV4)
F460BC2CV4)
Codes
Vehicle
7D2-200-F-9
7B1-250-C-28
7D1-250-C-2 7
7B1-250-F-24
7B1-250-F-43
7D1-250-F-21
7D2-250-F -26
7D2-250-F -42
7K2-2 50-F-2 3
7H2-302-F-32
7Z2-302-F-30
7B1-302-C-37
7D1-302-C-36
7Z1-302-C-38
701-302-F-40
7D1-302-F-31
7K1-302-F-35
7Z1-302-F-33
701-400-B-42
7A1-351M-F-57
7AI-351M-F-61
7A1 -400-F -32
701-351M-F-34
70 1 -400-F -33
701-400-F-47
701-400-F-65
7S1-351M-F-45
7A1-351M-F -91
7A1-400-F-56
701-351M-F -54
701-400-F-5 2
701-400-F-78
701-400-F-85
7W1-3 51M-89
7BI-351W-F —51
7HI-351W-F-34
7H1-351W-F -42
7BI-351W-F-33
7D1-351W-F-39
701-351W-F-46
7HI-3 51W-F-41
701-351W-F-36
7A1-460-F-19
7M1-460-F-1 8
7^1-460-F-20
7A 1-460-F-14
7Mi*460«F-34
7V1-460-F-II
-------
74
EXHIBIT A-3:
DATA BASE VEHICLE IDENTIFICATION (F =
FORD, G = GM)
(Continued)
Manufacturer
Codes
VRI ID Code
Engine Family
Vehicle
F7-20-4
F460B(2CV4)
7V 1-460-F-2 2
G 6 - 1-1
IOC ID
84 IE
G 6- 1-2
10C1D
843E
G 6- 1-3
IOC ID
844E
G 6- 1-4
IOC ID
845E
G6- 2-1
IOC 2
81 IE
G6- 2-2
IOC 2
81 3E
G6- 2-3
IOC 2
81 4E
G6- 2-4
IOC 2
815E
G6- 2-5
IOC 2
81 6E
G6- 3-1
10FIN
53 IE
G6- 4-1
1 OF 1
51 IE
G6- 4-2
10F1
513E
G 6- 4-3
10F1
514E
G 6 - 4-4
10F 1
515E1
G6- 4-5
1 OF 1
516E
G6- 5-1
10G2
11 IE
G6- 5-2
10G 2
112E
G6- 5-3
10G2
113E
G6- 5-4
10G2
114E
G 6- 5-5
10G2
115E
G6- 5-6
10G2
118E
G6- 6-1
10J2
57 IE
G6- 6-2
10J 2
573E
G6- 6-3
10J2
574E
G 6 — 6 —4
10J2
5 7 5E
G6- 7-1
1 0 J 4
5 6 IE
G6- 7-2
10J4
563E
G6- 7-3
10J4
56 4E
G 6 - 8-1
1GK4J
13 IE
G 6 - 8-2
10K4J
132E
G 6 - 8-3
10K4J
133E
G 6- 8-4
10K4J
134E
G 6 - 8-5
10K4J
135E
G 6 - 8-6
10K4J
136E
G6- 9-1
10R4
54 IE
G6- 9-2
10R4
544E
G6-10-1
10W1
12 IE
G6-10-2
10W1
122E
G 6-10-3
10W1
123E
G6-10-4
10W1
124E
G6-11-1
11B 0
83 1C
G6-11-2
11B0
833C
G6-12-1
11C2
82 1C
G6-12-2
11C 2
823C
G6-12-3
11C 2
824C
G6-12-4
11C 2
825C
G6-12-5
11C 2
82 6C
-------
75
EXHIBIT A-3:
DATA BASE VEHICLE IDENTIFICATION (G
= GM)
(Continued)
Manufacturer Codes
VRI ID Code
Eneine Family
Vehicle
G6-L 3-1
1 IF 1
52 1C
G6-13-2
1 IF 1
52 3C
G6-13-3
1 IF 1
524C
G6-14-1
11G2
501C
G6-14-2
11G2
502C
G6-15-1
11K4
591C
G6-15-2
11K4
592C
G6-15-3
11K4
593C
G6-15-4
11K4
594C
G6-15-5
11K4
595C
G6-15-6
11K4
596C
G6-16-1
liWlV
181C
G6-16-2
11W1V
182C
G6-16-3
11WIV
183C
G6-16-4
11W1V
184C
G6-17-1
20K2
21 LEI
G6-17-2
20K2
212E1
G6-17-3
20K2
213E
G 6-1 7-4
2 OKI 2
214E
G6-18-L
2004
3 2 IE
G6-18-2
2004
222E
G6-1 8-3
2004
223E
G6-18-4
2004
224E
G6-18-5
2004
225E
G6-18-6
2004
22 6E
G6-19-1
2054E
26 IE
G6-19-2
2054E
263E
G6-19-3
2054E
264E
G6-20-1
21K4
281C
G6-20-2
21K4
282C
G6-20-3
21K4
283C
G6-20-4
21K4
284C
G 6—2 0—5
21K4
285C
G6-21-1
2 IS 4
271C
G6-21-2
2 LS 4
271C1
G6-21-3
21S4
27 3C
G6-21-4
2 IS 4
27 3C1
G6-21-5
2 IS 4
274C
G 6-2 2-1
30H2J
371E
G6-22-2
30H2J
3 73E
G6-22-3
30H2J
374E
G6-22-4
3 OH 2 J
375EL
G6-23-1
30J 4
303 IE
G6-23-2
30 J 4
3034E
G 6-2 3-3
30J4
30 37E
G6-24-1
30S4
3044E
G6-24-2
30S4
3046E
G 6-24-3
30S4
3Q49E
-------
76
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION (G - GM)
(Continued)
Manufacturer Codes
VRI ID Code
Engine Family
Vehicle
G 6-2 4-4
30S4
333E
G6-24-5
30S4
334E
G6-25-1
31H2
3081C
G6-25-2
31H2
3083C 1
G6-26-1
31H2J
381C
G6-26-2
31H2J
383C
G 6-2 6-3
31H2J
384C
G6-26-4
31H2J
385C
G6-27-1
31J4
3051C
G6-27-2
31J4
3053C
G 6-2 8-1
31S 4
3141C
G 6-2 8-2
31S4
3143C
G6-28-3
3 IS 4
3144G
G 6-2 8-4
31S 4
3145C-1
G6-29-1
40E2Z
41 IE
G6-29-2
40E2Z
413E
G6-29-3
40E2Z
414E1
G6-29-4
40E2Z
415E
G6-29-5
40E2Z
416E
G6-30-1
40 J 2
42 IE
G6-30-2
40J2
423E
G6-31-1
40J4
441E
G6-31-2
40 J 4
443E
G6-31-3
40 J 4
444E
G6-32-1
40S4
461E
G6-32-2
40S4
463E
G6-32-3
40S4
464E
G6-33-1
41E2Z
4019C
G6-34-1
41E22
403C
G 6-34-2
41E22
404C1
G6-34-3
41E22
405C
G6-35-1
41J4
451C
G6-35-2
41J4
453C
G6-35-3
41J4
453C1
G6-35-4
41J4
454C
G6-36-1
41S4
471C
G 6-3 6-2
41S 4
473C
G 6-3 6-3
41S4
474C
G6-37-1
60J0
60 IE
G6-38-1
60V0
6 02 IE
G6-38-2
60V0
6022E
G6-39-1
60V 4
62 IE
G6-39-2
60V4
623E
G 6-40-1
61 JO
611C
G 6-4 0-2
61 JO
611C1
G6-41-1
61VO
602 1C
G6-41-2
61V0
6022C
G6-42-1
61V4
63 1C
-------
77
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION (G = GM)
(Continued)
Manufacturer Codes
V"RI ID Code
Enaiae Famllv
Vehicle
G 6-4 2-2
61V4
633C
G6-42-3
61V4
633C1
G6-43-1
740E2
7402F4
G7- 1-1
7 IOC 2
7125C6
G7- 1-2
710C2
7125F1
G 7 - 1-3
710C2
7125F2
G 7- 1-4
710C2
7125F3
G 7- 1-5
7IOC 2
7125F4
G7- 1-6
710C2
7125F5
G7- 2-1
71 OF1H
7143F1
G7- 2-2
71 OF1H
7143F3
G7- 2-3
71 OF1H
7143F4
G7- 2-4
71 OF1H
7L43F4-1
G7- 2-5
71 OF1H
7143F5
G7- 2-6
710F1H
7143F6
G 7- 3-1
710FISMU
7148C1
G7- 3-2
71 OF ISMU
7148C3
G7- 3-3
71 OF1SMU
7148C4
G7- 3-4
710FISMU
7148C5
G7- 4-1
710JA
7160F1
G 7 — 4 —2
710J4
7160F3
G7- 4-3
710J4
7160F4
G7- 4-4
710J4
7160F5
G7- 4-5
710J4
7160F6
G 7- 5-1
710J4S
7168C1
G 7 — 5 —2
710J4S
7168C3
G7- 5-3
710J4S
7168C6
G7- 5-4
710J4S
7168F4
G7- 5-5
710J4S
7168F5
G7- 6-1
710W1
71 OOF 1
G7- 6-2
710W1
7100F2
G7- 6-3
710W1
7100F3
G7- 6-4
71 OWl
7100F4
G7- 7-1
710W1QU
7113C 1
G 7- 7-2
710W1QC
7J13C2
G 7 - 7-3
710W1QU
7113C3
G7- 7-4
710W1QU
7113C4
G7- 8-1
710Y2
715071
G7- 8-2
710Y2
7150F3
G7- 8-3
710Y2
7150F4
G 7 - 8-4
710Y2
7150F5
G7- 8-5
710Y2
7150F6
G7- 9-1
710Y2V
7156C1
G7- 9-2
710Y2V
7156C3
G7- 9-3
710Y2V
7156C4
G*- 9-4
710Y2V
7156C5
G 7-10-1
720K4EH
7243F1
G 7-1 0-2
720K4EH
7243F1-1
-------
78
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION (G = GM)
(Continued)
Manufacturer Codes
VRI ID Code
Engine Family
Vehicle
G7-10-3
720K4EH
7243F2
G 7-10-4
720K4EH
7243F3
G 7-10-5
720K4EH
7243F4
G7-10-6
720K4EH
7243F5
G7-10-7
720K4EH
7243F6
G7-11-1
720S2E
7222F1
G7-11-2
720S2E
7222F3
G7-11-3
720S2E
7222F4
G7-11-4
720S2E
7222F5
G7-11-5
720S2E
7222F6
G 7-12-1
720X2E
7211C 1
G 7-12-2
720X2E
7211C3
G 7-13-1
720X2U
72 OOF 1
G 7-13-2
720X2U
72 OOF 2
G 7-13-3
720X2U
7200F3
G 7-13-4
720X2U
7200F5
G 7-13-5
720X2U
7200F6
G 7-14-1
730H2U
7300F1
G7-14-2
730H2U
7300F3
G7-14-3
730H2U
7300F4-1
G7-14-4
730H2U
7300F5
G 7-14-5
730H2U
7300F 5-1
G 7-14-6
730H2U
7300F6
G7-15-1
7 30M4AU
7331C 1
G7-15-2
73 0M4AU
7331C2
G7-15-3
730M4AU
7331C3
G7-15-4
730M4AU
7331C4
G7-15-5
730M4AU
7331C5
G 7-15-6
730M4AU
7331C 6
G 7-16-1
730M4U
7321F1
G7-16-2
730M4U
73 21F 3
G 7-16-3
730M4U
73 2 IF 4-1
G 7-16-4
730M4U
73 2 IF 5
G 7-16-5
730M4U
73 2 IF 6-1
G 7-17-1
730P4UY
7341F1
G 7-1 7-2
730P4UY
7341F3
G7-17-3
730P4UY
7341F4
G 7-17-4
730P4UY
7341F 5
G 7-18-1
740E2
7402F1
G 7-18-2
740E2
7402F2
G7-18-3
740E2
7402F3
G7-18-4
740E2
7402F5
G7-18-5
740E2
7402F6
G 7-18-6
740E2
7402F4
G7-19-1
740E2LU
7410C1
G 7-1 9-2
740E2LU
7410C2
G 7-19-3
740E2LU
74 IOC 3
G 7-19-4
740E2LU
7410C4
-------
79
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION
(G = GM, B = BMW, D = DATSUN)
(Continued)
Manufacturer
Codes
Li ID Code
Ermine Family
Vehicle
G7-19-5
740E2LU
7410C5
G 7-1 9-6
740E2LU
74 IOC 6
G 7-2 0-1
740J2
7420F1
G 7-2 0-2
740J2
7420F3
G 7-21-1
740J4U
7440F1
G 7-21-2
740J4U
7440F3
G7-21-3
740J4U
7440F4
G 7-2 2-1
760J0U
7600F1
G 7-2 2-2
760J0U
7600F3
G 7-2 3-1
760J0
7601C1
G7-24-1
7 60V0
7 64 IF 1
G7-24-2
760V0
7641F3
G 7-2 4-3
760V0
7 6 41 F'4
G 7-2 4-4
760V0
7641F5
G7-25-1
760V4S
7631CI
G 7-2 5-2
760V4S
7 631C 3
G 7-2 5-3
760V4S
7631C4
G7-25-4
760V4S
7631C5
G7-26-1
76QV4U
762 OF 1
G7-26-2
760V4U
7620F2
G 7-2 6-3
760V4U
762 OF 3
G 7-26-4
760V4U
7620F4
G 7-2 6-5
760V4U
7620F5
G7-26-6
760V4U
762 OF 6
B 7 - 1-1
BMW 120.8
5 400 001
B 7- 1-2
BMW 120.8
5 460 001
B7- 2-1
BMW 120.9
5 420 004
B7- 2-2
BMW 120.9
5 470 002
B 7 - 3-1
BMW 130.8
4 375 045
B 7- 3-2
BMW 130.8
4 375 046
B 7- 3-3
BMW 130.8
5 Oil 506
B 7- 3-4
BMW 130.8
5 031 418
D7- 1-1
A140C
AK0434
D7- 1-2
A140C
AK0436
D 7 - 1-3
A140C
AK0452
D 7- 1-4
A140C
A644
D7- 1-5
A140C
A645
D7- 2-1
A140F
AK0433
D 7- 2-2
A140F
AK0435
D 7- 2-3
A140F
AK0451
D7- 2-4
A140F
A643
D7- 3-1
A141F
AK0472
D 7- 4-1
L200C
AK0414
D7- 4-2
L200C
BWO158
D7- 4-3
L200C
BWQ15 9
D 7 - 4-4
L200C
B 1592
D7- 5-1
L200F
AK0415
D7- 5-3
L200F
BK0397
-------
80
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION
(D = DATSUN,
H = HONDA, I = FIAT, M = MAZDA, V
(Continued)
= VOLVO)
Manufacturer
Codes
II ID Code
Engine Family
Vehicle
D7- 5-4
L200F
BW0157
D7- 6-1
L240C
BW0161
D 7 - 6-2
L240C
B 15 96
D 7 - 7-1
L240F
BW0160
D 7 - 7-2
L240F
B 15 94
D 7 - 8-1
L280C
F565
D 7 - 8-2
L28QC
F 5 6 7
D 7 - 9-1
L280F
F566
D 7 - 9-2
L280F
F568
H 7 - 1-1
77EB
SBC-5000014
H 7- 1-2
77EB
SBD-5000013
H 7- 2-2
77ED-1
SG-E3000010
H7- 2-3
77ED-1
SJ-D2000005
H7- 2-4
77ED-1
SJ-E2000006
H 7 - 2-5
77ED-1
WB-A3000009
H7- 2-6
77ED-1
WB-B3000008
H 7- 3-1
77ED-2
SG-E3000007
H7- 3-2
77ED-2
SJ-D2000001
H7- 3-3
77ED-2
SJ-E2000004
H 7- 3-4
77ED-2
WB-A3000007
H 7 - 3-5
77ED-2
WB-B3000001
17- 1-1
128
0044125
17- 1-2
128
2122971
17- 2-1
128-CC1
0049449
17- 2-2
128-CC1
2088127
17- 3-2
132
0152508
17- 3-3
132
0159304
17- 3-5
132
0605152
17- 3-6
132
4054941
17- 4-1
132-CC1
0100101
17- 4-2
132-CC1
0108238
17- 4-3
132-CC1
0503001
17- 4-4
132-CC1
0606502
17- 4-5
132-CC1
4 0 4 6 4*7 7
17- 4-6
132-CC1
4056032
M 7 - 1-1
CNAP
77ECNAP-1
M 7 - 1-2
CNAP
77ECNAP-2
M7- 2-1
CTCP
77ECTCP-1
M7- 2-2
CTCP
7 7ECTCP-2
M7- 3-1
FNAP
7 7EFNA-1
M7- 3-2
FNAP
7 7EFNA-2
M7- 4-1
FTC P
77EFTCP-1
H7- 4-2
FTCP
77EFTCP-2
M7- 5-1
REP
77EREP-1
M7- 5-2
REP
7 7EREP-2
M7- 5-3
REP
77EREP-3
M7 - 5-4
REP
7 7EREP-4
V7- 1-1
4CA
77:12
-------
81
EXHIBIT A-3: DATA BASE VEHICLE IDENTIFICATION (V » VOLVO)
(Concluded)
Manufacturer Codes
VRI ID Code Engine Family Vehicle
V 7- 1-2
4CA
77
13
V7 - 1-3
4CA
77
14
V 7- 1-4
4C A
77
15
V7- 2-1
4FA
77
5
V7- 2-2
4FA
77
6
V7- 2-3
4FA
77
7
V7- 2-5
4FA
77
8
V7- 3-1
6CA
77
16
V7- 3-2
6C A
77
17
V7- 4-1
6F A
77
10
V7- 4-2
6FA
77
9
-------
82
-------
83
APPENDIX B
ANALYSIS OF EMISSION TEST VARIABILITY
A prerequisite to performing many of the analyses associated with the
study was the selection of an appropriate estimate of exhaust emission test
variability. A number of such estimates have been reported by previous investi-
gators [Matula, 1974], [Paulsell and Kruse, 1975], [Juneja, et at. , 1976],
[Juneja, et at1977], and [Ford, undated b]. These studies generally report the
test variability in terms of the standard deviation as a percent1 of the average
emission level for each pollutant. Such a percent assumes that test-to-test
variability increases in direct proportion to any Increase 1n the average
emission level of the vehicle being tested.2 Since there 1s evidence3 which
suggests this may not be the case and since the current study had access to
multiple test results from a number of vehicles (see the previous appendix),
1t was decided that this data might be used to provide Insight concerning this
relationship and further provide estimates of test variability which represent
the variability experienced in conducting the certification program.1*
The benefits of performing this analysis of emission test variability
are fourfold. First, the analysis permitted an examination of the functional
relationship between test variability and mean vehicle emissions. The out-
come of this examination would 1n part dictate the appropriate statistical
*Th1s percent is generally referred to as the coefficient of variation.
2That 1s, 1f this were not the case, each vehicle with different emission
levels would have a different coefficient of variation.
3For example, 1n Ford [uhdated-a], the analysis of test variability suggest
a nonlinear relationship between the test standard deviation and the mean
emission level.
'~That 1s, the analyses performed 1n the study require an estimate of test
variability which Includes variations not controlled for 1n EPA's certi-
fication testing program.
-------
84
techniques to be employed in analyzing the results of emission tests on differs
vehicles.1 Second, the results of this analysis might be used to predict
the degree of change in test variability with anticipated changes in the
emission levels of vehicles as a consequence of new standards.2 Third, the
analysis permits comparison of estimates of test variation recently
experienced in EPA's test facility to that of the manufacturers to determine
whether the manufacturer's estimates could be adopted for use in this study.3
Finally, it provides a mechanism to investigate the degree to which the test
data used in this study appeared representative and reasonable (i.e., one
of the several simple checks to ensure that the test results used here did
not contain a significant amount of error).
B.1 Approach
The analysts of emission test variability was conducted in three phases,
First, the dependence of test-to-test variability on vehicle emission levels
xThat Is, if the test variation were determined to be a constant and inde-
pendent from the average emission level of a vehicle, then the analyses
which used multiple vehicle results would not have to be adjusted to
account for differences in test variation,
2Such changes in test variability would influence the outcome of the effect-
iveness assessment methodology (discussed in appendix F) in that they would
alter the probability of miscertifying a vehicle.
3Although some estimates of test variability have been made using the EPA
facility data, they are somewhat outdated and for the most part relied on
a single vehicle rather than using several vehicles which is more applicable
to the current analysis. Therefore, it was considered desirable to be able
to use the more recent estimates made by the manufacturers under similar
controlled conditions.
-------
85
was examined by attempting to fit data to several models relating these
quantities. Second, using the model selected as appropriate an estimate of
test-to-test variance was obtained by combining the estimates of variance
from individual vehicles with multiple tests. Finally, the estimates of test
to-test variance were compared with those developed in previous studies and
appropriate values for test variability selected. The following paragraphs
discuss each of these steps in greater detail.
To examine the relationship between test variability and average
emission levels a special data base was created consisting of test results
from 1977 emission data vehicles CEDVs), with multiple tests performed on
the same vehicle using the same test procedure. The identification of
multiple tests was based on a screening of vehicle and test information to
ensure that multiple tests recorded for a given vehicle ID were performed
on the same vehicle using the same test procedure.1 The number of vehicles
and average number of tests per vehicle passing this screen is given in
exhibit B-l.2
1S1nce this screening used data contained in EPA's Light Duty Vehicle Data
System which does not record all operations performed on a vehicle between
tests (e.g., recallbratIon), 1t 1s possible that the vehicle may have
undergone some minor change between tests. On conferring with EPA staff
most likely to know about the occurrence of such changes, 1t was discovered
that little, if any, quantitative information (e.g., frequency nature, or
predisposing conditions) could be obtained about these changes. It was,
however, concluded that their existence could upwardly bias any estimates of
test variability 1f vehicles were systematically altered to reduce emissions
levels and possibly downwardly bias the estimates If recalibrated to original
specifications.
20ut of the 170 tests used to analyze test-to-test variability, thedata base
codes indicate that 58 were retested because of failure to meet federal
emission standards and the remaining vehicles were retested either as a
consequence of manufacturer requests or for fuel economy reasons.
-------
86
For each vehicle in this data base the average and the standard deviation
of the measured emission levels for each pollutant (HC, CO, and NO ) were
A
calculated.1 Since the number of tests for any one vehicle were small, a
reasonable estimate of test variability (i.e., the standard deviation) could
only be obtained by combining the population of individual vehicle estimates.
To combine these estimates, catalyst vehicles were separated from non-
catalyst vehicles and then the relationship between test variability and mean
emission levels analyzed for each of these vehicle groupings. This relation-
ship was examined by plotting the vehicle sample standard deviations against
the vehicle sample means for each pollutant (as shown in appendix A, section
A.l). A number of models which related standard deviation to the average
emission level were then tested to determine which model provided the closest
fit to the data. The six models tested are shown below (where S denotes the
test-to-test standard deviation, "X denotes the average emission level of a
vehicle and a., b. and c. are constants estimated from the data):
9
(1) S = C] ;
(2) S2 = b1 I;
(3) S2 * b2I+c2;
(4) S2 = a] X 2;
(5) S2 » a2 >T 2 + c3; and
(6) S2 = a3 I 2 + b3 I.
l
The data was further screened for outliers. Since each entry in this data
base was checked for validity, the probability of a true outlier is remote.
This data was also checked for outliers, but none of the data points could
be rejected with justification.
-------
87
The choice of these models was an attempt to consider combinations of
three types of contributions to the total variance which were plausible on
theoretical grounds. Constant errors are contributed to many laboratory and
other measurement processes, and seemed a plausible type of error to find in
the emission measurement process. This possibility was also indicated as
probable by the analyses conducted in the earlier study [VRI, 1976],
Additionally, errors contributing variances proportional to the squared
emission levels seemed plausible on two bases:
0) past researchers studying the topic had often presented their results
in terms of a constant coefficient of variation, which would make
the variance proportional to the squared emission level; and
(2) many variations in dynamometer, driver, and environmental conditions
could be expected to have effects of the form
emissions ¦ 0 + effect) • "normal" emissions
(where "normal" emissions are emission which would be achieved 1n
experiments 1n which the specific effect Involved was controlled).
Effects of this kind could also be expected 1n some aspects of
the measurement process proper, where measurement errors or
calibration errors would cause measurement effects proportional
to the actual emissions.
Beyond both these types of error, there is a third type of variation which
could be expected to contribute to the total variability in the measured
emissions. This type of variation arises from the addition of a number of
independent sources of emissions (produced, for example, by the Individual
ignitions of the cylinders during the test), each of which might reasonably
be modeled as having a random number of molecules of each type of pollutant
-------
88
with the variance of the number proportional to the mean.1 A summation or
averaging of independent yariates of this third type produces a variate with
the standard deviation proportional to the square root of the mean emissions
observed, and the variance proportional to the mean emissions.
The models which were actually examined in this analysis considered
these three types of effects taken individually and in pairs, as shown above.
It was felt that there was not sufficient data to estimate reasonably a model
with, all three effects present, and, as discussed in section B.2f the data
did not in fact prove sufficient to distinguish these six cases themselves
in any conclusive manner.
Each of the above six models was fit to the data by estimating values
for a.., b., and c.. which minimized the weighted sums of squares for the corre
sponding model. The weights for this computation were established to allow
data for vehicles with differing numbers of tests to be considered together.
The measure used to compare how well each model fits the data was the mean
square error between the data and the corresponding model.
The model selected as the best or most appropriate was then used to
combine single vehicle estimates of test variability to obtain an estimate
of test-to-test variability for the EPA testing facility, This estimate
was compared to that reported elsewhere and an appropriate value chosen for
use in the subsequent analyses,
^sing a stochastic version of fairly standard chemical rate equation
methodology results in Poisson distributions of reaction products for
which this property holds.
-------
89
B.2 Results
Exhibit B-2 displays the square root of the man square error for each
of the models examined. The results are grouped in terms of vehicle manu-
facturer type (foreign versus domestic) and vehicle emission control system
(catalyst versus non-catalyst). Since the data base containing vehicles
with multiple tests had only one domestic non-catalyst vehicle, it was not
possible to analyze this category.
The results in the table do not select any one of the models for all
pollutants and all manufacturers. In fact, the results appear to Indicate
that there is very little difference among the values of the mean square
error for any model of test variability for a given pollutant. The inability
to single out clearly the most appropriate model or models is In part ex-
plained by looking at the data, which display a wide scatter (see scatter
plots in exhibits B-5 through B-131). This scatter 1s 1n part attributable
to the limited sample of tests per vehicle used to estimate the mean variance.
Even with this scatter, several observations can be made regarding the
dependence of the test-to-test variance on the mean. First, the assumption
that these quantities are proportional and have a constant coefficient of
variation (i.e., model 4} 1s generally not contradicted by the results In
exhibit B-2. The data do not, however, clearly support this model over the
others tested. Second, the model of test-variability given by model 6 had
consistently lower mean square errors for N0X emissions from catalyst equipped
*The values, are plotted for HC, CO, and N<3 separately, and are further
differentiated by vehicle category (domestic and foreign wtth catalyst
and non-catalyst distinctions).
-------
90
vehicles,1 Finally, the data from vehicles without catalysts (all foreign
vehicles in this analysis) did not generally fit the same models as vehicles
with catalysts and tended to provide better fits to models 4 and 6 for HC and
CO respectively. This is however extremely tenuous since only ten vehicles
were not equipped with a catalyst.
In view of the generally inconclusive nature of the results in exhibit B-2
it was decided to duplicate most analyses under two assumptions:
(1) that test variability was independent from mean emission level
(model 1); and
(2) that test-to-test standard deviation was directly proportional to
the mean (model 4).
The first assumption is supported by the fact that there was little signifi-
cant difference in the mean square error from one model to the next and the
second assumption was selected since it permitted comparison of test
variability results from this study to previous estimates. Exhibit B-3
compares this study's estimate of a constant test-to-test standard deviation
with that derived from results reported by previous studies.2 These results
show that standard deviation estimates using EPA test data fall within the
range of previous estimates. However, the estimate of test-to-test standard
deviation for HC and CO is higher than estimated in [Juneja, 1976] for non-
catalyst vehicles and in [Juneja, 1977] for both types of vehicles.
JThe non-linearity of this relationship is also supported by other analyses
[Ford, undated a] which found an exponential best fit curve.
2Although the data in this study contained a relatively large number of vehicle5
which failed at least one certification test, the estimate of test-to-test
standard deviation can still be compared to others under the above assumption*
if it is conjectured that the only difference between vehicles with test
failures and those without is the mean emission level.
-------
91
This potential upward bias could be attributable to vehicle recalibration
to a different but legal standard or differences between the degree to which
test conditions were controlled. The effects of such a bias on analysis per-
formed in this study would be to reduce the ability to discriminate the dif-
ference among vehicles with different emission levels, and increase the like-
lihood of presenting a more conservative testing strategy1 than necessary.
The error introduced in both cases 1s relatively small; however, since any bias
is undesirable, analyses designed to differentiate vehicle emission level
differences also employed the most recent of the "other" estimates (i.e.,
[Juneja et al.3 1977]).
Under the assumption that test variability 1s proportional to the mean
emission level, exhibit B-4 provides a comparison of this study's estimate2
of the coefficient of variation with those reported elsewhere. Results 1n
the table show that estimates from this study are 1n the range of other
estimates for HC and CO emissions with the estimated variation of the NO
emissions slightly above and below the range of estimates being reported
elsewhere.
lFor example, a strategy designed to minimize the probability of miscertifying
a vehicle would test more vehicles with Increased test variability to assure
that certification or failure was not the .result of random test error.
2Test estimates of the coefficient of variation can again be compared to others
since the effect of the higher mean emission level due to failing certification
has been accounted for.
-------
92
EXHIBIT B-1: SAMPLE SIZE AND DISTRIBUTION BY MANUFACTURER
OF VEHICLES WITH MULTIPLE TESTS UNDER
"IDENTICAL" CONDITIONS
No. of
No. of
No. of
No. of
Manufacturer
Vehicles
Tests
Manufacturer
Vehicles
Tests
Chrysler
27
58
Honda
2
4
Ford
18
36
Fiat
4
8
General Motors
20
40
Mazda
1
2
BMW
2
4
Vol vo
6
12
Datsun
3
6
TOTAL
83
170
-------
EXHIBIT B-2: SQUARE ROOT OF THE MEAN SUMS OF SQUARES FOR EACH MODEL
(MINIMUM VALUES IN EACH VEHICLE CATEGORY ARE UNDERLINED)
MODEL
HC
CO
NO,
Catalyst
Non-
Catalyst
Catalyst
Non-
Catalyst
Catalyst
Non-
Catalyst
Domestic
Foreign
Both
Foreign
Domestic
Foreign
Both
Foreign
Domestic
Foreign
Both
Foreign
(1) s2 = c,
.025
.040
.027
.025
9.486
1.408
8.867
3.214
.269
.242
.264
-W7
(2) s2 -
.024
.039
.026
.026
9.300
1.138
8.674
3.079
.261
.248
.258
.007
(3) S2 = b2X + Cg
.024
.042
.026
.027
9.361
1.080
8.723
3.236
.253
.255
.256
.007
(4) S2 = a^X2
.024
.041
.027
.028
9.425
1.008
8.792
3.037
.252
.253
.251
.007
(5) S2 = a2X2 * c3
.024
.042
.027
.027
9.447
1.065
8.808
3.272
.246
.251
.249
.007
(6) s2 > «3i2 + b3)r
.024
.04)
.026
.022
9.348
1.066
8.719
3.278
.241
.231
.245
.007
-------
94
EXHIBIT B-3: ESTIMATES OF A CONSTANT TEST-TO-
TEST STANDARD DEVIATION (MODEL 1)
This Study
[Matula,
19741
[Juneja,
19761
[Juneja,
19771
Catalyst
Non-
Catalyst
Both
Non-
Catalyst
Non-
Catalyst
Both
HC
0.094
0.126
0.101
0.35
0.068
.045
CO
1.442
1.154
1.411
1.90
0.733
.782
N0X
0.282
0.071
0.267
0.31
0.156
.096
EXHIBIT B-4: ESTIMATES OF THE COEFFICIENT OF VARIATION
OF TEST RESULTS FOR EACH POLLUTANT1
-A
This Study
[Ma tula,
19741
[Juneja, 2
1976]
[Juneja»'
1977L
Catalyst
Non-
Catalyst
Both
Non-
Catalyst
Non-
Catalyst
Both
HC
13%
13%
13%
14%
18%
10%
—
CO
14%
7%
13%
11%
26%
16%
NOx
9%
4%
9%
6.5%
7%
4%
1A11 estimates are the weighted mean of individual vehicle estimates of the
coefficient of variation (the standard deviation as a percent of the mean).
2Values are recalculated from data in table 1 of both reports which provide
slightly different values due to a difference in definition. (Juneja used the
average standard deviation for all vehicles divided by the average emission
levels of these vehicles.)
-------
95
EXHIBIT B-5: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR HC USING DATA ON 73 DOMESTIC
AND FOREIGN MANUFACTURED VEHICLES WITH CATALYSTS
.50000 +
.40000
.30000
S .20000
.10000
0.
0.
I * » *
#
I *
•• #2 • • * »
i I
#2* * * * * *
«*2 ***** **** * *
•22"2* 2 * * *
«2«h t • » » i
.60000 1.2000
,30000 .90000
1.5000
(Single observations are indicated by a *.
Multiple observations are indicated numerically.)
-------
96
EXHIBIT a-6: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR CO USING DATA ON 73 DOMESTIC
AND FOREIGN MANUFACTURED VEHICLES WITH CATALYSTS
10.000
8,DQQG
S.OODO +
+
S 4.0000
*
* *
2.0000 + * * *
» * ¦ *
2 *
+ * ****** » *» »
* a * « * « *
k"J2* * ** *** * ¦ *
0. + 3**2 * * 2
0. 8.0 0 00 16. 000
*1.0000 12.000 20.000
(Single observations are indicated by a *.
Multiple observations are indicated numerically.)
-------
97
S
EXHIBIT B-7: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR NOx USING DATA ON 73 DOMESTIC
AND FOREIGN MANUFACTURED VEHICLES WITH CATALYSTS
1 .5000
1.2000
.90000
.60000
«
* * *
.30000 +
t
* # *
+ # * #2 * *
* 2* **»2 **
2 2** * »»"**. 32* * ^ *
0. + * * *#*##« 2*** * « 2 * #*
0. 1.2000 2.4000
.60000 1.8000 3.0000
(Single ooservations are indicated by a *.
Multiple observations are Indicated numerically»)
-------
98
EXHIBIT B-8: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR HC USING DATA ON 8 FOREIGN
MANUFACTURED VEHICLES WITHOUT CATALYSTS
.50000 +
.40000 +
+
.30000
.20000 +
+
.10000 +
*
* *
0. +
+ + + + + +
o. .60000 1.2000
.30000 .90000 1.500®
(Single observations are indicated by a *.
Multiple observations are indicated numerically.)
-------
99
EXHIBIT B-9: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR CO USING DATA ON 8 FOREIGN
MANUFACTURED VEHICLES WITHOUT CATALYSTS
10.000 +
+
8.0000 +
+
6.0000 +
U.0000 +
2.0000 +
* * t •
0. + * *
0. 8.0000 16.000
*>.0000 12.000 20.000
(Single observations are Indicated by a *.
Multiple observations are indicated numerically.)
-------
100
EXHIBIT B-10: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR N0X USING DATA ON 8 FOREIGN
MANUFACTURED VEHICLES WITHOUT CATALYSTS
1 . 5000
1.2000 +
.90000 +
s +¦
.60000 +
+
.30000 +
3.0000
+ *
* *
*
0. + * * #*
1.2000 a.uooo
.60000 1.8000
(Single observations are indicated by a *.
Multiple observations are indicated numerically.)
-------
101
EXHIBIT B-ll: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR HC USING DATA ON 83 DOMESTIC
AND FOREIGN MANUFACTURED VEHICLES WITH AND WITHOUT
CATALYSTS
50000
1(0000
.30000 +
* « a
*
.20000 + *
#
+ # *
* « * »
**
.10000 + * *
** *2 * * #11 *
« * *
+ * 2* * * * * * *
********* Z *
*32*2* 2 * » * *
0. + >2"**** * * *
+ + +«._« + ««. + +
0, .60000 1.2000
.30000 .90000 1.5000
(Single observations are indicated by a*.
Multiple observations are indicated numerically.)
-------
102
EXHIBIT B-12: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR CO USING DATA ON 83 DOMESTIC
AND FOREIGN MANUFACTURED VEHICLES WITH AND WITHOUT
CATALYSTS
10.000
*
3.0000 +
+
6.0000
k.0000
2.0000 +
*
* * *
* # *
s * *
* * * *
2* *
+ * **»**«« * * * *
* s * # * # #
4* 1*72***2 2a* *** * * *
+ 3**2 * * 3 *
0. 8.0000 16.000 ,
it. 0000 12.000 20.0°
X
(Single observations are indicated by a *.
Multiple observations are indicated numerically.)
-------
103
EXHIBIT B-13: PLOT OF TEST-TO-TEST STANDARD DEVIATION VERSUS THE
MEAN EMISSION LEVEL FOR N0X USING DATA ON 83 DOMESTIC
AND FOREIGN MANUFACTURED VEHICLES WITH AND WITHOUT
CATALYSTS
1 . 5000
1.2000 +
. 90000
.60000
.30000 +
« » *
+ ***2 2*
** *2* * *2*2 **
2 3** * **»* 32* * 4 *
0. + # * *»**2* 2*2*** * * 2 * **
0. 1 . 2000 2.14000
.60000 1.8000 3.0000
(Single observations are indicated by a *.
Multiple observations are indicated numerically.)
-------
104
-------
105
APPENDIX C
ANALYSIS OF ENGINE FAMILY CLASSIFICATION
This appendix contains a description of the analytical methods used to
examine EPA's engine family classification and the detailed results of these
analyses. It is organized into three sections which parallel the discussion
presented in the three sections of chapter 2.0 in the main body of the report.
The first section describes the analysis of variance model used to investigate
the performance of the current family classification scheme. The second section
presents the results of the regression analysis which was used to analyze poten-
tial relationships between vehicle parameters and family mean emission levels.
The last section provides the information necessary to identify the engine
families used in analyzing subjectively-derived family definitions.
C. 1 Analyses of Variccnoe
To determine whether the current engine family classification grouped
vehicles with similar emission levels the following model was developed for
an analysis of variance.1 Let Y^kjl denote the measured emission level of
a given pollutant for the test on the vehicle in family i. Then the
model for the analysis of variance Is given 1n the following equation:
E1U " " + f1 + Vfk + e1U •
xThe model was first developed to include differences 1n vehicle selection-class.
The ANOVA results presented 1n appendix D showed that the selection-class factor
should be removed and that the above model was more appropriate.
-------
106
where y = the overall mean of the observations,
f^ = the effect of family i,
V.k = the effect of vehicle k and family i, and
-------
107
Overall, the consistency of results of these analyses of variance show
that there is a significant difference in the emission levels of vehicles
in different families which cannot be accounted for by the differences between
vehicle configurations within a family. Further, the ANOVA results for
configuration-to-configuration differences within a family show a significant
difference in emission levels for domestic vehicles with catalyst and foreign
vehicles without catalyst1 which cannot be explained by test-to-test variability.
To analyze whether the above difference in the mean emission levels among
families might be explained by the family deterioration factor (df), histograms
of these mean emissions were examined with and without application of the df.
If the df explained the observed difference, then the emission levels of families
with the df applied should be much more similar than those without application
of the df. The results (see exhibit C-5 and C-6) indicate that the differences
in engine family emission levels are similar under both circumstances. Therefore
the ANOVA cannot be solely explained by differences in the deterioration factor.
C. 2 Regression Analysis
For each manufacturer, a stepwise multiple linear regression was per-
formed to determine which engine parameters can be used to predict the mean
emission level for an engine family. This stepwise algorithm proceeds as
follows. The process begins by selecting the one parameter which best predicts
the mean emission level. If this regression 1s not significant at the .05
level, then no variables are selected for the regression. If it is significant,
^hen the [Ouneja^et al.t 1977] test variability estimates are used all vehicle
category results display significant configuration-to-configuration differences
within a family.
-------
108
then the algorithm looks for the next most significant variable. After a
variable has been added to the regression, the algorithm examines whether
any of the variables in the regression can be eliminated. If the least
significant variable renders a significance level of higher than .10, it is
removed from the regression. This process continues until the algorithm is
unable to add any of the variables not yet included in the regression.
Exhibits C-7 through C-9 display the values of engine parameters and
mean emission levels for each engine family in the 1977 model year data base
for domestic vehicles. The results of the stepwise regression (see exhibit C-10
on these engine parameters did not consistently select any of the engine para-
meters to predict the mean emission levels of engine families. Hence, the
results of this examination were inconclusive.
C.3 Heuristic Analysis
Two forms of heuristic analysis of engine families were performed: (1) a
review of two-way clustering by observing scatter plots of family mean emission
levels for one pollutant versus another1 and (2) an examination of subjective
family merging schemes using 1977 certification vehicle summary sheets. The
review of the scatter plots did not reveal any significant clustering of famili®*
Minor clustering for CO versus N0X was found for Chrysler families (see exhibit
C-ll).
Exhibit C-15 identifies the engine families which were subjectively analyz^
to determine whether two or more individual families might be aggregated into
Several representative scatter plots are presented in exhibits C-11 through C-^
-------
109
a single family. The exhibit shows the families which were combined on the
basis of an aggregation rule specifying that the vehicles in a family should
have similar displacements,1 carburation and engine configurations and identical
control systems. As can be seen in this exhibit, the rule only aggregated
families from manufacturers with a total of nine or more original family classes.
Similar displacements were those within 15% or 50 cubic inches of one another.
-------
no
EXHIBIT C-l: NUMBER OF EMISSION DATA VEHICLES AND
ENGINE FAMILIES PER MANUFACTURER
Manufacturer
Catalyst
Non-Catalyst
# Vehicles
# Families
# Vehicles
# Families
Chrysler
67
14
5
1
Ford
64
18
0
0
General Motors
116
26
0
0
BMW
0
0
8
3
Datsun
14
5
11
4
Honda
0
0
12
3
Flat
8
2
6
2
Mazda
6
3
6
2
Volvo
12
4
0
0
-------
EXHIBIT C-2: ANALYSIS OF VARIANCE RESULTS FOR TEST OF FAMILY DIFFERENCES
UNDER CONSTANT TEST VARIABILITY ASSUMPTION (MODEL 1)
F-Ratto Test4 For
Catalyst
Non-Catalyst
Both
Domestic
Foreign
Both
Foreign
Both
HC
€0
NQX
HC
CO
HO*
HC
CO
N0X
HC
CO
N0X
HC
CO
W>x
Family Differences
8.14
<».4)
4.64
(1.4)
4.72
(14)
14.5
(3J)
10.6
(2.2)
17 J
(2.9)
8.68
<1-4)
4.83
(1-4)
6.37
(1-4)
6.81
(2.1)
5.12
(2-1)
2.07
(2.1)
11.1
(1,3)
4.98
(1.3)
5.78
(1.3)
Configuration
Differences within
Faally
3.53
(1.4)
4.08
(K4)
1.48
(1-4)
0.35
(2.7)
1.33
(2.7)
0.38
(2.7)
2.79
(1.4)
3.98
(1.4)
1.34
(1.4)
4.81
(3.1)
3.42
(3.1)
8.52
(3.1)
2.99
(1.4)
3.B9
(1.4)
1.41
(1.4)
EXHIBIT C-3: ANALYSIS OF VARIANCE RESULTS FOR TEST OF FAMILY DIFFERENCES UNDER THE
ASSUMPTION THAT TEST VARIABILITY IS PROPORTIONAL TO THE MEAN (MODEL 4)
F-Ratio Test1 For
Catalyst
Non-Catalyst
Both
Domestic
Foreign
Both
Foreign
Both
HC
CO
MOx
HC
CO
NO*
HC
CO
NO*
HC
CO
NO*
HC
CO
N0X
Family Differences
12,4
<1-4)
6.88
(1.4)
5.62
0.4)
13.0
(2.9)
7.19
<2;2)
12.17
<2.4)
13.9
0.4)
7.06
(1 T 4)
11.3
(1.4)
8.04
(2.1)
7.06
(2.1)
2.16
(21)
14.6
(13)
7.30
(1.3)
10.2
(1.3)
Configuration
Differences within
Faatily
3.23
<1.4)
4.89
<1.4)
2.57
(1.4)
0.39
(2.7)
2.69
(2.7)
0.61
(2.7)
1.98
(14)
4.70
(1.4)
1.26
(1.4)
5,61
(3-1)
6.07
(3.1)
8.23
(3.1)
2.28
(1-4)
4.64
(1-4)
1.31
(1.4)
'F-ratlo results are significant (at the 5X level) when greater than the value in parentheses.
-------
EXHIBIT C-4: ANALYSIS OF VARIANCE RESULTS FOR TEST OF FAMILY DIFFERENCES USING TEST VARIABILITY
DATA FROM JUNEJA, ET AL [1977], Af4D ASSUMING CONSTANT VARIABILITY (MODEL 1)
F-Ratio Test1 For
Catalyst
Non-Catalyst
Both
Domestic
Foreign
Both
Foreign
Both
HC
CO
NO*
HC
CO
NO*
HC
CO
«0X
HC
CO
NOx
HC
CO
N0X
Family Differences
7.99
(1.*)
4.57
(1.4)
4.47
(1.4)
10.0
(2.2)
10.4
(2.2)
12.7
42.2)
8.46
(1.4)
4.75
(1.4)
5.98
(1.4)
6.73
(2.1)
5.07
(2.1)
2.08
(2.1)
10.8
(1.3)
4.91
(1.3)
5.47
(1.3)
Configuration
Differences within
Family
13.4
(1.3)
15.2
(1.3)
12.7
(1.3)
2.76
(1.6)
1.69
(1.6)
3.29
(1.6)
12.1
(1.3)
13.5
(1.3)
11.6
(1.3)
37.7
(1.6)
7.45
(1.6)
4.69
(1.6)
15.0
(1.3)
12.7
(1.3)
10.9
(1.3)
'F-ratlo results are significant (at the 5% level) when greater than the value in parentheses.
-------
113
EXHIBIT C-5:
HISTOGRAMS OF MEAN ENGINE FAMILY EMISSION LEVELS BY
VEHICLE MANUFACTURER (WITH AND WITHOUT APPLICATION
OF DETERIORATION FACTOR, df)
HC
HC*df
GM
(26 families)
y = .46
a =.15
0.04
1.60
_J1
r
M =.61
a *.28
yk
n
0.04
FORD
(18 families)
L
vi —. 56
o =.19
om
J=L
1.60
y *.75
24
1.60 0.04
VC,
1,60
CHRYSLER
(14 families)
y *.43
a =.27
0.04
1.60
JT,
0.04
xfl
y *.50
a *. 24
1.60
CO
y *5.8
a*2.5
13.6
1^*1 r1
y»7.1
ru cf®3
-------
114
EXHIBIT C-6:
HISTOGRAMS OF MEAN ENGINE FAMILY EMISSION
LEVELS FOR TOTAL OF THREE MANUFACTURERS
(WITH AND WITHOUT APPLICATION OF DETERIORATION
FACTOR, df)
GM, FORD AND CHRYSLER
(58 families)
0.04
vj =.49
a = .20
i
1.60
HC
"1—i u =.63
^ a =.29
m
0.04 1.60
HC*df
Ty =6.1
a=2.9
13.6
CO
y =6.8
a =3.2
HJ-H-L
X
0.40
C0*df
13.6
ru
y=l .5
cr = . 33
X
ru
0.63
2.43
N0>
J1
v =1.6
a = .32
0.f)3
NOx*df
2.43
-------
EXHIBIT C-7: MEAN OF ENGINE PARAMETERS FOR CHRYSLER
ENGINE
FAMILY
WEIGHT
CID
BORE
STROKE
sut>
m
COMP
RATIO
TIMING
RPM
AXLE
RATIO
HC
CO
*°*
i.
3
225.
3. 4ft
4. 12
90.
3. 4
7 50.
3.tl7
.36*
2.820
1.200
•> #
1*15ft «
3 18.
3.5.
b.
tt.4
7l|il.
i.'il
.927
8.600
1.873
14.
iron.
225.
3. 4U
4. 12
115.
a. 4
Jill.
3.U41
.B>I0
10.7*3
1.67M
15.
4I>44.
3i».
3.11
3. 31
145.
11.
3. 5
}I)H.
2. 7'J
.66a
10.767
1. 627
14.
4 32 5.
360.
4.0H
3. 58
170.
H.
6. 4
7iiU.
2. 66
.505
6.633
1.790
-------
EXHIBIT C-8: MEAN OF ENGINE PARAMETERS FOR FORD
ENGINE
COHP.
TIMING
AXLE
FAMILY
WEIGHT
CID
BORE
STROKE
RHP
CYL
RATIO
RPM
RATIO
HC
CO
N0*
1.
2606.
140.
3.80
3. 10
•)2.
4.
0
6 2 5.
3. 07
.*05
3. 150
.725
2.
2603.
140.
3. ao
3. 10
92.
4.
9.0
5 i! 0.
3.0 9
.612
10.480
1.232
2 15 91.
171.
3. 66
2. 70
93.
<•.
a. 7
7 50.
3. 06
.733
8.567
1.39 3
5.
2 f! 9 9.
171.
3.66
2. 70
93.
6.
8. 7
7 50.
3. OU
.265
3.800
1.060
6.
31 59.
200.
3. 68
3. 1 3
74.
6.-
a. 5
750.
2. 99
.720
4.667
2.000
7.
1303.
250.
3.6 n
3. 91
104.
6.
8. 3
7 5 0.
2. 79
.295
1.200
1. 150
a.
3161 .
250.
3.68
3. 91
107.
6.
8. 6
750.
2.81
.643
8.667
1.497
9.
3510.
302.
4.00
3.00
135.
8.
8. 4
50 (J.
3. 00
.625
7.850
1.435
I n.
34 If!.
302.
4. 00
3. 00
1 35.
8.
3. 3
500.
2. 75
.2ii7
2.300
1.233
11.
4236.
307.
4. 00
3. 00
1 35.
8.
8.4
500.
2. 50
.670
8.600
2.030
I ?.
34 16.
302.
4. 00
3. 00
1 35.
8.
8. 4
500.
2. 75
.737
9.367
1.540
14.
/.12ft.
3 76.
4.00
3. 75
167.
8.
8. 1
700.
2.61
.517
6.662
1.315
I 5.
44 7 7.
379.
4. 00
3. 7 9
168.
b.
8. 0
714.
2.6 3
.<•20
7.729
1.714
16.
3763.
3*1.
3. 75
3. 7 5
149.
8.
8. 1
625.
2.4 b
.597
9.775
2. 150
1 7.
3748.
351.
4.00
3. 50
149.
8.
3. 2
625.
2.48
.617
9. 133
1.577
IK.
4311.
351.
4.00
3. 50
149.
8.
a. 4
625.
2. 50
1.000
11.600
2. 000
19.
4 36 7.
460.
4. 36
3.35
1 98.
b.
a. o
525.
2. 83
-393
5.033
1.280
2(1.
4936.
460.
4. 36
3. 85
1 98.
8.
8. 4
531.
2. 81
.567
8. 900
1.420
-------
EXHIBIT C-9: MEAN OF ENGINE PARAMETERS FOR GENERAL MOTORS
ENGINE
FAMILY
r
WEIGHT
CID
80tt£
I
STROKE
RHP
CYL
1.
2 Hit.
140.
3. 50
1.6 3
84.
4.
2^
Ift 71 .
2 50.
3. fi»
3. 51
105.
6 .
1.
3ft 21.
2 50.
3. 88
3. 53
105,
fa .
4.
4 0 2.
4.
H.
1646.
305.
3. 7 3
3. 4!(
1 40.
tt.
9.
IV1S.
10 5.
3. 73
3.43
140.
it.
lit.
4ot7.
in*,.
4. 05
3. 75
133.
8,
1 1.
icoi.
301.
4. 00
3. 00
1 35.
a.
12.
291 5.
151 .
4. 00
J. 00
8 7.
4.
1 I.
2'tOl .
151 .
4. 00
3.00
85.
4/
I
3 '14 2.
2f>0,
3. 50
1. 3ft
-0.
8.
15.
4 3 17.
J 77.
4. 20
3. 3ft
-0.
a.
1ft.
4211.
371.
4. 1 7
3. 3ft
1 76.
8.
17.
4 5 21.
403.
4. 35
3. 3ft
193,
a.
11«.
33*5.
231.
3. MO
3. 40
105.
6.
It.
114 7.
231.
3. fiO
3. 40
135.
6.
20.
40 a4.
110.
3, ftO
3. 85
140,
tt.
21.
If.?.,
350.
3. 8ft
3. B5
155.
P.
2 2.
43f7.
3 50.
4.05
3. 3"
1 80.
8.
21.
4 I"*.
350.
4.05
3. 3R
1 to.
u •
24.
4 111 2.
425.
4.08
4.06
1 fiO.
8.
21.
¦>107.
4 25.
4. 08
4. Oft
180.
u •
2 ft.
417(>.
425.
4.08
4.0ft
130.
8.
J CCMP
RATIO
tt. u
H. I
tt. 0
tt. 2
a. 4
a. i
8. 2
8. 4
8. 4
7. 7
H.2
ii. 4
a.2
7.4
J. 9
7.9
7.9
8.0
a. t
7.8
8. 0
ii. 0
a.o
a. i
8. 2
8. 2
bull.
092.
sua.
>20.
ftuo.
UtiU.
hoa.
5a«.
5uo.
S'.'4.
<> I u.
1000.
7/(i.
I I (Hi.
1.117.
II DO.
963.
(>04.
696.
S2«.
4i»ii.
(.50.
65'}.
1400.
140U.
1400.
J. J j
2. iiS
2. Hi
J. 04
J. «2
1.
3.
-------
EXHIBIT C-10: SUMMARY OF STEPWISE MULTIPLE LINEAR REGRESSION
ON ENGINE FAMILY CHARACTERISTICS1
MANUFACTURER
DEPENDENT VARIABLE
HC
CO
N0X
In HC
In CO
In N0X
GMC
None
None
Timing RPM
(.194)
None
None
None
Ford
None
None
Axle Ratio
(.383)
None
None
Axle Ratio
(.376)
Chrysler
Bore
. Timing RPM
(.781)
Displacement
Timing RPM
(.601)
None
Displacement
Timing RPM
(.813)
Compression
Ratio
Timing RPM
(.660)
Timing RPM
(.322)
All
Bore
1 (.094)
None
Weight
(.085)
Rated HP
(.171)
None
Weight
(.092)
Table indicates variable(s) selected in each case (if any were significant) and corresponding
r2 value (in parentheses). Independent variables considered were weight, displacement, bore,
stroke, rated horsepower, number of cylinders, compression ratio, timing RMP, axle ratio, bore
divided by stroke, displacement per cylinder, and inertia weight divided by displacement.
-------
119
N°x
2. 5000
2.0000
t .5000
1.0000
.50000
+
0. +
0.
+
+
+
EXHIBIT C-ll: PLOT OF FAMILY MEANS OF N0X
VERSUS MEANS OF CO FOR CHRYSLER
VEHICLES WITH CATALYSTS
*
*
2
« *
»
5.0000 10.000 CO
2.5000 7.5000 12.50Q
-------
120
EXHIBIT C-12: PLOT OF FAMILY MEANS OF N0X
VERSUS MEANS OF CO FOR FORD
VEHICLES WITH CATALYSTS
N0X
2.5000 +
2.0000 +
*
1.5000 + * *
* * a
#
+ * * 3
1.0000 +
.50000 +
+
0. +
0. 5.0000 10.000 CO
2.5000 7.5000 12.500
-------
121
H0X
2.5000 +
EXHIBIT C-13: PLOT OF FAMILY MEANS OF N0X
VERSUS MEANS OF CO FOR GENERAL
MOTORS VEHICLES WITH CATALYSTS
2.0000 +
o
* *
+ * * *
* a * *
* * #
1.5000 + * *
* » #
* *
+ *
*
1.0000 + *
.50000
0.
0. 5.0000 10.000
2.5000 7.5000
-------
122
EXHIBIT C-14: PLOT OF FAMILY MEANS OF NOx
VERSUS MEANS OF CO FOR DOMESTIC
MANUFACTURED VEHICLES WITH CATALYSTS
NOx
2.5000 +
* *
2.0000 + # *
* * *
* * *
+ * « * a *
* * * » * .
* * * 2
1.5000 + * ** *
* * * * * *
* * * «
+ * •* * * *
ft » x
* * *
1.0000 + *
ft
ft
*
.50000 +
+
0. +
+ _ + + +
0. 5.0000 10.000
2.5000 7.5000
-------
123
EXHIBIT C-15: GROUPING OF CURRENT FAMILIES USING
HEURISTIC CLASSIFICATION CRITERIA
MANUFACTURER
SINGLE FAMILIES
AGGREGATED FAMILIES
FA-400-4-NE
FB-360-4-CE
FD-225-1-A and FD-225-1-C
FB-440-4HP-DE
FD-360-2-C and FD-318-2-C
CHRYSLER
FD-225-2-C
FB-400-4-CE and FB-440-4ST-CE
CD-225-1-EP
CD-318-2-GP
CD-360-4-GP
CD-440-4HP-FP
CD-440-4ST-GEP
F1.6G1CV3
F302A1CV5 and 302C2CV4
F1.6H1CV5
F351MD2CV4 and F351WC1CV1
F2.3A1CV5
and F351WC1CV3
F2.3B1CV5
F460A2CV1 and F460B2CV4
F2.8BV1CV1
F2.8B1CV5
F200A1CV5
FORD
F250A1CV1
F250A1CV5
F250B1CV5
F302D1CV5
F302AV1CV1
F351WC2CV4
F351MB2CV1
F351MA2CV1
710Y2V
710J4S
71004 and 740J4U
710W1QU
710Y2 and 720S2E
710W1
730M4U and 730P4UY
710C2
710F1H
710F1SMU
720K4EH
GENERAL MOTORS
720X2E
720X2U
730M4AU
730H2U
740E2LU
740E2
760V4U
760V0
76000U
760V4S
760J0
-------
124
EXHIBIT C-15: GROUPING OF CURRENT FAMILIES USING
HEURISTIC CLASSIFICATION CRITERIA
(Concluded)
MANUFACTURER
SINGLE FAMILIES
AGGREGATED FAMILIES ^
BMW!20.8
BMW
BMW120.9
BMW130.8
128
FIAT
128-CC1
132
132-CC1
77EB
HONDA
77ED-1
77ED-2
A140C
L200F and A140F
A141F
L240F and L280F
DATSUN
L200C
L240C
L280C
—*
4CA
VOLVO
4FA
6CA
6 FA
CNAP
FNA
MAZDA
CTCP
FTCP
REP
-------
125
APPENDIX D
ANALYSIS OF ENGINEERING JUDGMENT
This appendix describes the analysis methods and results of analyses
on the effectiveness of engineering judgment at selecting prototype config-
urations and Identifying the running changes to be tested by EPA. It is
organized Into five sections. The first section (0.1) describes the analysis
comparing certification rejection rates of type B vehicles to type A vehicles.
The second section (D.2) presents the results of the analysis of correlation
among pollutants. The next two sections then discuss the results of comparing
the mean (D.3) and the variance (D.4) of type B vehicle emissions to those
for type A vehicle emissions. The final section (D.5) describes the analysis
of running change vehicle emissions levels.
D*l. Rejection Rates for Type A and Type B Vehicle®
A chi-squared test was performed to determine 1T the probability that
type B vehicles would be rejected during certification was greater than that
for type A vehicles. Exhibit 0-1 displays the number of vehicles rejected
and accepted for each class and also displays the number of vehicles expected
to pass or fall the certification criterion if the probability of failing 1$
the same for both classes.
Exhibit D-t reveals that the number of type A and type B vehicles that
actually failed is very close to what one would expect 1f the probability
of falling the certification criterion was the same for vehicles in both
classes, A chi-squared test can be used to test the hypothesis that the two
probabilities are the same in the case of domestic manufacturers. This test
was used on the data shown in exhibit D-l to determine whether certiffcatlon
-------
126
disposition was independent from the criteria for selecting an emission data
vehicle. The statistic Y used in this test is given by the following equation:
1 = 1 E,
where
0^ 55 the observed number of vehicles in cell i of the matrix
in exhibit D-l, and
E.j = the expected number of vehicles in cell i.
Since the statistic in this test has a chi-squared distribution with one
degree of freedom, the hypothesis will be accepted at the 5 percent signifi-
cance level for all values of the statistic less than 3.84. The value in
exhibit D-l for domestic manufacturers of 0.25 shows acceptance of the hypo-
thesis (i.e., the rejection rate of vehicles selected as high emitters is
equal to that for those selected on the basis of projected sales).1
V. 2 Correlation Among Pollutants
VRI's analysis of engineering judgment compared the emissions of type
B vehicles with those of type A vehicles to determine whether EPA's engi-
neers were successfully identifying vehicles with high emission levels. In
order to determine how to make this comparison, it was first necessary to
determine whether the emissions of different pollutants by a vehicle are
negatively correlated. If no such negative correlation exists, successful use
of engineering judgment should result in the average emissions for one or more
Although it might be informative to look at this statistic on a manufacturer
by manufacturer basis, the number of EDVs for any manufacturer is not
sufficiently large to apply the chi-squared test^
-------
127
pollutants being greater for the B vehicles in an engine family than for the A
vehicles in the same family. If, on the other hand, negative correlation exists,
as for example between NGX and CO emissions, a B vehicle selected for potentially
high N0X levels 1s likely to have low CO levels, and vice versa. If the engi-
neers select roughly equal numbers of vehicles for testing based on each of
these two pollutants, the average over all these vehicles of the emission
levels for each pollutant may not differ much from the average for randomly
selected vehicles.
This phenomenon is illustrated in exhibit D-2, which Indicates a
hypothetical population of vehicles graphed for their CO and N0X emission
levels. As the exhibit indicates, engineering judgment may not Increase the
mean of emissions of those vehicles selected over that of vehicles selected
at random. However, effective engineering judgment should increase the variance
of emissions of selected vehicles, since the selection process should tend
to choose to test those vehicles with extreme {high or low) emission levels
and Ignore vehicles with emissions levels that are about average.
Thus if, on the one hand* an Investigation of the correlation of pollu-
tion rates were to reveal negative correlations, the subsequent analysis of
engineering judgment would attempt to determine whether the variance of
emissions of vehicles selected as high emitters (B vehicles) 1s greater than
the variance of emissions of the overall population of Chicles (represented
by A vehicles), If, on the other hand, no negative correlations were found,
the difference 1n the mean emissions of type A and type B vehicles would be
examined.
The theory of combustion engines indicates that there will be a
negative com! a tlon between and the other two pollutants under certain
-------
128
circumstances. According to the theory, the level of emissions for a given
vehicle will change with changes in the air fuel ratio as pictured in
exhibit D-3. However, this assumes other factors such as spark timing,
engine speed, load level, surface temperature, humidity, exhaust back
pressure, intake manifold pressure, valve overlap, combustion chamber deposit
buildup, stroke to bore ratio, displacement per cylinder, compression ratio,
surface to volume ratio, and combustion chamber design are held constant
[Patterson and Henein, 1972]. Although some of these variables would be the
same for all vehicles in a family, many of them cannot be assumed to be the
same. In discussions with engineers at General Motors Corporation and at
Ford Motor Company it was felt that, since each vehicle is calibrated differ-
ently and since there are other variables affecting emission levels which
are not held constant, the correlation among pollutants for vehicles within
an engine family would be nearly zero.
To Investigate whether negative correlation between pollutants does in
fact exist, a correlation analysis was performed on the emission levels
of emission data vehicles for various stratifications of the population,
including domestic versus foreign vehicles, catalyst versus non-catalyst
vehicles, vehicles stratified by engine family, vehicles stratified by inertia
weight, vehicles stratified by manufacturer, vehicles stratified by engine
displacement, and some combinations of these stratifications. The possible
existence of negative correlation was investigated and linear and quadratic
regression analyses were performed.
-------
129
D.2.1 Vehicles Within An Engine Family
Since each set of B vehicles is selected on an engine family by engine
family basis, the possibility of a negative correlation among pollutants was
examined within engine families. Under the assumption that the correlation
among pollutants within engine families is a constant, but the mean and
standard deviation of emission levels is not, two techniques were used to
investigate the possibility of negative correlation coefficients.1 The first
was a non-parametric test to determine if any of the correlations were signi-
ficantly different from zero. The second technique was to use a Bayesian method
to estimate the probability distribution of the correlation coefficient.
The correlation coefficients were estimated for those families which
had more than two vehicles. Under the assumption that a correlation 1s zero,
the estimated correlation coefficient has a .5 probability of being negative
(see page 174, [Jeffreys, 1961]). Exhibit D-4 displays the number of engine
families for which the estimated correlation coefficient was negative. Under
the assumption that the correlation coefficients are zero, these numbers
have a binomial distribution with p equal to 0.5. These numbers were
tested to determine 1f they were significantly larger than would be expected
under the assumption of a zero correlation coefficient. At the .05 signif-
icance level, the hypothesis of a zero correlation between CO and NO^ was
rejected for the categories of domestic manufactured vehicles with catalysts.
Since the number of negative estimates was significantly large, there Is
reason to believe that at least some families 1n these categoriesJiave a
xNo classicaT methods wefe available to estimate the correlation coefficients
for veh-icles^within families since the number of emission data vehicles within
a family is at most eight.
-------
130
negative correlation between CO and NO -1 The next step was therefore
to estimate the correlation coefficients.
Since no classical methods for estimating correlation coefficients
from multiple, extremely small samples are available, a Bayesian method
for estimating the probability distribution of the correlation coeffi-
cient was used. This problem of estimating on the basis of several in-
adequate data sets, each of which provides only limited information about
the question of interest, is one of the problems to which Bayesian
statistical methods are well-suited (see for example [Box and Tiao, 1973],
[Jeffreys, 1961], [Lindley, 1965], or [Savage, 1954] for discussions of
Bayesian statistical methods). The first three authors mentioned have
described Bayesian methods suitable for dealing with the precise problem
analyzed here. The Bayesian method assumes that the researcher has an
a priori distribution for the parameter in question which reflects his
knowledge as to its likely values. The data is then used with this a priori
distribution to estimate an a posteriori distribution for the parameter.
The methods described by these authors differ only as to the precise form
of the researcher's a priori distribution for the correlation coefficient.
Following Jeffreys, the a priori distribution of the correlation
coefficient was assumed to be uniform on [-1, 1] which reflects the belief
that any value 1n this interval is equally likely. ([Box and T1ao, 1973] and
[Lindley, 1965] propose prior distributions that are heavily concentrated on the
values +1 and -1, which seems totally inappropriate for this problem. No other"
*It may be of interest to note that the data revealed a significantly positive
correlation between HC and the other two pollutants for these categories.
-------
131
forms for a non-informative prior distribution have been identified in the
literature.) Given an initial state of ignorance so described, the
posterior distribution following the result of a single experiment is
given in equation 24 on page 177 of [Jeffreys, 1961]. The final posterior
belief function, based on a sequence of such experimental observations,
is a product of such functions.
Using these techniques a Bayesian 95% confidence interval was con-
structed for each of the correlation coefficients on the assumption that
the intra-family correlations are constant, and that the emission levels
for designs within a family have, or are closely approximated by, a
multivariate normal distribution. The resulting confidence intervals are
displayed in exhibit D-5. A good constant estimate would be the median of the
posterior distribution presented 1n exhibit D-5, which 1s most negative for
the correlation between CO and NO (-.22) for foreign vehicles with catalyst.
The confidence intervals for this category are, however, much wider than for
the categories which Include domestic manufactured vehicles. The width of
this interval 1s attributable to the smaller data sample for foreign manu-"
factured vehicles.
The results 1n exhibit D-5 Indicate there 1s a negative correlation
coefficient between CO and N0X and that a good central estimate of Its magni-
tude would be -.2, or an R2 of .04. Thus only about 4 percent of the 1ntra-
famlly variance In emissions of NO or CO for different designs can be explained
A
by the association between the two pollutants. The results 1n exhibit D-5 also
Indicate a positive correlation between HC and CO of about .5 and a weak posi-
tive correlation between HC and N0X of about .1.
-------
132
D.2.2 Vehicles Stratified in Other Mays
The conclusion from the above analyses is that negative correlation exists
between CO and N0X, but that the magnitude of this correlation is small. There'
fore, the linear correlation among emission levels was also computed for other
vehicle groupings to see if any negative correlation might be found in these
stratifications. The first of these analyses investigated the correlation for
all catalyst, non-catalyst, domestic, and foreign vehicles. The results, dis-
played in exhibit D-6, were that no negative correlation coefficients existed
for this stratification.
It was felt that all the data should be stratified by inertia weight and
displacement since these appear to be important factors in determining emission
levels. The results of the analysis of the correlation among pollutants for
vehicles in similar ertia weight classes are shown in exhibits D-7 through
0-10. Under this stratification, a few negative values do occur for the
estimated correlation coefficients (particularly between CO and N0X). However*
at the .01 significance level none of these negative values is significantly
different from zero. The correlation among pollutants is shown in exhibit D-H
for the data stratified by displacement groups with a sample size of 20 or more-
The exhibit reveals two displacements (351 and 460) for which the estimated
correlation is significantly less than zero.1 : The correlation was recalculated
by grouping data from displacements near these two values. When the data are
regrouped in this manner, the significance of the negative correlation between
NO and CO disappears (displayed in exhibit 0-12).
A
Scatter plots of NO versus CO for these two cases revealed that the negative
correlation may be Striven 2 of 3 data points.
-------
133
D.3 Emission Level Magnitudes for Type A and Type B Vehicles
The above results which indicate the correlation between vehicles is
generally non-negative; however, some negative correlation, albeit minor
appears to exist between NO and CO. Therefore two types of analyses were
A
performed, one assuming a positive correlation and the other assuming a
negative correlation. In this section the emission levels of type A and
type B vehicles are compared 1n terms of the difference between the mean
emission levels of each type of vehicle (i.e., assuming a positive correlation).
To determine whether the mean of one class is significantly different from
that of the other three analyses were performed—an analysis of variance,
a Mann-Wh1tney test and a test of (-statistics. The following three sub-
sections discuss the results of each of these analyses-
D.3.1 Analysis of Variance
An analysis of variance was performed on a model explaining the effects
of engine family, type of selection, vehicle configuration, and test-to-test
differences 1n the measured emission levels of a particular vehicle. Since
these factors are hierarchical or nested 1h nature (I.e., each engine famllv
contains both type A and type B vehicles which 1n turn may have different
configurations each of which Is tested a number of times), the following
analysis of variance model was developed
Let Yijia denote the ured session level of a given pollutant for
the Ith test on the vehicle of type j in family 1. Then the model for
-------
134
the analysis of variance is given in the following equation:
EijkJt = y + fi + lij + Vijk + eijk* •
where u = the overall mean of the observations,
f. = the effect of family i,
t.. = the effect of type j and family i,
¦ J
¦ the effect of vehicle k of type j and family i, and
xL
e..,, = the error in the 1 test on vehicle k in class j and
'i jk£
family i.
It is further assumed that {f-j}, {tjj}, and {ejjj^} are independently
normal with zero means and constant variances- The data was analyzed as a
random effects completely nested design with unequal numbers of observations'
The formulas for the analysis of variance are presented on pages 248-258 in
[Scheffe, 1963]. Due to the fact that there are unequal numbers of observation
the F tests for testing whether the mean emission level for type A vehicles
equal to those of type B vehicles and for testing whether the mean emission
levels are the same for all families are approximate F tests. The approxima-
tion is discussed on pages 254-255 in [Scheffe, 1963] and in more detail on
pages 302-303 in [Ostle, 1963].
Since the results of the analysis of the dependencies between test
variability and mean emission level in appendix B were inconclusive and
since there is some evidence and rationale that the estimates of test-to-test
variance using this study's data base may be biased upwards, ANOVAs were
performed using three different test variability assumptions. The first
assumption was that test-to-test standard deviation is a constant (model 1
in appendix B). The second assumption was that test-to-test standard
deviation is proportional to the mean (model 4 in appendix B). The third
-------
135
ANOVA was performed using estimates of test-to-test standard deviation
as presented in [Juneja et ai.t 1977].
Analysis results using the first of these assumptions are presented in
exhibit D-13. The Exhibit presents the computed F-ratio values for each factor
in the above model under the assumption that the test-to-test standard deviation
is a constant and independent of the mean emission level for a vehicle. The
number in parentheses beneath each F-ratio value indicates the critical value
for accepting the appropriate hypothesis stated above at the 5 percent signifi-
cance level. As can be seen from the results in this exhibit, the hypothesis
that mean emission levels of type A vehicles differ from those of type B
vehicles is rejected for all pollutants in all vehicle categories. On the
other hand, the existence of differences in the mean emission levels among
families and among configurations within a family are accepted in nearly
every case, with the only notable exception occurring for configuration
differences in foreign vehicles with catalysts.
The second series of ANOVAs were performed under the assumption that
the test-to-test standard deviation was proportional to the mean. Since the
analysis of variance model assumes that the test-to-test standard deviation
is a constant, test result data were normalized to remove the assumed
dependence between mean emission level and test variability. Under the
proportional assumption, the logarithm of the data has a constant test-to
test standard deviation.1 To check this transformation a review was made of
scatter plots of the test-to-test standard deviation versus mean, after
the logarithm of the data was taken, and these did not reveal any apparent
trends or relationships which might refute the porportionality assumption.
*See page 365, [Scheffe, 1963].
-------
136
Exhibit D-14 provides the results of the ANOVA on the logarithm of the
data. These results are nearly identical to those in exhibit D-13. Mi>nor
differences between the specific ANOVA results under these two test variability
assumptions do occur; however, both analyses point to the same conclusions,
i.e., the mean emission level of vehicles in different families are different,
while differences between the mean emissions of type A and B vehicles in these
families are not apparent.
Finally, there exists some question concerning the validity (as discussed
in appendix B) of any estimate of test variability which is based on data
from multiple tests on emission data vehicles. That is, estimates from this
data may either contain too much variation due to the few tests available
per vehicle or an upward bias due to changes from such recalibrations which
alter the conditions between tests. For this reason the analysis was repeated
using estimates of test-to-test standard deviation as calculated in [Juneja
et al.* 1977] assuming test-to-test standard deviation is a constant (model 1
in appendix B).1 The results of this analysis are presented in exhibit 0-15.
These results are nearly identical to those in the previous analyses and
hence support the same conclusions.2
D.3.2 Mann-Whitney Test
A non-parametric method for comparing the rank of the emission levels
for type A vehicles to type B vehicles was devised to test the hypothesis that
these levels are not significantly different within an engine family.3
xThis estimate was chosen because it represented the most recent available and
because it would produce conclusions which were most conservative (i.e., the
lowest test-to-test standard deviation estimate).
2Since the results did not differ from those in exhibit D-13, the analysis was
not repeated with the data in [Ouneja et al., 1977] assuming test-to-test
standard deviation is proportional to the mean (model 4).
3In this analysis only the first test made on each vehicle was used.
-------
137
This analysis was performed to determine whether a significant difference
between types would be detected without any parametric assumptions on the
distribution of emission test results (e.g., the normal distribution assumed
in the ANOVA). The non-parametric analysis consisted of calculating a Mann-
Whitney statistic on the mean emission level of vehicles in class A versus
class B within a family.
The Mann-Whitney statistic for family i, denoted by U.., is calculated
by the following equation:
NA NB
u, • E E 0jk .
j-1 k=l
where
NA = number of class A vehicles in family i,
NB = number of class B vehicles in family i, and
il if the mean emission level for vehicle k in class B is greater
than or equal to the mean emission level for vehicle j in class
A, and
0 otherwise.
The expected value and variance of are given in the following
equations.
E(Uj) - NA • NB/2, and
Var(Ui) - NA • NB • (NA+NB+L)/l2.
Let
U.-E(U,)
vi ¦ 7=
^VarO^)
If both NA and NBjire at least 10, has a distribution that is approximately
normal with a mean of 0 and a variance of 1. Since the values of NA and NB
-------
138
for a given family are not large enough to allow for a good approximation,
the V.j were summed over all engine families. Using the central limit theorem,
the distribution of the sum of V.. is approximately normal when the number of
engine families is at least 20. The standardized sum of the V.., denoted by
V, is presented in the following equation:
I
where
I = the number of engine families.
The statistic V has a distribution that is approximately normal with a mean
of 0 and a variance of 1 when the number of engine families is at least 20.
Exhibit D-16 displays the normalized sum of the statistic for domestic
vehicles by manufacturer and foreign vehicles.1 To reject the hypothesis
that the emission levels of type A vehicles are equal to type B emissions
and hence accept the alternative that type B emissions are greater than
those for type A, the numbers in the table must be positive and exceed 1.65
for a 5 percent level of significance. Since there are no such values, one
cannot conclude the emission levels for type B vehicles are "greater" than
the class A vehicles.
D.3.3 Test of t-Statistics
In addition to the Mann-Whitney test a special purpose statistical method
was created to permit comparison of type A and type B vehicle emission levels
without requiring any specific assumption concerning the pattern of association
Catalyst and non-catalyst distinctions were also examined with the same results'
however, the number of non-catalyst vehicles was insufficient to consider
the resultant distribution of the statistic approximately normal.
-------
139
between mean levels within an engine family and variability within that family.
In this test a t-statistic was calculated for each family1 by subtracting the
mean emission levels of type B vehicles from those for type A vehicles, and
the difference was normalized to account for the pooled variance of emission
levels in each family. The t-statistic calculated for a family is presented
in the following equation:
* - B
t *
(i ~ i)
\na ml
NA , NB 9
£ (Ai - A)2 + £ (B1 - IT)
NA + NB -2
where
NA » number of observations in class A,
NB » number of observations 1n class B»
NA+NB-2 = the degrees of freedom for the t-stat1sties,
A| ¦ observation in class A,
ft ¦ average for class A,
B.j » observation in class B, and
F ¦ average for class B-
These calculations produced an array of values (one for each engine family),
each of which has a t distribution with the appropriate, family-specific,
degrees of freedom. Next, the corresponding cumulative probability was determined
for each of these values using the appropriate t distribution. The resulting
collection of probabilities was then arranged in ascending order and a sample
cumulative distribution constructed.
JAs in the Mann-Whitney test, the data set used in this analysis included only
the first test made on each vehicle.
-------
140
Under the assumption that the mean emission level of type A vehicles
equals that of type B vehicles, each engine family statistic calculated above
is a t-distributed random variable. Thus the cumulative probability
corresponding to the statistic is distributed uniformly on (0, 1). Therefore,
to test the hypothesis that type A and B emission levels are equal against
the alternative hypothesis that they are different, the collection of
cumulative probabilities is tested to determine if it corresponds to a
sample from a random variable uniformly distributed in (0, I).1
The sample cumulative distribution of these probabilities was plotted
for vehicles with catalyst, vehicles without catalyst, and a combination of
both classes of vehicles. In every case the sample distribution looked
approximately uniform on (0, 1). Exhibit D-17 displays the Kolmogorov-
Smirnoff test statistic for the test of the goodness of fit of a uniform
distribution to the sample cumulative for these cases. The critical values
for rejecting the hypothesis at the .05 significance level are also presented.
As can be seen, the hypothesis that the mean emission level for type A vehicles
is equal to that for type B vehicles cannot be rejected at the .05 level.
D,4 Emission Level Variability for Type A and Type B Vehicles
As noted in section D.2 the selection of type B vehicles which have,
in general, greater emission levels than the type A vehicles may result in
an observed difference in the variance rather than the magnitude of certain
pollutants.; That is, if the correlation between pollutant pairs is negative,
^he cumulative probabilities for any continuous, strictly increasing distri-
bution are uniformly distributed on (0, 1) (see page 901, [Wagner, 1969]).
Although the probabilities in this case are derived from t distributions
with differing degrees of freedom, they are still uniformly distributed on
(0, 1) under the null hypothesis.
-------
141
then the type B vehicle emissions would be expected to demonstrate a greater
variance than that for the type A vehicles, under the assumption that the
high emitters can be identified.
In order to determine whether the variance of emission levels for type B
vehicles is in fact greater than the variance for type A vehicles, a 95 percent
confidence interval was constructed for the ratio of these variances for
those 1977 engine families with at least two vehicles of each type. If the
variance for type B vehicles equals that for type A vehicles (i.e., indicating
the emission levels are similar), then the calculated value for the above
ratio should fall within this confidence interval 95 percent of the time.
To examine whether this was true, the 95 percent confidence interval for
the ratio of the standard deviation of type B vehicles to type A vehicles
was obtained as follows. Let g denote the ratio of the variance for type
vehicles to the variance of type A vehicles. Then the estimate of g, denoted
by g, divided by a g has an F distribution with nB-1 and nA-1 degrees of freedom,
where nB denotes the number of type B vehicles in the engine family and
nA denotes the number of type A vehicles. Let F^ and Fu denote the end
points of a 95 percent confidence interval for the FnB-1, nA-1 distribution.
This confidence interval can then be obtained using the following equations:
.95 » P
.95 = P
<
9
<
or
-------
H2
Exhibit D-18 displays the confidence intervals using the above equations
for all three pollutants. If the ratio of the variances is a constant equal
to 1.0 (i.e., type B variance equals type A variance), one would expect the
number 1.0 to be outside 3 of the 66 intervals (i.e., 95 percent of the time).
The number of times 1.0 is outside these intervals is 4.1 The confidence
intervals for ^1/g, the standard deviation of type A vehicles divided
by the standard deviation for type B vehicles, could be obtained by dividing
the values in exhibit 0-18 into the number 1. When this was done the confident
interval for yfug Included higher values than those for in 47 out of the
66 intervals constructed. This indicated that, if the variances for type A
and type B vehicles do differ, then there is a tendency for type A vehicles
to have the greater variance (i.e., type A vehicles generally have higher
emissions than type B vehicles under the assumption of a negative correlation
between po11utants).
There is, therefore, no evidence that the negative correlation between
CO and N0X has caused the variance of emission levels for type B vehicles
to exceed the variance for type A vehicles. Thus, the results of this and
the preceding analyses indicate that no matter what is assumed about the
correlation among pollutants the emission levels of type B vehicles are not
significantly different from those of the type A vehicles.
JIn 3 out of the 4 cases the interval included values which were all less
than 1.0, indicating that where the B variances were not equa? to the A
variances, the A variances were actually greater.
-------
143
D.5 Running Change Vehicle Emission Levels
Three types of analyses were conducted to investigate whether the run-
ning change vehicles selected for testing by EPA had emission levels which
were higher than those measured on the vehicle prior to implementation of
the change. The first of these analyses examined the degree to which the
mean emission level of sequential running changes was similar to the
certified emission levels of the original emission data vehicle (EDV). The
second analysis examined the number of times the sequential running change
emission levels exceeds the levels of the original EDV. A estimate of the
variance of running changes was then obtained in the final analysis to de-
termine the likelihood a tested running change vehicle would fall given the
EDV passed the certification test. Each of these analyses are discussed In
greater detail below after a brief description of the running change data base.
D.5.1 Running Change Data Base
The data base used 1n the analysis of running changes was constructed
by identifying 66 emission data vehicles (EDVs) in the VRI data base which
also had EPA test data on running changes (RCs) to these configurations.
The number of RCs per EDV was subject to interpretation because the
EPA data base consistently did not differentiate among the following:
(1) a test on one running change and a test on a subsequent change,
(2) two tests on the same running change (the second one, possibly
made for the fuel economy testing program),
(3) back-to-back tests to examine emissions before and after
implementation of the running change.
The EDV could, however, be easily Identified, so most of the analyses
discussed below cbmpared the emission levels of all running changes to their
-------
144
original certified EDV. The RC data base used in this analysis contained
only EPA test results. It was comprised of 66 EDVs each with one test, and
77 computer differentiated RCs tested a total of 109 times. Fifty of these
running changes had one EPA test, 24 had 2 tests, one had 3 tests, and 2 had 4
tests. Exhibit D-19 provides the identification of the vehicles and test
results examined in the following analyses.
D.5.2 Mean Emission Levels
One criterion used by EPA in deciding to test proposed running changes
is the degree to which the change is expected to increase emissions. There-
fore it seemed reasonable to expect that EPA running change tests would have
higher emission levels than their corresponding emission data vehicles. This
occurence could be observed from the data in exhibit D-19 in one of two forms—
as an increase of the mean emission levels for one or more of the pollutants
or as an unexpectedly high fraction of RC tests which showed an increase in
at least one pollutant, even though no single pollutant showed a consistent
increase in the various tests.
An analysis of variance was conducted to test the hypothesis that the
mean RC emission levels were above the emissions of their counterpart EPV.
The results of this test concluded that, at a five percent level of signi-
ficance, running changes do not have higher mean emissions in any pollutant
than their counterpart original EPVs. A x2 test was then performed to compare
the frequency that RC emissions exceeded those of the original EDV with the
frequency that would be expected where there are no differences in emission
levels. Except for the correlation of CO and HC emissions, the distribution
-------
145
of running change test results was statistically indistinguishable from
that expected if running changes had no effect on the emission levels of the
three pollutants. That is, the effects on each pollutant were symmetrically
distributed as to increases and decreases and changes which showed an increase
on at least one pollutant were not more frequent than would be expected from
random effects.
D.5.3 Emission Level Variability
Given the above result, It became possible to measure the degree of
variability between the emissions of the original vehicles and those of their
running change counterparts 1n terms of a variance, rather than a shift
in the mean emissions. Because of the fact that test-to-test errors would
be expected to cause some variance even if the running change designs were
identical to their original counterparts, 1t was necessary to perform an
analysis of variance on the running change data.
This analysis of variance was conducted 1n two steps, because of the
previously mentioned problems in telling whether a test was on a vehicle
that had in fact had the running change applied, or an original design vehicle
with the test run for back-to-back purposes. These two steps were:
(1) first, the withln-group coefficient of variation of each emission
level was computed for groups of EDVs and all their Identical RC
counterparts; and then
(2) the test-to-test variance was eliminated and the change-to-change
coefficient of variation was estimated.
-------
146
The coefficient of variation,1 including both the effects of test-to-
test variation and change-to-change variations, was 18 percent for HC, 34 per-
cent for CO, and 12 percent for NO . Eliminating the test-to-test variation2
A
left 8 percent for HC, 22 percent for CO, and 5 percent for NO . In order
/\
to determine the RC-to-RC coefficient of variation from this data, it was
necessary to estimate the numbers of tests per RC configuration tested. It
was clear that this number could not be less than 1.0. Further an analysis
of the data base showed that the number of tests per computer differentiated
RCs was 2.07, so that the number of tests per configuration had to lie in the
interval 1.0-2.07. On this basis, the change-to-change coefficient of
variation has a range of 1.0 to /2.07 times the above estimate, i.e., 8 percent
to 12 percent for HC, 22 percent to 32 percent for CO, and 5 percent to 7
percent for N0x<
The precise estimate of the EDV-to-EDV coefficient of variation within
a family is dependent on the statistical model used, but the results from
the ANOVA described in section D.3.1, indicate that all estimates are
approximately 20 percent for HC, 30 percent for CO and 1,5 percent for N0X.
Consequently the variability of RCs is significantly less than the variabilty
among the EDV configurations within a family.
The results from this and the previous running change analyses are:
(1) the average of RC emissions cannot be dlstinquished from the EDV
emissions levels,
Estimated using the logarithmic data transformation discussed in section
D.3.1. (See page 365 of [Scheffe, 1963].)
2Test-to-test coefficient of variation estimates were taken from the results
of this study for catalyst vehicles (see exhibit B-4 in appendix B).
-------
147
(2) the RC emissions are as likely to be above as below the EDV
emission levels, and
(3) the variability of RC-to-RC emissions is less than that for EDV-to-
EDV emissions.
Thus, it is reasonable to assume that given the emission levels of an
EDV, the emissions of running changes to the EDV will be approximately
the same levels. Therefore the decision to test or not test a proposed
running change should, for the most part, be based on the emission levels
of the original certified EDV. If the change is proposed to a configuration
not originally certified, then the average EDV emissions for the engine family
should serve as a good proxy.
-------
148
EXHIBIT D-l: ACTUAL AND EXPECTED CERTIFICATION DISPOSITION
OF 1977 MODEL YEAR VEHICLES IN VRI DATA BASE
BY TEST SELECTION CRITERIA
Domestic
Foreign
Type A
Type B
Type A
Type B
Number Passing
72
159
40
42
Number Failing
5
16
0
1
Expected Number to Pass
71
160
40
42
Expected Number to Fail
6
15
0
1
Chi-square Statistic
0.25
*** 1
^hen the expected number of observations in any one cell is less than 5, the
distribution of the test statistic cannot be approximated with a chi-square
(see page 127, [Ostle, 1963]). An exact test of this hypothesis as described
on page 132, [Ostle, 1963] had a significance level of .48.
-------
149
EXHIBIT D-2: EFFECT OF ENGINEERING JUDGMENT
IN DISTRIBUTION OF SELECTED
VEHICLES IF EMISSIONS ARE
NEGATIVELY CORRELATED
NO emissions
Engineering judgment
selects these vehicles
with high NOv levels
land correspondingly
Each point represents
emissions levels of
a single vehicle.
Engineering judgment
selects these vehicles
with high CO levels
(and correspondingly
. \ low NO levels)
(^marginal density
function1of NOx
level for entire
population
marginal
.density function1
of NOx level for
selected vehicles
only
CO emissions
marginal density
function1 of CO
level for entire
population
marginal
density
function1 of CO
level for
selected
vehicles only
^he marginal density function represents the relative frequency of
occurrence of vehicles with the given emissions level. In the cases
Illustrated, the density for the entire population and the density for
selected" vehicles have the same mean, but the density for the selected
vehicles has a larger variance.
-------
150
EXHIBIT D-3: EFFECT OF AIR FUEL RATIO ON EMISSIONS1
H : 1 1 h
11 73 15 17
1Taken from [US Environmental Protection Agency, 1975], p. 7.
-------
151
EXHIBIT D-4: NUMBER OF ENGINE FAMILIES
WITH NEGATIVE ESTIMATES OF
THE CORRELATION COEFFICIENT
CORRELATION
TOTAL
NO. OF
HC and CO
HC and NOy
A
CO and NOy
A
ENGINE
FAMILIES
Domestic Manufactured
Vehicles With Catalysts
9
18
33*
48
Foreign Manufactured
Vehicles With Catalysts
2
4
3
5
Foreign Manufactured
Vehicles Without Catalysts
1
4
5
7
All Vehicles
12
26
41*
60
~Significant at .05 level.
-------
EXHIBIT D—5: 95 PERCENT CONFIDENCE INTERVALS
FOR THE CORRELATION COEFFICIENTS
Domestic Manufactured
Vehicles With Catalysts
Number
of
Engine
Families
Correlation Between
HC and CO
HC and NO
A
CO and NO
Lower
Point
Median
Upper
Point
Lower
Point
Median
Upper
Point
Lower
Poi nt
Median
Upper
Point
48
.39
.53
.71
-.04
.13
.30
-.35
-.19
-.03
Foreign Manufactured
Vehicles With Catalysts
5
-.35
.17
.61
-.56
-.11
.37
-.63
-.22
.26
Foreign Manufactured
Vehicles Without Catalysts
7
.22
.62
.85
-.56
-.12
.34
-.58
.00
.44
All Vehicles
60
.39
.52
.64
-.07
.09
.24
-.32
-.18
-.03
-------
153
EXHIBIT D-6: CORRELATION AMONG POLLUTANTS
Catalyst
Non-Catalyst
Both
Domestic
Foreign
Both
Foreign
Both
HC and CO
.76
.69
.77
.34
.65
HC and NOY
.27
.42
.36
00
«
.29
CO and NOx
.11
.32
.20
.25
.18
EXHIBIT D-7: CORRELATION AMONG POLLUTANTS
BY INERTIA WEIGHT (DOMESTIC
VEHICLES WITH CATALYSTS)
INERTIA.
WEIGHT
CORRELATION BETWEEN
.01 Critical
Value
HC and CO
HC and N0X
CO and N0X
2250
.95
.14
.33
.83
2750
.35
.80
-.10
.92
3000
.45
.13
-.16
.54
3500
.79
.59
.21
.38
4000
.82
.42
.32
.26
4500
.82
.31
.30
.33
5000
.81
-.06
-.16
.35
5500
.84
-.17
-.37
.51
-------
154
EXHIBIT D-8: CORRELATION AMONG POLLUTANTS
BY INERTIA WEIGHT (FOREIGN
VEHICLES WITH CATALYSTS)
INERTIA
WEIGHT
CORRELATION.BETWEEN
.01 Critical
Value
HC and CO
HC and NOx
CO and NO
2250
.66
.13
.21
.68
2500
,53
.09
o
00
•>
.99
2750
,97
-,26
-.06
.99
3000
.61
.43
.56
.62
3500
.71
,61
.36
.68
EXHIBIT D-9: CORRELATION AMONG POLLUTANTS
BY INERTIA WEIGHT ( FOREIGN
VEHICLES WITHOUT CATALYSTS)
INERTIA
WEIGHT
CORRELATION BETWEEN
..01 Critical
Value
HC and CO
HC and N0X
CO and N0X
2000
.17
.48
.37
.96
2250
.71
.33
,09
.62
2500
.60
.67
.39
.64
3000
-.02
CO
in
*
.62
.80
3500
m
CO
•
\o
CM
•
.52
.99
-------
155
EXHIBIT D-10: CORRELATION AMONG POLLUTANTS BY
INERTIA WEIGHT (DOMESTIC AND
FOREIGN VEHICLES WITH CATALYSTS)
INERTIA
WEIGHT
CORRELATION BETWEEN
.01 Critical
Value
HC and CO
HC and N0X
CO and N0V
A
2250
.78
.22
.40
.55
2500
.53
.09
.80
.99
2750
.59
.30
-.21
.76
3000
.58
.39
.21
.41
3500
.79
.52
.28
.34
4000
.82
.42
.32
.26
4500
.82
.31
.30
.33
5000
.81
-.06
-.16
.35
5500
.84
-.17
IN
CO
•
1
.51
-------
156
EXHIBIT 0-11: CORRELATION AMONG POLLUTANTS
BY DISPLACEMENT (DOMESTIC VEHICLES)
DISPLACEMENT
SAMPLE
SIZE
COP
RELATION BE1
¦WEEN
.-01 CRITICAL
VALUE
HC&CO
HC&NO
CO&NO
140
42
.21
-.04
-.01
.39
225
61
.81
.44
.28
.33
231
36
.67
.22
.37
.42
250
54
.80
.04
-.11
.35
260
22
.78
.00
.00
-54
302
27
.49
.00
-.28
.49
318
41
.40
.74
.42
.40
350
87
.60
.27
.06
.27
351
40
.45
.04
-.40
.40
360
40
.56
.20
.15
.40
400
77
.61
.41
.28
.29
425
33
.86
.11
-.23
.44
440
44
.40
.08
.02
.38
455
29
.78
.24
.15
.47
460
46
.41
.12
-.40
.38
-------
157
EXHIBIT D-12: CORRELATION AMONG POLLUTANTS BY
DISPLACEMENT GROUP (DOMESTIC VEHICLES)
Correlation Between
Displacement
Group
Sample
Size
(HC, CO)
(HC, N0X)
(CO, N0X)
,01
Significance
Level
440, 4541, 455, 460
121
.50
.07
-.17
23
350, 351
127
.60
.17
-.13
.23
^his displacement does not appear 1n exhibit D-ll because the
sample size is less than 20.
-------
EXHIBIT D-13: ANALYSIS OF VARIANCE RESULTS INCLUDING TYPE A VS B DIFFERENCES
UNDER CONSTANT TEST VARIABILITY ASSUMPTION (MODEL 1)
F-Ratio Test? For
Catalyst
Hon-Catalyst
Both
Donestic
Foreign
Both
Foreign
Goth
HC
CO
NO*
HC
CO
N0X
HC
CO
H0X
HC
CO
K0X
HC
CO
N0X
Family differences
8.79
(1.9)
4.44
(1.9)
14.5
(4.?)
9.59
(3.8)
28.1
(5.2)
13.5
<3-9)
9.82
(1.8)
S.09
(1.7)
17.7
(2.6)
9.79
(3.1)
6.18
(3.0)
1.46
(2.7)
12.2
(1.6)
5.50
(1.6)
13.4
(2.2)
Type A vs. Type B
Differences within
Family
0.94
(1-4}
1.04
(1.4)
0.53
(1.4)
1.82
£3.4}
0.35
(2.6)
1.47
(3.2)
0.90
(1.4)
0.95
(1.4}
0.53
(1.4)
0.61
(2.4)
0.76
(2.5)
] .94
(2-4)
.91
(1.4)
.91
(1.4)
.56
(1.4)
Configuration
Differences within
j Type and Family
3.56
11.4}
4.00
(1-4)
1.71
(1.4)
0.29
(2.9)
1.82
(2.9)
0.34
(2.9)
2.86
(1.4)
4.00
(1.4)
1.57
(1.4)
5.58
(3-2)
3.66
(3.2)
5.53
(3.2)
3.05
(1.4)
3.96
(1.4)
1.64
(1.4)
'F ratio results are significant (at the 5* level) vfhen yreater than the value in parentheses.
-------
EXHIBIT D-14: ANALYSIS OF VARIANCE RESULTS INCLUDING TYPE A VS B DIFFERENCES UNDER
THE ASSUMPTION THAT TEST VARIABILITY IS PROPORTIONAL TO THE MEAN (MODEL 4)
F-Ratlo TestlFor
Catalyst
Non-Catalyst
Both
Domestic
Foreign
Both
Foreign
Both
HC
CO
N0X
HC
CO
M0X
HC
CO
N0X
HC
CO
N0X
HC
CO
N0X
Family Differences
11,9
0.9)
5.70
(1.8)
13.3
(3-0)
8.41
(3.6)
16.8
(4.2)
8.21
(3.2)
13.0
(1.7)
6.59
(1.7)
11.6
(18)
11.8
(3.2)
6.20
(2.9)
1.52
(2.7)
14.4
(1.6)
7,06
(1.6)
10.6
(1.6
Type A vs. Type B
Differences within
Faaitly
1.04
0-4)
1.18
04)
0.58
(1.4)
1.92
(3.3)
0.38
(2.5)
1.90
(2.9)
1.06
0-4)
1.07
(14)
0.98
(1.4)
0.60
(2.4)
1.24
(24)
1.99
(2-4)
1.01
(14)
1.03
(1.4)
0.9/
(1.4]
Configuration
Differences within
Type and Faaily
3.17
04)
4.59
0.4)
2.91
(14)
0.31
(2.9)
3.62
(2.9)
0.46
(2.9)
1.93
(1-4)
4.55
(14)
1.26
(1.4)
6.55
(3.2)
5.13
(3.2)
5.26
(3.2)
2.26
(14)
4.54
(14)
1.3;
(1.4]
*F ratio results are significant (at the SX level) when greater than the value In parentheses.
-------
EXHIBIT D-15: ANALYSIS OF VARIANCE RESULTS INCLUDING TYPE A VS B DIFFERENCES
FROM USING JUNEJA ET AL. [1977] TEST VARIABILITY ESTIMATES UNDER
MODEL ASSUMPTION
F-Ratio Test1 For
Catalyst
Non-Catalyst
Both
Domestic
Foreign
Both
Foreign
Both
HC
CO
N0X
HC
CO
N0X
HC
CO
N0X
HC
CO
N0X
HC
CO
N0X
Family Differences
8.72
(1.9)
4.41
(1.9
13.5
(3.7)
8.13
(3.0)
27.2
(4.8}
11.4
(3.0)
9.68
(1.8)
5.05
(1.7)
16.3
(2.6)
9.87
(3-2)
6.22
(3.0)
1.46
(2.7)
12,1
(1.6)
5,46
(1.6)
12.7
(2.1)
Type A vs. Type B
Differences within
Family
0.93
(1.4)
1.03
(1.4)
0.52
(1.4)
1.42
(2.6)
D.35
(2.6)
1.19
(2.6)
0.89
(1.4)
0.95
(1,4)
0.51
(1,4)
0.60
(2.4)
0.75
(2.4)
1.99
(2.5)
0,90
(1.4)
0.91
(1.4)
0.54
(1.4)
Configuration
Differences within
Type and Family
13.5
(1-3)
14.9
(13)
14.7
(1.3)
2.30
(1.8)
2.32
(1.8)
2.98
(1.8)
12.4
(1.3)
13.6
(1.3)
13.5
(1.3)
43.8
(1.7)
7.97
(1.7)
3.05
(1.7)
15.3
(1.3)
12.9
(1.3)
12.7
(1.3)
^-ratio results are significant (at the 5% level) when greater than the value in parentheses
-------
161
EXHIBIT D-16: NORMALIZED SUM OF MANN-WHITNEY TEST STATISTIC
TO COMPARE TYPE A TO TYPE B VEHICLE EMISSIONS
Domesti
c
Foreign
All
Chrysler1
Ford1
GM
All
HC
.97
-2.60
1.43
i
«
o
¦vi
-.57
-.26
CO
-.13
-.95
.57
-.19
.39
.05
NOx
.53
-.56
-.11
-.10
-T.53
CM
o>
~
1
EXHIBIT D-17: KOLMOGOROV-SMIRNOFF TEST OF THE SAMPLE CUMULATIVE DISTRIBUTION
HC
CO
NO*
.05
Critical
Value
Catalyst
.09
.10
.15
.19
Non-Catalyst
.27
.23
.29
.40
Both
.09
.11
.13
.17
JThe numbers of samples in these categories fall slightly below the necessary
minimum to .consider the distribution of the statistic approximately normal.
-------
162
EXHIBIT D-18: 95 PERCENT CONFIDENCE INTERVAL ON
THE STANDARD DEVIATION OF TYPE B
VEHICLES DIVIDED BY THE STANDARD
DEVIATION OF TYPE A VEHICLES
Domestic Manufactured Vehicles With Catalysts
Engine
Number
Number
HC
CO
N0X
Tvno "A"
Tune "R"
type t\
iype ~
Family
Vehicles
Vehicles
Lower
Upper
Lower
Upper
Lower
Upper
Point
Point
Point
Point
Point
Point
F7- 2
2
3
0.04
7.13
0.02
3.58
0.03
6.07
F7-14
3
5
0.04
0.77
0.08
1.57
0.09
1.73
F7-15
2
5
0.22
22.53
0.01
1.44
0.74
77.65
F7-16
2
2
0.03
16.66
0.11
74.58
0.08
54.75
F7-20
2
2
0.01
9.79
0.00
1.30
0.07
47.03
G7- 1
2
4
0.01
1.25
0.02
2.69
0.06
6.80
G7- 6
2
2
0.05
31.35
0.04
28.17
0.03
18.34
G7- 7
2
2
0.02
9.93
0.05
34.43
0.01
7.38
G7-10
3
4
0.21
5.15
0.09
2.26
0.08
2.02
G7-13
2
3
0.04
7.54
0.02
4.22
0.01
0.89
G7-15
2
4
0.01
1.47
0.29
36.02
0.03
3.87
G7-18
2
3
0.06
10.40
0.11
19.11
0.02
3.89
G7-19
2
4
0.03
3.96
0.01
1.09
0.04
4.42
G7-26
2
4
0.05
6.33
0.02
2.08
0.08
9.41
Foreign Manufactured Vehicles With Catalysts
D7- 4
2
2
0.02
15.89
0.23
150.62
0.03
17.81
17- 4
2
4
0.03
4.25
0.00
0.44
0.59
72.87
V7- 1
2
2
0.02
12.91
0.01
3.69
0.01
5.03
V7- 2
2
2
0.22
142.03
0.32
213.01
0.02
10.24
Foreign Manufactured Vehicles Without Catalysts
B7- 3
2
2
0.01
7.04
0.01
3.83
0.07
46.80
H7- 2
2
3
0.15
26.80
0.15
25.70"
0.06
9.83
17- 3
2
2
1.30
855.66
0.02
13.64
0.03
20.66
M7- 5
2
2
0.57
372.54
0.01
8.26
0.02
10.83
-------
163
EXHIBIT D-19: VEHICLE EMISSION DATA USED IN ANALYSIS OF RUNNING CHANGES
Vehicle ID Change No. Type
C 6- 1-3 0 EDV
C 6- 1-3 0 RC
C 6 - 1-3 0 RC
C 6 - 2-1 0 ED V
C6- 2-1 0 RC
C6- 2-1 0 RC
C6- 4-5 0 EDV
C 6- 4-5 0 RC
C6- 6-4 0 EDV
C 6- 6-4 0 RC
C6- 6-4 0 RC
C6- 9-3 0 EDV
C6- 9-3 0 RC
C6- 9-3 0 RC
C6-11-2 0 EDV
C 6-11-2 0 RC
C 6-12-4 0 EDV
C6-12-4 1 RC
C 6-12-5 0 EDV
C 6-12-5 2 RC
C 6-12-5 2 RC
C7- 5-5 0 EDV
C 7- 5-5 1 RC
F6- 6-1 0 EDV
F6- 6-1 0 RC
F6- 6-1 0 RC
F6- 6-3 0
F6- 6-3 I RC
F6- 6-7 0 EDV
F6- 6-7 0 RC
F6- 6-7 0 RC
F6- 6-8 0
F6- 6-8 0 RC
F6- 8-4 0 EDV
F 6- 8-4 0 RC
F6- 8-4 0 RC
HC
CO
NOX
0.26600
0.37000
0.33200
2.43000
2. 68000
1.93000
1.14000
1.22000
1.25000
0.21100
0.27300
0.21400
1.89000
3.67000
3.16000
1.49000
1.60000
1.29000
0.35400
0.35400
1.47000
1.78000
1.30000
1.21000
0.27600
0.32000
0.32000
5.47000
4.38000
4.78000
1.60000
1.75000
1.59000
0.43700
0.57600
0.47000
3.60000
4.11000
3.32000
2.00000
2.30000
1.97000
1.04000
0.46100
11.20000
2.79000
1.92000
2.41000
0.63200
0.85700
5.14000
2.56000
1. 89000
2.84000
0.93600
1.38000
0.83500
9.87000
11.00000
9.18000
2.86000
2.99000
3.44000
0.31000
0.20000
3.20000
2.80000
1.07000
1.08000
0.46400
0.41400
0.43600
5.68000
2.75000
7.17000
2.14000
2.17000
1.45000
0.52200
0.73000
3.75000
8.23000
2.26000
1.92000
0.75500
0.92700
0.91400
3. 18000
4.31000
8.11000
2.55000
2.38000
1.95000
0.71900
0.80300
1.96000
3.21000
1.96000
2.26000
0.60000
0.66400
0.50400
4.58000
5.87000
4.28000
1.29000
2.09000
1.77000
-------
164
EXHIBIT 0-19: VEHICLE EMISSION DATA USED IN ANALYSIS OF RUNNING CHANGES
(Continued)
Vehicle ID
Change No.
Type
HC
CO
NOX
F6- 8-5
0
ED V
0.36300
1. 50000
2.03000
F 6 - 8-5
0
RC
0.61500
2.26000
2.23000
F 7 - 1-1
0
ED V
0.25000
3.10000
0.76000
F 7 - 1-1
1
RC
0.27000
3.90000
0.72000
F 7- 2-1
0
ED V
0.56000
14.80000
0.85000
F 7- 2-1
1
RC
0.54000
13.00000
0.99000
F 7- 2-2
0
ED V
0.74000
4.90000
1.78000
F 7- 2-2
1
RC
0.55000
3.70000
1.57000
F 7- 2-2
1
RC
0.63000
3.80000
1.77000
F 7 - 2-4
0
ED V
0.78000
5.50000
1.56000
F 7- 2-4
1
RC
0.63000
4.10000
1.09000
F 7 - 2-4
2
RC
0.76000
4.90000
1.34000
F 7 - 2-4
2
RC
0.67000
5.00000
1.07000
-J
1
¦P-
1
0
ED V
0.74000
8.00000
1.47000
F 7 - 4-1
2
RC
0.55000
6.40000
0.83000
F 7 - 6-1
0
ED V
0.56000
2.90000
1.56000
T7- 6-1
2
RC
0.61000
2.50000
1.92000
F 7- 6-2
0
ED V
0.64000
3.60000
1.79000
F 7- 6-2
1
RC
0.61000
2.90000
1.88000
F7- 6-3
0
ED V
0.92000
5.60000
1.95000
F 7- 6-3
1
RC
0.94000
4.60000
2.05000
F 7- 7-1
0
ED V
0.28000
0.80000
0.96000
F7- 7-1
0
RC
0.28000
0.60000
0.94000
F7- 8-2
0
ED V
0.58000
3. 10000
1.82000
F7- 8-2
1
RC
0.52000
2.30000
2.01000
F7- 8-3
0
ED V
0.60000
8.40000
1.56000
F7- 8-3
2
RC
0.47000
4.90000
1.68000
F 7 - 8-3
3
RC
0.43000
4.70000
1.88000
F7- 9-2
0
ED V
0.62000
5.30000
1.60000
F7- 9-2
1
RC
0.67000
5.70000
1.50000
F 7-1l-l
0
ED V
0.67000
8.60000
2.03000
F 7-11-1
1
RC
0.68000
6. 10000
1.70000
F7-11-1
1
RC
0.72000
6.60000
1.63000
-------
165
EXHIBIT D-19: VEHICLE EMISSION DATA US!
(Continued)
Vehicle ID
Chanae No.
Type
F7-12-1
0
ED V
F7-12-1
2
RC
F 7-12-1
2
RC
F 7-12-1
3
RC
F7-12-1
4
RC
F 7-12-1
4.
RC
F 7-14-1
0
ED V
F7-14-1
1
RC
F 7-14-1
1
RC
F 7-14-3
0
ED V
F 7-14-3
0
RC
F 7-14-3
0
RC
F 7-14-4
0
ED V
F 7-14-4
0
RC
F7-14-6
0
ED V
F7-14-6
1
RC
F7-14-8
0
ED V
F 7-14-8
1
RC
F7-16-3
0
ED V
F7-16-3
0
RC
F 7-20-1
0
EDV
F 7-20-1
1
RC
F 7-20-2
0
EDV
F 7 -2 0-2
1
RC
F 7-20-2
2
RC
F7-20-2
9
RC
F 7-2 0-4
0
EDV
F 7-20-4
1
RC
F 7-2 0-4
2
RC
G6- 1-4
0
EDV
G6- 1-4
0
RC
G6- 3-1
0
EDV
G6- 3-1
0
RC
G6- 3-1
0
RC
G6- 4-1
0
EDV
G6- 4-1"
0
RC
G6- 4-1
0
RC
G6- 4-1
0
RC
IN ANALYSIS OF RUNNING CHANGES
HC
CO
NOX
0.88000
0.89000
0.72000
0.78000
1.00000
0.79000
14.80000
12.30000
8.70000
9.00000
17.70000
12.80000
1.16000
1.21000
1.36000
1.56000
1.03000
1.13000
0.55000
0.56000
0.60000
7.30000
10.00000
9.10000
1.48000
1.79000
1.52000
0.43000
0.53000
0.40000
5.00000
5.20000
2.30000
1.20000
1.46000
1.25000
0.48000
0.34000
6.70000
5.70000
1.23000
1.30000
0.41000
0.27000
4.50000
1.40000
1.53000
1.45000
0.51000
0.56000
8.10000
5.90000
1.37000
1.50000
0.67000
0.63000
13.40000
8.70000
1.52000
1.88000
0.64000
0.41000
7.00000
7.50000
1.07000
0.95000
0.55000
0.44000
0.33000
0. 29000
8.40000
9.90000
7. 10000
2.60000
1.32000
1.33000
1.37000
1.40000
0.57000
0.5 7000
0.51000
6.50000
7.40000
6.30000
1.64000
1.51000
1.74000
1.02000
1.45000
1 1. 80000
9.07000
2.44000
2.26000
0.46500
0. 37800
0.36200
7.28000
4.88000
1.74000
2.38000
1.98000
2.62000
0.25400
0.37700
0.47400
0.31700
1.71000
2.12000
4.96000
2.51000
2.72000
2.33000
2.55000
2.39000
-------
166
EXHIBIT D-19: VEHICLE EMISSION DATA USED IN ANALYSIS OF RUNNING CHANGES
(Continued)
'ehicle ID
Change No.
Type
HC
CO
NOX
G6- 4-3
0
ED V
0.74800
2.54000
1. 94000
G6- 4-3
0
RC
0.41100
1.92000
2.34000
G6-13-1
0
ED V
0.53200
2.60000
1.42000
G6-13-1
0
RC
0.46900
0.97000
1.35000
G6-13-2
0
ED V
0.43300
1.69000
1.20000
G6-13-2
0
RC
0.40100
1.70000
1.29000
G6-15-2
0
ED V
0.45200
3.84000
1.91000
G 6-15-2
0
RC
0.41600
3.47000
1.42000
G6-26-3
0
ED V
0.50800
7.26000
1.58000
G 6-2 6-3
0
RC
0.57200
6.61000
2.04000
G6-37-1
0
ED V
0.55100
7.36000
2. 16400
G 6-3 7-1
0
RC
1.41200
9.00000
2.37200
G6-37-1
0
RC
0.84700
8.84000
2.39400
G6-37-1
0
RC
0.85400
8.91000
2.18300
G6-40-2
0
EDV
0.40600
5.74000
1.34000
G6-40-2
0
RC
0.46300
5.13000
1.33900
G6-40-2
0
RC
0.45700
6.04000
1.24800
G6-43-1
0
EDV
0.35000
3.70000
1.76000
G6-43-1
2
RC
0.46000
7. 10000
1.93000
G6-43-1
2
RC
0.57000
10.00000
1.99000
G6-43-1
3
RC
0.47000
7.40000
1.77000
G6-43-1
3
RC
0.35000
5.70000
1.79000
G7- 8-4
0
EDV
0.62000
5.40000
1.47000
G7- 8-4
1
RC
0.76000
6.40000
1.45000
G7- 8-4
1
RC
0.71000
5.50000
1.77000
G7-12-1
0
EDV
0.29000
3.50000
1.66000
G7-12-1
1
RC
0.25000
2.50000
1.30000
G7-12-2
0
EDV
0.42000
4.50000
1.25000
G7-12-2
1
RC
0.38000
4.60000
0.93000
G7-12-2
1
RC
0.34000
4.60000
1.02000
G7-12-2
2
RC
0.24000
2.70000
0.98000
G7-14-1
0
EDV
0.53000
8.20000
2.02000
G7-14-1
1
RC
0.53000
8.00000
1.87000
G7-14-5
0
EDV
0.48000
4.60000
1.56000
G7-14-5
1
RC
0.62000
5.40000
1.43000
-------
167
EXHIBIT D-19: VEHICLE EMISSION DATA USED IN ANALYSIS OF RUNNING CHANGES
(Concluded)
Vehicle ID
Change No.
Type
HC
CO
NOX
G7-15-3
0
ED V
0.34000
4.40000
1.56000
G7-15-3
1
RC
0.28000
2.60000
1.56000
G7-15-3
1
RC
0.30000
2.60000
1.42000
G7-15-6
0
EDV
0.35000
8.90000
1.35000
G7-15-6
2
RC
0.34000
6.50000
1.67000
G7-15-6
2
RC
0.34000
5.70000
1.69000
G7-15-6
2
RC
0.36000
5.90000
1.68000
G7-15-6
2
RC
0.38000
8.30000
1.38000
G 7-1 6-5
0
EDV
0.50000
4.30000
1.67000
G7-16-5
1
RC
0.54000
5.70000
1.39000
G7-17-4
0
EDV
0.64000
5.10000
1.80000
-*
i
i—
.—t
i
r>
O
2
RC
0.45000
2.00000
1.45000
G 7-18-1
0
EDV
0.48000
9.90000
1.77000
G 7-18-1
1
RC
0.59000
12.90000
1.70000
G 7-18-1
1
RC
0.55000
10.40000
1.92000
G7-18-3
2
EDV
0.38000
4.90000
1.44000
G7-18-3
2
RC
0.62000
8.90000
1.51000
G 7-1 8-3
3
RC
0.59000
9.30000
1.44000
G7-19-1
0
EDV
0.36000
4.50000
0.87000
G7-19-1
1
RC
0.37000
5.20000
0.98000
G7-21-3
0
EDV
0.41000
9.90000
1.42000
G7-21-3
1
RC
0.40000
3.60000
1.81000
G7-25-2
0
EDV
0.29000
4.80000
1.69000
G7-25-2
1
RC
0.23000
5.30000
1.09000
G7-25-3
0
EDV
0.24000
3.50000
1.42000
G7-25-3
1
RC
0.25000
2.50000
1.06000
G 7-26-1
0
EDV
0.38000
3.40000
1.58000
G7-26-1
1
RC
0.40000
4.80000
1.78000
G7-26-1
2
RC
0.39000
1.50000
1.82000
G 7-2 6-1
4
RC
0.42000
4.40000
1.58000
G7-26-5
0
EDV
0.45000
5.20000
2.00000
G7-26-5
1
RC
0.46000
3*90000
2.12000
G7-26-5
1
RC
0.49000
6.70000
2.16000
-------
168
-------
169
APPENDIX E
ANALYSIS OF ALTERNATIVE TESTING STRATEGIES
This appendix discusses the application of the effectiveness methodo-
logy described in appendix F to alternative procedures for EPA's conduct
of the certification program. Four general types of strategies were
analyzed using this methodology. Following a summary description of the
approach and results of this analysis in section E.l, the analysis of
each of these four areas is described in turn:
(1) Application of the methodology to EPA's current procedures is
described in section E.2.
(2) Analysis of alternative criteria for deciding whether a single
emission data vehicle (EDV) exceeds the standards is considered
in section E.3.
(3) Analysis of sequential testing strategies (in which the number
of EDV's to be tested in an engine family depends on the results
of tests on a portion of the family) 1s described in section E.4.
(4) Consideration of the potential effectiveness of modified uses of
engineering judgment in testing decisions is discussed in section
E.5.
E.l Approach and Sunmary
This section summarizes the application of the effectiveness methodology
to the evaluation of the types of strategies listed above. The effectiveness
-------
170
measures computed are defined and discussed in section E.l.l. The basic
assumptions and data employed in the analysis are included in section E.1.2.
Finally, section E.1.3 presents a summary of the results of the analysis
which is described in greater detail in the remainder of this appendix.
E.l.l Effectiveness Measures Used
For each strategy evaluated by the effectiveness methodology, several
measures of the effectiveness and risk of the strategy were computed. These
included averages of the following probabilities:
(1) the probability an EDV is failed by the certification process;
(2) the probability an EDV whose true emission rates exceed the
standards is failed by the process;1 and
(3) the probability an EDV whose true emission rates are within the
standards is failed by the process.2
To obtain a more direct measure of the effectiveness of a given strategy,
an attempt was also made to predict differences in pollution emitted by ve-
hicles certified under various strategies. In particular, the decrease from
current levels in lifetime emissions of all vehicles certified in a given
model year was computed for each strategy analyzed.
In addition to the above effectiveness and risk measures, the cost
impacts of the alternative strategies were evaluated. For each strategy,
^ne minus this probability is referred to elsewhere in this report as the
probability of erroneously passing a vehicle.
2This is referred to elsewhere in this report as the probability of er-
roneously failing a vehicle.
-------
171
changes in EPA testing requirements from those of current procedures were
estimated, and the effects on EPA personnel requirements were explored.
E.1.2 Assumptions and Data Used
This section provides a brief summary of the assumptions and data used
in this analysis. Additional details appear 1n appendix F.
The effectiveness analysis was conducted using certification data for
1977 model year vehicles. The 1977 emission standards of 1.5 grams per mile
for HC, 15 grams per mile for CO, and 2.0 grams per mile for NO were there-
A
fore assumed to be in effect.1 While most of the analysis was conducted on
VRI's data base for domestic vehicles with catalysts, some analysis of the
major types of strategies investigated was also performed on VRI's foreign
non-catalyst data base to assure that recommendations resulting from the
analysis would be generally applicable. Unless otherwise specified, results
presented 1n this appendix are those associated with the domestic data base.
Actual family means from this data base for each pollutant (Including deterlora
tlon effects) were used to compute effectiveness results for an empirical
distribution of family means. Values of these means before deterioration,
and the corresponding deterioration factors, are presented 1n exhibit E-l
for both the domestic and foreign data bases. The data base was used to
develop estimates of design-to-design differences within a family. The
pollutant level for a given design within a family was assumed to be a
normally-distributed random variable with mean equal to the family mean
and standard deviation as shown In exhibit E-2. The levels for the three
pollutants were assumed to be uncorrected.
*In actual computations, standards of 1.55, 15.5, and 2.05 were used to allow
for the effects of EPA's round-off procedures.
-------
172
A portion of the analysis alternatively represented the pollutant level
of a vehicle selected at random from the entire domestic fleet as a random
sample from a mixture of two normal distributions, where each distribution
was weighted by a factor of 0.5. The means (e-| and 92) and standard devia-
tions (yj and y^) of these distributions for each of the three pollutants
are presented in exhibit E-3. These parameter values were developed from
the data of exhibit E-1, as described in appendix F,
The results of a test on a single vehicle were assumed to be normally
distributed with means equal to the actual pollution levels for that design
and standard deviations as shown in exhibit E-2. Pollutant levels were
again assumed to be uncorrected.
E.I.3 Summary of Results
Exhibit E-3 summarizes the results of the cost and effectiveness analysis
of representative examples of the testing strategies considered. These
examples include the following:
(1) EPA's current strategy;
(2) EPA's current strategy except that, when a manufacturer requests
a retest, the results of the two tests are averaged to determine
whether the vehicle is passed.
(3) EPA's current strategy except that retests are not allowed (i.e.,
a one-test strategy);
(4) a strategy in which every vehicle is tested twice and the results
of the two tests are averaged (a two-test strategy);
(5) a sequential strategy in which considerable evidence is required
-------
173
before a decision is made whether to test the remaining EDVs in
the family1 (a conservative sequential strategy);
(6) a sequential strategy based on less evidence per family1 (a non-
conservative sequential strategy);
(7) a strategy in which engineering judgment is used to identify high-
emitting families, which are then tested more extensively than
other families; and
(8) a strategy in which engineering judgment and a sequential testing
strategy are used in conjunction with one another to Identify high-
emitting families.2
The table presents results in terms of the effectiveness and cost measures
described earlier. The following sections develop in more detail the material
summarized 1n the table.
E. 2 Current Procedure Effectiveness
Use of the effectiveness methodology to analyze EPA's current certification
procedures provides both a check of the methodology and a set of "baseline"
effectiveness values against which to judge alternative strategies. A
summary of the results for this strategy was presented previously 1n exhi-
bit E-4. A frequency distribution of failure probabilities by engine family
1s shown 1n exhibit E-5. More than 90 percent of all failed vehicles are
concentrated 1n 50 percent of the families .
xSee section E.4 for detailed specifications of this strategy.
2See section E.5 for detailec* specifications of this strategy.
-------
174
It is of interest to note that the overall percent of EDVs predicted by
the methodology to be failed is 9.8 percent, whereas the actual percent failed
from these families was 8.3 percent. The effectiveness computations thus pro-
duce a reasonably accurate estimate of total failure in the fleet.1 In addition,
the estimate generated from the analysis of the expected number of tests re-
quired per EDV tested is very close to that used by EPA in its planning. Using
EPA's assumption of a test void rate of .175, and assuming two EDVs are submitted
by the manufacturer after modifications associated with a failed EDV, the fail-
ure rates predicted by the methodology result in an average of 1-64 tests con-
ducted per EDV tested. EPA's estimate used for internal planning [EPA, undated]
is 1.69 tests per EDV.
It is assumed that the effect on total emissions of EPA's failure of
vehicles is that the vehicles are redesigned so that their emissions allow them
to be passed. Under this assumption, the current certification process has a
direct benefit of reducing the emissions of the average vehicle by approximately
0.4 percent for HC, 3.0 percent for CO, and 1.4 percent for N0V (e.g., the NO
A A
saving equals 9.8 percent times the average improvement of N0X emissions of
failed vehicles of approximately 14.3 percent). This benefit is as small as it
is because most vehicles (approximately 87 percent,2 according to the analysis)
submitted for certification have already been designed to meet the standards.
Furthermore, even those EDVs failed by the certification process doe not generally
^•Application of the methodology to the foreign families in VRI's data base result^
in a prediction that 5 percent
-------
175
exceed the standards by much. The average excess over the standard for such
vehicles is approximately 0.5 percent of the standard for HC, 3.4 percent for
CO, and 4.5 percent for N0X» Failures are more than 90 percent due to excessive
CO and N0X emissions, so the typical failed vehicle can be expected to have
its true emissions about 8 percent above the standard on either CO or N0X>
A change in the certification process might be expected to have an effect
on these true emission rates resulting from manufacturer response to the new
procedures. To evaluate the impact of such a change, the effectiveness method-
ology was applied to a fleet of vehicles whose average emissions for each pol-
lutant were 10 percent lower than the emissions of the empirical fleet. The
result was that the failure rate dropped by approximately 8 percentage points.
A family-by-family study of these results led to the conclusion that this change
1n the failure rate could be achieved by reducing the emissions of only half of
the families), producing an overall decrease 1n emissions of about 7.5 percent.
On this basis, one can predict that a one percent increase (or decrease) 1n
emissions would result 1n approximately a one percent increase (or decrease) 1n
the failure rate. This result can be used as an approximate representation of
the extent to which a manufacturer would have to change the emissions of his
fleet in order to maintain an acceptable over-all failure rate as EPA's certifi-
cation procedures changed.
E. S Alternative Compliance Criteria
EPA's current criterion for passing of an EDV by the certification process
1s that the vehicle's measured emissions must be below the standard for each
-------
176
pollutant (after adjustment for projected deterioration) on at least one of
a possible two tests. An analysis was conducted of alternative criteria to
this one, emphasizing differences in two areas:
(1) the possibility of averaging the results of multiple tests on
a single vehicle, and
(2) the possibility of changing the number of tests conducted on each
vehicle.
Three strategies were analyzed to gain insights into these two areas:
(1) EPA1s current strategy except that the results of any retests
are averaged with the results of the first valid test on the
vehicle,
(2) a strategy in which one test is conducted on every EDV (and
retests are not allowed), and
(3) a strategy in which every vehicle is tested twice and the results
of the two tests are averaged.
The results for each of these strategies were summarized in exhibit E-4.
The savings in emissions tests shown in the exhibit (for these and other
strategies) were computed from the previously-stated assumptions that the
test void rate is .175 and that a failure results in the later resubmission
and testing of two modified EDVs. The totals were based on projected num-
bers of EDVs for the 1978 model year given in [EPA, undated].
E.4 Sequential Testing Strategies
A sequential testing strategy can be defined as follows:
In each engine family, test n EDVs. If the emission level of
pollutant i (i = 1, 2, 3) for at least k of the n vehicles (k < n)
-------
177
is less than a threshold t^ for all i, certify the family without
further testing. Otherwise, test all EDVs 1n the family. (The
values of t^ will generally be somewhat below the standard for
pollutant i.)
Results of application of the effectiveness methodology to several se-
quential testing strategies are summarized 1n exhibit E-6. The exhibit
defines each strategy 1n terms of the parameters n, k, t^, tg, and tg. Values
of the thresholds were selected heurlstlcally based on an analysis of the re-
presentation of the distribution of fleet emissions as a mixture of two nor-
mal distributions (see section.E.l.2). Additional results for two represen-
tative examples of these five strategies were displayed in exhibit E-4, where
the "conservative" sequential strategy corresponds to the case in which n » 2
and k * 2, while the "non-conservative" strategy corresponds to the case 1n
which n ¦ k ¦ 1, t^ * .7, tg ¦ .7, and t3 ¦ .75. For both of these strategies,
57 percent of all families were tested completely (so that 43 percent of the
families had only one or two vehicles tested).1
In addition to the sequential strategies discussed above, one sequential
strategy was analyzed in conjunction with a certification policy of averaging
test results whenever a manufacturer requests a retest. The particular se-
quential strategy used had n * k • 1, tj • .6, t2 • .6, and t3 » .7. The
result was that 11 percent of all vehicles were failed, 3 percent of
vehicles whose true emissions levels met the standards were erroneously
failed, and 61 percent of vehicles whose true emissions exceeded the
standards were failed. Approximately 3®) tests would be saved annually
by adoption of such a combined strategy.
¦ ii————
Comparable results were obtained from application of a sequential strategy
1n which n * %* 1 to the foreign families 1n VRl's data base. In this case,
60 percent of all ^families were tested completely, and 90 percent-of failures
under the current strategy were tested under the sequential strategy (can-
pared to 80 to almost 100 percent for the domestic fleet).
-------
178
E.S Strategies Involving Modified Use of Engineering Judgment
Some analysis was conducted to determine the potential effectiveness of
the use of engineering judgment to identify high-emission families, which
could then be tested more extensively than other families. To obtain an
upper bound on this effectiveness, a case was analyzed under the assumption
that EPA's certification engineers could identify the 50 percent of engine
families having the highest average rejection probabilities. (Earlier analysis
indicated that these families contain 91 percent of all vehicles rejected
under EPA's current procedures.) If only vehicles in these families were
tested, the average failure probability (for the entire fleet) would be .09,
the probability of erroneously failing a vehicle would be less than .02,
and the probability of failing a vehicle whose true emissions exceed the
standards would be .58. These results are quite close to those for EPA's
current procedures, and would be achieved with a 50 percent reduction in
the number of EDVs tested.
Probably a more realistic representation of the capability of engineers
to identify families with high failure rates is outlined in exhibit E-7.
The exhibit hypothesizes a probability that each family in the domestic data
base is selected as a function of that family's rank by average failure pro-
bability. Application of these probabilities resulted in selection of 70
percent of all families. The average probability that a vehicle in a
selected family exceeds one or more standards is .16 (compared to .13 for
the fleet as a whole). Results were presented in exhibit E-4 for the strategy
in which all EDVs are tested in families selected by engineering judgment
(with these probabilities), and no other EDVs are tested.
-------
179
This same assumed capability to identify families with high emissions
was combined with two previously-introduced sequential strategies in the
following way:
For families selected by engineering judgment (using the probabilities
given in exhibit E-7), apply the sequential strategy in which n = k = 2.
For all other families, use the sequential strategy in which n » k » 1,
and t.| = .7, tg a .7, and tg « .75.
Results for this case were also presented in exhibit E-4.
-------
180
EXHIBIT E-1: FAMILY MEAN EMISSIONS AND DETERIORATION FACTORS
USED IN EFFECTIVENESS ANALYSIS
HC CO NOX
Family ID
# EDVs
Mean
DF
Mean
DF
Mean
DF
C 7- 1
5
0. 388
1.412
3. 180
1. 000
1. 223
1. 000
C 7- 2
5
0. 254
1. 276
3.337
1. 186
0.950
1.199
C 7- 3
2
0.215
1. 524
2. 800
1. 105
1.370
1. 000
C 7- 4
3
0. 180
1. 824
4. 117
1. 286
1. 200
1. 088
C 7 - 5
5
0. 216
1. 259
3. 090
1. 354
1. 063
1.015
C 7- 7
5
1. 134
1.216
5. 990
1. 685
1. 644
1. 000
C 7- 8
4
0. 230
1. 000
3. 733
1.420
1.771
1. 000
C 7- 9
6
0. 244
1.512
2. 325
1. 649
1. 914
1. 000
C 7-1 0
2
0. 220
1. 000
1. 750
1. 000
1. 630
1.018
C 7-11
5
0.297
1. 590
3. 430
1. 000
2. 038
1. 000
C7-12
5
0. 900
1. 000
9. 540
1. 000
1. 552
1.080
C 7-1 3
6
0. 901
1. 000
8. 467
1. 000
1.870
1.021
C7-14
7
0. 854
1. 005
10.800
1. 038
1. 685
1. 092
C 7-1 5
6
0. 661
1.099
10. 817
1. 000
1.618
1. 095
C 7-1 6
6
0. 507
1.000
6. 908
1. 000
1. 786
1. 000
F 7 - 1
4
0. 402
1. 580
3. 112
1. 288
0.726
1.161
F 7- 2
5
0. 594
1. 034
9. 490
1. 000
I. 244
1. 000
F 7 - 4
3
0. 733
1. 695
8.567
1. 284
1. 393
1. 124
F 7 - 5
2
0. 265
1. 000
3. 800
1. 027
1. 060
1.023
F 7 - 6
3
0. 713
1. 524
4.350
1. 372
1. 883
1. 000
F 7- 7
2
0. 295
1. 273
1. 200
1.086
1. 150
1.043
F 7 - 8
6
0. 652
1.453
8. 600
1. 442
1.629
1. 000
F 7- 9
2
0.632
1. 167
7.550
1.571
1. 587
1. 000
F 7-1 0
3
0. 240
1.212
2. 150
1. 305
1.237
1. 000
F7-1 1
1
0. 670
1.168
8. 600
1. 000
2. 030
1. 000
F7-12
3
0. 743
1.316
9.467
1. 000
1. 503
1. 107
F7-14
8
0.515
1. 426
6. 456
1. 656
1. 322
1. 063
F 7-1 5
7
0. 419
1. 182
7.714
1. 000
1.714
1. 000
F 7-1 6
4
0. 634
1. 426
10. 325
1. 008
1. 924
1. 000
F 7-1 7
3
0. 633
1. 799
9.917
1. 000
1.535
1. 000
F7-18
1
1. 000
1. 004
11.600
1. 000
2. 000
1.000
F7-19
3
0. 393
1. 162
5. 033
1.091
1. 280
1. 000
F 7-20
4
0. 650
1. 323
9. 612
1.412
1. 384
1. 000
G7- 1
6
0. 270
1. 112
5. 583
1. 000
1.713
1. 000
G 7 - 2
6
0.589
1. 000
9. 142
1.000
1.577
1.066
G 7 - 3
4
0. 173
1.471
1. 925
1.078
1. 270
1.000
G7- 4
5
0.420
1.217
5. 120
1. 000
1. 698
1.000
G 7 - 5
5
0. 384
1.113
1. 560
1. 000
1.410
1. 000
G 7 - 6
4
0. 502
1. 221
8. 600
1. 000
1. 535
1. 004
G 7 - 7
4
0. 300
1. 000
4. 100
1. 238
1. 065
1. 000
G 7 - 8
5
0. 524
1. 000
5. 260
1. 000
1. 568
1. 000
G7- 9
4
0. 262
1. 000
2.525
1. 000
0. 972
1.011
G 7-10
7
0. 434
1. 306
6. 779
1.000
1.516
1. 008
G7-11
5
0. 566
1. 000
5. 980
1. 000
1.668
1. 000
G7-12
2
0. 348
1. 301
4. 000
1. 124
1. 445
1. 129
G7-13
5
0. 759
1. 145
8.070
1. 059
1. 801
1.042
G7-14
6
0. 532
1. 161
6. 792
1.024
1. 873
1. 000
G7-15
6
0. 331
1.015
5.117
1. 000
1. 366
1. 000
-------
181
EXHIBIT E-l: FAMILY MEAN EMISSIONS AND DETERIORATION
FACTORS USED IN EFFECTIVENESS ANALYSIS
(Concluded)
HC
Family ID
#EDVs
Mean
DF
G 7
-I 6
5
0. 673
1. 908
G 7
-17
4
0. 550
1. 669
G 7
-18
5
0. 614
1. 853
G 7
-19
6
0. 282
1. 103
G 7
-2 0
2
0. 560
1. 649
G 7
-21
3
0. 480
2.217
G 7
-2 2
2
0. 600
1. 000
G 7
-2 3
1
0.365
1. 300
G 7
-2 4
4
0. 699
1. 118
G 7
-2 5
4
0.277
1.210
G 7
-26
6
0. 414
1. 374
B 7
- 1
2
0. 220
1. 000
B 7
- 2
2
1. 065
1. 000
B 7
- 3
4
0. 320
1.000
D 7
- 2
4
1.245
1. 050
D 7
- 5
3
1.312
1. 164
D 7
- 7
2
1. 155
1. 157
D 7
- 9
2
1. 230
1. on
H 7
- 1
2
0. 695
1. 000
H 7
- 2
5
0. 260
1. 000
H 7
- 3
5
0.911
1.000
17
- 1
2
0. 867
1.089
I 7
- 3
4
0. 699
1. 204
M 7
- 3
2
0.565
1. 000
co nox
Mean
DF
Mean
DF
7. 780
1.018
1. 648
1.017
4. 750
1.059
1. 760
1. 000
7. 540
1. 440
1. 298
1. 028
3. 175
1. 000
0. 852
1. 000
10.250
1. 174
1. 235
1. 076
6. 133
1.476
1. 657
1. 000
7. 050
1.316
1. 560
1. 000
3. 200
1. 416
1.415
1. 074
11.150
1. 001
1.731
1.052
3.275
1. 112
1. 702
1.000
3. 867
1. 593
1. 829
1. 000
6. 975
1. 000
1.047
1. 000
8.550
1.000
1. 400
1.000
8. 850
1. 000
1. 275
1. 000
9. 650
1. 000
1. 360
1. 000
10. 567
1. 156
1. 620
1.000
7. 550
1.210
1. 385
1. 000
6. 000
1.000
1. 555
1.002
9. 650
1.000
1.525
1.000
3. 060
1.000
1. 226
1. 000
5.040
1.000
1. 583
1.000
8. 675
1.061
1.467
1. 000
10.962
1.000
1.324
1. 000
10.150
1.000
1.550
1. 118
-------
182
EXHIBIT E-2: TEST-TO-TEST AND DESIGN-TO-DESIGN VARIABILITY
ASSUMED IN EFFECTIVENESS ANALYSIS
Pollutant
Design
Standard
Deviation1
Test
Standard
Deviation1
Domestic
(With Catalyst)
Forei gn
(Non-Catalyst)
Domestic
(With Catalyst)
Foreign
(Non-Catalyst)
HC
.20
.20
.13
.13
CO
.30
.15
.14
.07
NOx
.16
.10
.09
.04
Expressed as a fraction of the family mean for the corresponding pollutant.
-------
183
EXHIBIT E—3: PARAMETERS OF DISTRIBUTION
OF VEHICLE EMISSION LEVELS
\ Parameter
Pollutant
91
Y1
92
Y2
HC
.25
.15
.60
.20
CO
.25
.20
.60
.25
N0X
.65
o
CM
•
.85
.15
^ach value 1s expressed as a fraction of the corresponding standard.
-------
EXHIBIT E-4: SUMMARY OF RESULTS OF EFFECTIVENESS ANALYSIS
AVERAGE PROBABILITIES
SAVINGS IN POLLUTION1 (LIFE-
TIME TONS PER VEHICLE
IN CERTIFIED FAMILIES)
REDUCTION IN EDVs
TESTED1 (VEHICLES
PER YEAR)
SAVINGS IN
EMISSIONS
TESTS1 (TEST
STRATEGY
FAILURE
FAILURE GIVEN
WITHIN STANDARDS
FAILURE GIVEN
EXCEEDS STANDARDS
HC
CO
"Ox
PROCEDURES
PER YEAR)
1. Current
0.10
0.02
0.62
0.0
0.0
0.0
0
0
2. Current with
averaging on
failure
0.14
0.05
0.85
.00016
.0121
.00105
0
-94
3. One test
always
0.16
0.07
0.77
.00013
.0126
.00094
0
84
4. Two tests
always
0.15
0.05
0.85
.00016
.0121
.00105
0
-1332
5. Conservative
sequential
testing
0.09
0.02
0.62
0.0
0.0
0.0
125
158
6. Non-
conservative
sequential
testing
0.0a
0.02
0.46
-.0006
-.0056
-.00058
265
392
CO
-p»
'Savings are in comparison witti current strategy. A negative value indicates an increase over the current strategy.
Values assume no change in manufacturer behavior in response to new strategy.
-------
EXHIBIT E-4: SUMMARY OF RESULTS OF EFFECTIVENESS ANALYSIS
(Concluded)
AVERAGE PROBABILITIES
SAVINGS IN POLLUTION1 (LIFE-
TIME TONS PER VEHICLE
IN CERTIFIED FAMILIES)
REDUCTION IN EDVs
TESTED1 (VEHICLES
PER YEAR)
SAVINGS IN
EMISSIONS
TESTS1 (TEST
STRATEGY
FAILURE
FAILURE GIVEN
HITHIN STANDARDS
FAILURE GIVEN
EXCEEDS STANDARDS
HC
CO
N0X
PROCEDURES
PER YEAR)
7. Judgmental
fanny
selection2
0.08
0.02
0.50
-.00006
-.0046
-.00047
281
418
8. Judgmental
family
selection
and sequen-
tial test-
ing2
0.09
0.02
0.59
-.00002
-.0015
-.00014
167
228
'Savings are in comparison with current strategy. A negative value Indicates an increase over the current strategy.
Values asswe no change in manufacturer behavior in response to new strategy.
2Results for this case are hypothetical in that they are based on an assumed capability to identify high-emission
families (see section E.5 for details).
-------
EXHIBIT E-5: FREQUENCY DISTRIBUTION OF FAILURE PROBABILITIES BY ENGINE FAMILY
FOR EPA'S CURRENT CERTIFICATION PROCEDURES
PROBABILITY OF fAILURE
-------
EXHIBIT E-6: RESULTS FOR SEVERAL SEQUENTIAL TESTING STRATEGIES
STRATEGY
RESULTS
NUMBER OF
VEHICLES
TESTED
(n)
NUMBER WHICH
MUST BE WITHIN
THRESHOLDS
-------
188
EXHIBIT E-7: REPRESENTATION OF HYPOTHETICAL
CAPABILITY TO IDENTIFY FAMILIES
WITH HIGH FAILURE RATES
FAMILY RANK1
PROBABILITY FAMILY IS SELECTED
1, 2, 12
.9
13, 24
.8
25, ..., 36
.7
37, 48
.6
49, ..., 59
.5
tanked in descending order of average failure probability.
-------
189
APPENDIX F
METHODOLOGY FOR EFFECTIVENESS ASSESSMENT
This appendix describes a methodology used to assist in predicting the
effectiveness of modifications to EPA's Light-Duty Vehicle certification
process.1 A set of probabilistic models has been developed to predict
EPA's certification decisions associated with a fleet of vehicles submitted
by manufacturers, and to compute the effectiveness of such decisions in
terms of several effectiveness measures selected for the analysis. Data for
the models were taken from EPA's records of emission test results and other
records maintained by EPA.
The effectiveness methodology can be conceptually divided into four
parts, as follows:
(1) A fleet representation predicts the probabilistic characteristics
of the vehicles which will be submitted to EPA for certification.
(2) A representation of the vehicle selection process describes how
EPA processes information submitted by manufacturers so as to
select emission data vehicles for testing.
(3) A representation of the emissions testing and certification process
predicts the outcome of EPA's testing process on the emission data
vehicles within each engine family. Several representations of this
process are required to represent different types of policies for
certifying engine families based on emissions test results.
:The methodology excludes that part of the certification process dealing with
EPA's procedures to predict emissions deterioration, since VRI's current
study is not addressing deterioration estimation.
-------
190
(4) An effectiveness analysis is performed to compute the values
of selected effectiveness measures resulting from certification
decisions.
Exhibit F-l illustrates the inputs, outputs, and interactions of the four
parts of the methodology. The following sections describe the details
of each of the four parts in turn.1
F.1 The Manufacturers ' Fleet
The fleet representation is used to predict characteristics of the vehicles
submitted to EPA for certification. Historical certification data have been
used to generate probabilistic descriptions of
(1) family mean emission levels for HC, CO, and N0X, including the
effects of deterioration, and
(2) configuration-to-configuration differences in emission levels
within a family, given the family means.
The nature of these two types of descriptors is outlined in the following
paragraphs.
For most uses of the effectiveness methodology, an empirical distribution
of family means was used. The effectiveness computations were performed for
each of a set of families having the same mean emission levels as the families
in the VRI 1977 model year data base. In addition, for some sensitivity analyse5
sets of various hypothetical family means were also used in the computations.
1Implementation of some of these models has involved certain numerical approxi-
mations which are not reflected in the text.
-------
191
Design-to-design emission differences within an engine family are
represented by three independent1 normal distributions centered at the family
mean for each pollutant. Estimates of the standard deviation of this distri-
bution for each pollutant were generated from an analysis of variance on the
data base. In each case, the standard deviation was assumed to be propor-
tional to the family mean with coefficients of variation which were presented
in appendix E.
A portion of the analysis has used an alternative representation of the
true emissions of vehicles in the fleet. A heuristic fit was made to the
overall distribution of vehicle emissions. The result was a mixture of two
normal distributions, allowing the true emissions of pollutant 1, of a
vehicle selected at random from the entire fleet to be represented as
Pr[yi < a] » .5P ~ + .5p " Q2 j
where
P(a) * cumulative standard normal distribution
1 •",/!«•
empirically-determined means of the two normal distributions, and
empirically-determined standard deviations of the two normal distributions.
Actually, HC and CO emissions are positively correlated. "rSle*10"
is sufficiently small that 1t may be neglected for these "
cription of an analysis of the correlation among pollutant levels
engine family can be found in appendix D.
0102 „
Y1Y2 *
-------
192
The parameters of this distribution were selected to give the same percent of
vehicles above and below the standard for designs which are near the standard
as does the full empirical distribution above. In addition, use of this
distribution produces the same percent of vehicles failed by the certifica-
tion process as does the full empirical distribution for each of three very
different strategies to which it was applied. The parameter values actually
used were presented in appendix E.
F.2 EPA's Prototype Vehicle Identification
The analysis of EPA's current vehicle identification scheme described in
appendix D was used to characterize the vehicle identification process as
represented in the effectiveness methodology. As a result, it is assumed i;n
most of the effectiveness analyses that each emission data vehicle is a randomly
selected configuration from the engine family to which it belongs.
F, S EPA's Emissions Testing and Certification Process
Given a vehicle has been selected for testing, the representation of the
emissions testing and certification process predicts probabilistic results of
testing that vehicle and specifies a probability that the vehicle passes
the certification process. The results of a single emissions test on a vehicle
(after application of the appropriate deterioration factor1) are assumed to be
three independent normally-distributed random variables (one for each pollutant)
with means equal to the corresponding true emission levels for the configuration
being tested. Development of estimates of the standard deviation for this
1Because the scope of this study was limited to the emissions measurement
program, estimation of deterioration was assumed to be exact in this analysis.
Thus, the effects of variability due to the deterioration estimation process
not reflected in the results of the analysis.
-------
193
distribution was described in appendix B. The standard deviation for each
pollutant is assumed to be proportional to the mean with coefficients of
variation described in appendix B (and also presented in appendix E).
The assumption of independence among pollutants was developed from an
analysis of data presented by [Paulsell and Kruse, 1974].1
To express this distribution notationally, let
u.| = true emission level of pollutant 1 for the EDV being
tested,2
ck = test-to-test standard deviation for pollutant i, and
P(a) » cumulative standard normal distribution.
Then the probability that x^, a vehicle's measured emissions for pollutant i
after application of the appropriate deterioration factor, is less than or
equal to a is given by
xThis was the most recent data available for which a sufficient nuirtier of tests
had been conducted on a single vehicle to allow for an analysis of correlation
among pollutants.
2Note-that is itself assumed to be a random variable whose distribution was
described in section F.l. The randomness of y, is treated numerically as
described in section F.4.
-------
194
Given the above characterization of the results of a single test,
prediction of a decision as to whether a vehicle is in compliance with
the Clean Air Act is a function of the emission standards and the specific
policy by which EPA compares test results to these standards. Five types
of policies have been investigated in this study:
(1) EPA's current policy, in which a vehicle passes the
certification process only if the measured emission levels of
all pollutants fall within the standard on at least one of two
tests conducted on the vehicle;1
(2) a policy which allows only one test to be conducted;
(3) a policy by which every vehicle is tested twice and the results
of the two tests are averaged;
(4) EPA's current policy except that, when the vehicle exceeds one
or more standards on the first test, a retest is conducted and the
results of the two tests are averaged; and
(5} policies involving sequential testing, in which a vehicle may be
passed without testing if the results of testing one or more other
vehicles in the same engine family fall within acceptable limits.2
Procedures to predict the probability a vehicle is failed by the certification
process under each of the above types of policies are described in the following
sections.
XEPA's actual procedure allows the manufacturer to request a retest if the
vehicle exceeds one or more standards on the first test. The representation
of the procedure described here assumes the manufacturer always exercises
this option.
2In this case, EPA's current policy is applied to those vehicles which are
tested. One case was also run in which the policy of averaging results in
case of a retest is applied to vehicles tested.
-------
195
F.3.1 EPA's Current Policy
Under EPA's current policy, a vehicle fails the certification
process if the measured levels of its three pollutants (including deterioration)
are not all within the standard on at least one of two tests. Assuming that the
results of the two tests are independent, the probability a vehicle is failed
under this policy can be expressed in previously-introduced notation as
where F is the event that the vehicle is failed and s.. is the emissions
standard for pollutant i.
F. 3.2 Single-Test Policy
A policy which does not allow the manufacturer to request a retest but
which simply accepts the result of a single test (and rejects the vehicle if
any pollutant exceeds the standard) results fn the following simple expression
for the probability the vehicle is failed.
Pr[F] -
-------
196
F.3.3 Two-Test Policy
A policy which requires every vehicle to be tested twice and then averages
the results of the two tests also produces a simple expression for the proba-
bility the vehicle is failed. For a given pollutant i, the average result of
two tests is normally distributed with mean equal to and standard deviation
ai
equal to —7—= Therefore, the probability a vehicle fails the
V2
certification process is simply
to ¦n .
F.3.4 Current Policy With Averaging on Failure
Consider a variant of EPA's current policy in which a failure of a single
test results in a retest, but the results of this retest are averaged with
those of the first test to produce the values which are compared with the
standards. To derive the expression for the probability of failure for this
situation, some additional notation is needed. Let
(xp Xg, Xj) = the result of the first test on the vehicle (a random
vector consisting of the measured emission level for each
of the three pollutants), and
(r-|, f*2' **3) = ^e result of the second test (if it is conducted).
Note that, as before, x.. and r^ are independent, normally-distributed random
variables each with mean and standard deviation a.. Define
the average result of the first and second tests. Then y^ is normally-distribut®''
-------
197
with mean y. and standard deviation ai^r- frurtheir» the joint distribution
of x.. and is a bivariate normal whose correlation coefficient can be
shown to be equal to 1/^ for any i.
The probability a vehicle is failed by the certification process can
thus be expressed as a sum of products of normal and bivariate normal distri-
bution functions. The exact expression is one minus the expression given in
exhibit F-2 for the probability a vehicle is accepted. The exhibit uses the
following notation (adopted from [Abramowitz and Stegun, 1970]):
L(h,k,p) = standard bivariate normal distribution function
The distribution functions in the exhibit are expressed in terms of the
parameters u., v^, and p, where
oo
00
h k
where
and
p
= correlation coefficient =
-------
198
F.3.5 Policies Involving Sequential Testing
A particular sequential testing strategy can be characterized by the
following parameters:
t.. = threshold for pollutant i, and
n = number of EDVs to which the thresholds are applied.
The strategy then can be expressed as follows. In each engine family, test
n EDVs, where n _< N, the total number of EDVs in the family. If the emission
level of pollutant i for all n vehicles is less than t^ for all i, certify
the family without further testing. Otherwise, test all EDVs in the family.1
The following paragraphs develop an approximate form for computing Pr[F],
the average probability a vehicle is failed by the certification process, in
the special case in which n = 1. Similar approximations can be used for cal-
culating P[F] for other classes of sequential strategies.
Assuming t^ <_s^ (i=l,2,3) in the case in which n = 1, the value of Pr[F]
can be expressed as the sum of (1) the probability a given vehicle is
selected as the first vehicle tested in the family and is rejected, plus
(2) the probability another vehicle is selected as the first vehicle and
exceeds one or more of the thresholds (so that the given vehicle must be
tested) and the given vehicle is then tested and rejected. Assuming the
first vehicle to be tested is selected uniformly randomly from among the
N EDVs in the family, the probability the given vehicle is rejected is
Strategies could also be devised for which testing of a family is stopped
if tests on k of the n vehicles produce results less than t., for some k
less than n.
-------
199
Pr[F] • J-Pr[F] + ^Pr[£] Pr[F],
where e is the event that the first tested vehicle in the family exceeds
at least one of the thresholds t^. The probability of the event e is
computed as:
using earlier notation. The value of Pr[F] is computed as described in
section F.3.1.
F. 4 Effectiveness Computations
This section describes how the mathematical structures developed in
the previous three sections are combined to compute the values of effective-
ness measures under the various testing strategies examined. As discussed 1n
appendix E, the methodology is used to compute various average failure proba-
bilities for the EDVs submitted to EPA. The effectiveness methodology
directly computes the following probabilities, which are then used to gen-
erate the failure probabilities presented 1n appendix E:
(1) the probability an EDV from a particular family 1s failed by
the certification process,
(2) the probability the EDV's true emissions exceed the standard
for at least one pollutant,
-------
200
(3) the joint probability the vehicle is failed and exceeds one or
more standards, and
(4) the joint probability the vehicle is failed and is within all
standards.
In addition to these probabilities, estimates are made of the total
lifetime emissions of vehicles certified under most of the strategies investi-
gated. To develop these latter estimates it was necessary to adopt several
assumptions:
(1) Emission rates and deterioration rates as measured by EPA provide
a reasonable proxy for true in-use emissions of certified vehicles.
(2) The average life of a certified vehicles is 100,000 miles.
(3) The ultimate effect of a failure of an EDV is to cause the manufacture1*
to modify the vehicles in the portion of the engine family associated
with the failure in such a way that the average emission of the
pollutant on which the failure was based is reduced in these vehicles
to the mean emission level of the family as a whole. These modified
vehicles are then submitted and certified.
(4) The average fraction of an engine family modified as the result
of a failed EDV is equal to 1/N, where N is the number of EDVs in
the family.
(5) The manufacturer's projected sales for a family are an adequate
approximation of the true sales realized by the family.
-------
201
The first of these assumptions recognizes that many considerations will cause
in-use emissions and deterioration to differ from emissions and deterioration
predicted by the certification process, but asserts that changes in the predicted
values resulting from a change in the certification process would be accompanied
by a similar change in the in-use values. This is an assumption inherent in
the certification process itself. The second assumption is generally accepted
and has been used by EPA in other analyses. The third assumption is a simplified
representation of the extremely complex events which follow the failure of
an EDV. It is VRI's understanding that most failed vehicles are eventually
modified to the extent that they can be certified, and that this simplified
representation therefore reasonably reflects the ultimate impact of a failure.
The fourth assumption is consistent with the earlier assumption that each EDV
is randomly selected from its engine family. For any portion of the analysis
which does not assume vehicles to be randomly selected, corresponding estimates
of lifetime emissions have not been generated. Finally, the assumption concern-
ing projected sales is consistent with information provided verbally to VRI
by members of the EPA staff.
The following two sections describe, 1n turn, the computation of the
above probabilities and the computation of projected emissions.
F.4.1 Computation of Probabilities
Recall that the true emission levels p^ (1=1,2,3) for a given EDV
are assumed to be normally-distributed random variables with means and
-------
202
standard deviations dependent on the family from which the vehicle was
selected. To compute the probability a vehicle selected at random from
a given family is failed by the certification process under a given testing
strategy, the effectiveness methodology must use this distributional assumption
to compute the expected value of Pr[F] for that strategy.1 It does this via a
numerical integration over all possible values of the of the product of
Pr[F] and the joint probability density function of the u^. Similarly, the
joint probability that the vehicle is failed and that its true emission levels
are within the three standards is computed by means of the same numerical inte-
gration over values of the y. less than s.. The difference between the values
of these two integrations provides the joint probability that the vehicle is
failed and exceeds one or more of the standards. Finally, the simple probabili^
that the vehicle exceeds one or more of the standards is just the probability
that one or more of the normally-distributed ^ is greater than the corres-
ponding s...
F.4.2 Computation of Projected Emissions
The total tons of pollutant i emitted by all vehicles in an engine family
throughout their lives is given by
"i - G.Cv (i = l ,2,3),
2And the expected value of Pr[e] in the case of a sequential strategy.
-------
203
where
G. = average emissions in grams per mile of pollutant 1 by a vehicle
in the family during its 100,000 mile life,
C ¦ a proportionality constant to convert grams per mile to tons
per 100,000 miles, and
v ¦ projected sales of vehicles in the family
Because deterioration is assumed to be a linear function of mileage, the value
of G.| 1s given by the average emissions of the family at 50,000 miles.
That is, it is equal to the mean emission level E [u^], adjusted to account
for the reduced emissions of vehicles which have been modified as the result
of a failed EDV. Thus, the value of is approximately given by
6, - P[F] (E[U1|F] - E[Ui]),
where E[^|F], the conditional expectation of the true emission level
given a failure, can be determined by a means of a numerical Integration
similar to those used to compute failure probabilities.
-------
EXHIBIT F-l: STRUCTURE OF EFFECTIVENESS METHODOLOGY
Description of fleet
of vehicles manufac-
turers wish to produce
(emission characteris-
tics of families and of
configurations within
families)
Emissions Testing and
Certification Process
(Policy Type 1)
>Certification decisions-
<
(Policy Type 2)
(Policy Type 3)
Test vehicles
selected
/N
v
Effectiveness
Analysis
Effectiveness
of
certification
process
ro
o
.p*
Emissions
Standards
-------
205
EXHIBIT F-2: EXPRESSION FOR THE PROBABILITY A VEHICLE DOES NOT
FAIL CERTIFICATION UNDER A POLICY OF AVERAGING
ON FAILURE OF THE FIRST TEST
TERMS
CONDITIONS
P(u1)P(u2)P(u3)
+L(u-j «-v-j ,-p)L(-u2,-v2,p)L
+L(-u1,-v1,p)L(u2,-v21-p)L
+L(-u1,-v1,p)L(-u2,-v2,p)L
+L{u-j ,-vT,-p)L(u2,-v2,-p)L
^(up-Vp-pjLt-Ug.-Vg.piL
+L(-u1,-v1,p)L(u2,-v2,-p)L
+L(u-j »~v-j ,-p)L(u2,-v2,-p)L
"u3'~v3,p)
~u3'~v3,p)
u3'"v3'"p^
-u3,-v3,P)
u3.-v3,-p)
u3.-v3,-p)
U3*"V3,"P^
Note; The above terms use the facts that
,p)dbdc
L(
and
-h,-k,P) -/ J g(b,c,f
09 k
L(h,-h,-p) - J / g(b,c,p)dbdc.
h -»
(1) Pass first test
(2) Fails only pollutant 1
on first test
(3) Fails only pollutant 2
on first test
(4) Fails only pollutant 3
on first test
(5) Fails pollutants 1 and 2
on first test
(6) Fails pollutants 1 and 3
on first test
(7) Fails pollutants 2 and 3
on first test
(8) Fails all 3 pollutants
on first test
Note: In term (1), no second
test 1s conducted; 1n
terms (2)-{8), vehicle
passes standard for
6veery pollutant when
the first and second
tests are averaged.
-------
206
-------
207
APPENDIX G
DESCRIPTION OF EPA FUEL ECONOMY TESTING PROGRAM
One task of the present study was to develop an understanding of the
relationships between the fuel economy and emissions certification programs.
This task was necessary to insure that Emission Data Vehicle test strategies
developed during the course of the present study would take into account the
constraints imposed by fuel economy testing. The descriptions developed in
this task are therefore intended to show the study team's understanding of
the fuel economy process and its implications for emission vehicle testing.
For this reason, the descriptions were submitted to the EPA Fuel Economy
Unit for review to ascertain that this understanding is adequate.
The results of VRI's review of EPA's comments on the preliminary
versions of this document are incorporated 1n this appendix. It contains
a flowchart overview of the entire fuel economy process and detailed explana
tions of the steps involved in carrying it out. The appendix describes such
items as vehicle classification and selection methods, Interactions with the
emissions certification program, calculation and use of the manufacturer's
average fuel economy, the labeling program, etc.
In developing this understanding of the process, the study team inter-
viewed members of the Fuel Economy Unit of the Light Duty Certification
Branch. Since many of the policies for 1978 and subsequent years are still
in a development stage, and not fully documented, their contribution was an
essential component of the study team's review. In addition, a number of
documents were collected and reviewed, including ones on the Energy Act
[US General Accounting' Office, 1974], [US Congress, Senate, Committee of
Conference, 1975], and [US Congress, Senate, and House, 1975]; procedures
-------
208
published in the Federal Register [US Environmental Protection Agency,
1976b and 1976c]; EPA position papers [US Environmental Protection Agency,
1976f and 1976g]; MASPC Advisory Circulars [US Environmental Protection
Agency, 1976d, 1976e, and 1976h]; and miscellaneous papers on lab policy
and fuel economy factors [US Department of Transportation, and US Environ-
mental Protection Agency, 1975]; and [US Environmental Protection Agency,
1975 and 1976a]. Pertinent factors in the present fuel economy process
were derived from these documents and incorporated in the description in
this appendix.
As a result of these activities the study group concluded that the fuel
economy process does not place a severe constraint on Emission Data Vehicle
test strategies, i.e., the primary focus of this study. Further, because of
the large degree of freedom left to the Administrator in the statute and
the primacy accorded the emissions testing process, it is clear that most
changes to emissions certification procedures suggested here would have
limited effect on the fuel economy program. For example, most changes in
selection schemes for Emission Data Vehicles will have only a minor effect
in numbers of separate vehicles necessary to be tested for fuel economy
purposes.1 In fact, the number of extra vehicles required for fuel econ-
omy purposes was approximately 12 for the 1977 model year. Any change in
policies which mandated a large increase in fuel economy testing would alter
1Changes in the selection scheme may influence EPA fuel economy testing when
they eliminate EDVs which are the sole representatives of base levels
(defined later) for calculation of the average fuel economy. Unrepre-
sented base levels must have data for fuel economy data vehicles (FEDV)
submitted by manufacturers to fill that void. Present policy is that a
minimum of 25 percent of all FEDVs must be tested for confirmation of the
A selection scheme which dramatically reduces the number
of EDVs will, however, have a significant effect on the number of FEPVs
produced.
-------
209
the degree to which the suggested changes to emission testing procedure would
influence this program. This situation does not however seem imminent at
present. In general, presently foreseen policies appear not to cause any
major increase in workload for the fuel economy unit. In fact, a recently
created EPA policy of requiring manufacturers to perform the initial checks
to determine that all the specified classifications (base levels) are repre-
sented by test vehicles will have an effect of decreasing the EPA workload.
G.1 Background
The Energy Policy and Conservation Act sets standards for the average
fuel economy of passenger vehicles produced for sale in the US. These
standards begin with a minimum average of 18.0 miles per gallon for the
1978 model year and increase to a minimum of 27.5 in 1985. The Act also
sets out a civil penalty to be assessed against any manufacturer failing
to meet these standards. This penalty is five dollars per passenger auto-
mobile produced per .1 mile per gallon that the manufacturer's average
fuel economy (AFE) is below the standard for that model year. Since this
penalty could easily be several million dollars for a manufacturer, there
is good reason for attempting to meet the standards prescribed. It 1s
also a good reason to have a defensible way of determining AFE.
The Act authorizes EPA to develop in conjunction with each manufactur-
er an average fuel economy to be sent to the Secretary of Transportation
for compliance action.1 In addition, 1t imposes constraints as to the
*An additional requirement is that EPA must separate a manufacturer's output
into groupings of domestically produced and imported vehicles for calculation
of the average (i.e., there are both domestic and imported averages for some
manufacturers).
-------
210
procedures to be employed to derive these averages. Specifically, the fuel
economy testing must be done in conjunction with the emissions tests for
certification under the Clean Air Act using the procedures for the 1975 model
year (or procedures yielding comparable results). Because of the large degree
of freedom left to the Administrator, it is clear that any changes made in
the emissions certification testing process could be adapted to the fuel
economy program, as long as it remains comparable to the 1975 process.
The present procedure for selecting vehicles for testing by EPA has
been termed the "secure base level approach". In essence, this method
starts with data from the emissions certification testing program and data
supplements from the manufacturers for use in the fuel economy labeling
program (described later). These data are for calculation of an initial
average fuel economy (AFE) prior to the beginning of the model year. If a
manufacturer's average is sufficiently above the following year's standard
or sufficiently above the applicable model year's standard and he waives
any credit for the following year, then no additional test data is required,
since there 1s little chance that a penalty will be assessed or a credit
allowed (.described later). A manufacturer has the option to submit additional
data to increase the accuracy of the average, since the number of data points
is extremely small compared to a statistical sampling scheme. Additional
tests may also be necessary if a manufacturer makes mid-year changes or
additions to his product line. (Of course, this allows the manufacturer to
make design improvements to enhance his average if he initially does not
comply with the standards.)
-------
211
The following section presents VRI's understanding of the procedures
for selecting and testing vehicles and the method of calculating the average
fuel economy values. The procedures described are those in effect for the
1978 model year, with appropriate comments as to proposed changes for 1979
and succeeding years. It should be noted that the procedures for the labeling
process and the AFE process have been combined in the description, even though
they constitute separate requirements, since they rely on the same sets of
test data which are produced during the course of the year.
G.2 The Fuel Economy Testing Process
In order to comply with the intent of the energy act, EPA designed
the fuel economy process to be employed in conjunction with the emissions
certification process. That 1s, most data for fuel economy calculations
are derived directly from vehicles tested for emissions, and the program
itself 1s conducted at the Motor Vehicle Emissions Laboratory at Ann Arbor.
The following subsections describe the activities conducted by EPA in
carrying out this process, and correspond for the most part to the Items
1n exhibit G-l, the fuel economy process flowchart. This flowchart shows
the sequence of steps of the process and its reliance on sharing test
facilities and data sources with the emissions certification process (these
points of Interaction are the items on the left of the vertical dotted line).
It 1s anticipated that the volume of vehicles required for testing by
EPA will not increase dramatically as the full fuel economy process is
effected as of the 1978 model year. A possible exception to this could
result if a change proposed for 1979 and later years is Instituted. This
change wouTd redQce by 1/2 the range of each vehicle weight class constituting
-------
212
a base-level (defined later) from 500 pounds to 250 pounds, and from 250
pounds to 125 pounds for the two class sizes, which could create up to twice
as many base level vehicles. This option is only at proposal stage} however,
even if implemented, EPA does not expect it to significantly affect the fuel
economy testing workload. To maintain current workload levels using this
option one test vehicle would be used to reDresent multiple base level
classes by testing the vehicle at its true weight.
G.2.1 Manufacturer Determines Base Levels, Notifies EPA
For purposes of testing and calculation of fuel economy averages, EPA
established a system of classification called base levels,1 which are almost
the same as model types,2 except that weight is substituted for car line.3
The base level classes are delineated by those factors which have the greatest
XA base level is a unique combination of inertia weight class, basic engine,
and transmission class. Note: The significance of base levels is that
they are the means of subdividing a manufacturer's product line into smaller
groups for determining test requirements. Example: A 3,500 pound vehicle,
with a 231 cubic inch, 6 cylinder engine with 2 barrel carburetor, catalyst,
and manual transmission.
ZA model type is a unique combination of car line, basic engine, and trans-
missions class. (A basic engine is an engine of particular displacement,
number of cylinders, fuel system, and catalyst usage; a transmission class
is the type of transmission such as manual, semi-automatic, or automatic.)
Since this definition includes key items which affect fuel economy and are
used and understood by the average consumer, it is used for publishing fuel
economy information as well as for calculating a manufacturer's average
fuel economy,
3A car line is a group of vehicles within a make or car division which has a
degree of commonality in construction. Car line does not consider any level
of decor or opulence and is generally not distinguished by characteristics
such as roof line, number of doors, seats, or windows, although station
wagons are distinct car lines from sedans. Example: Buick, a division of
General Motors, lists nine car lines—Electra, Skylark, Skyhawk, Opel,
Century Wagon, LeSabre, Estate Wagon, and Riviera.
-------
213
influence on fuel economy. At least one vehicle from each base level, and
from vehicle configurations representing 90 percent of the sales within each
significant1 base level must be tested in order to calculate the fuel economy
average for a manufacturer. It is the responsibility of the manufacturer to
determine these base levels and to compare the base levels to the vehicles which
are being submitted for emissions certification pruposes (the emissions data
vehicles). For each base level not represented by an emissions data vehicle
(where only one vehicle test is required to represent that base level), the
manufacturer must test a representative vehicle configuration,2 and prepare
and submit a Fuel Economy Data Vehicle3 (FEDV) "package" for that vehicle.
This package includes vehicle specifications, mileage accumulation data,
maintenance records, and fuel economy test results.
6.2.2 Manufacturer Submits Voluntary and Required FEDVs
The FEDVs required of a manufacturer to fill a base level "slot" are
not expected to produce a great number of test cases. Overall, for all
manufacturers for the 1977 model year, only about 12 FEDV packages were
submitted to fulfill the base level rules. However, the manufacturers have
an option to and are encouraged to submit voluntary FEDVs to augment the
1A significant base level 1s one which represents one percent of total
production.
2Vehicle configuration—a unique combination of inertia weight class, basic
engine, and transmission class (which define a base level) plus engine
code, transmission configuration, and axle ratio. (Engine code denotes
a basic engine further specified for variations in carburetor, distributor,
and other key engine and emission control system components; transmission
configuration 1s a transmission class further specified by number of forward
gears.) Note: Base levels are subdivided into vehicle configurations in
order to identify individual test vehicles. Example: A 3,500 pound vehicle
with 231 cubic Inch, 6 cylinder, 2 barrel carburetor engine of engine code
4, with catalyst, 4-speed manual transmission, and 2.56 axle ratio.
3 A Fuel Econjwny Data Vehicle (FEDV) is a vehicle used for fuel economy
purposes which js not an original certification vehicle.
-------
214
minimum representation of only one test vehicle per base level. It should
be expected that these voluntary packages will be selected by the manufacturer
to enhance his calculated average fuel economy, rather than simply to improve
the accuracy of the calculation of the average,
G.2.3 EPA Reviews FEDV Package, Decides on Confirmatory Test
For each FEDV package submitted by a manufacturer, voluntary or required,
EPA performs a review to determine the acceptability of the vehicle and the
data. This review is directed to assuring the completeness of vehicle
information, mileage accumulation, and maintenance records, as well as
judging the relation of the weight, engine specifications, etc., to expected
ranges of fuel economy values. If after this analysis there is some doubt
as to the validity or accuracy of the data, a confirmatory test is required
by EPA.
In addition to the results of this review, the availability of test
facilities influences the decision as to whether to accept manufacturer
data without EPA testing. A total of approximately 200 FEDVs were submitted
for the 1977 model year, and of'these, 84 were selected for confirmatory
testing. Of these 84, 20 vehicles required the retest sequence. This testis
includes checks on emission levels, and if standards are not met, the fuel
economy data will not be used, but the emission data cannot be used against
a manufacturer for purposes of certification compliance. However, consistent
failures by a vehicle configuration to demonstrate compliance with emission
standards are referred to the mobile source enforcement division. In other
words, the emission test results are not furnished to certification for direct
use in the certification process.
-------
215
G.2.4 EPA Performs Confirmatory FE Test, Retests if Necessary
If it is decided that a confirmatory test is needed, EPA requires the
manufacturer to submit the vehicle on which the FEDV package was based for
testing. This testing, while not directly involved with certification,
nonetheless causes a certain interaction by creating an additional load on
the available test facilities.
The confirmatory test procedure involves making an initial fuel economy
test, then comparing the results to those submitted by the manufacturer.
If the initial value is within approved limits of the manufacturer's data,1
then these Initial test results (not the manufacturer's) are taken as the
final values for that FEDV. However, if the initial results are not close
to the manufacturer's, then a series of retests are performed, and the average
of these test results may be used as the value for that FEDV.
The retest criteria are the same as those used in performing tests on
EDVs, which are the main source of data for the AFE. CEDV testing is
described below. 1 These criteria involve; (.1) comparing the two city fuel
economy test bags to determine whether they are within the same range,
(2) comparing the new city and highway fuel economy results to previous EPA
and/or manufacturer results (must be within 10 percent, and (3) comparing
the ratio of the latest EPA highway fuel economy test result to the latest
city result to determine whether they are within acceptable ranges.1 If
the new results fail any one of those tests, the corresponding city or high-
way test 1s rescheduled.
1Based on a memorandum from L. I. Ranka to EPA Light Duty Vehicle teams
and Fuel Economy groups dated July 27, 1976.
-------
216
The responsibility for making these checks and scheduling retests, if
necessary, lies with the certification team for required FEDVs and with the
Fuel Economy Group for Voluntary FEDVs.1 If more than one retest is necessary
for either required or voluntary FEDVs, the Fuel Economy Group may decide to
terminate testing.2 The finally-accepted FE value (which is selected by the F£
Group) can be:
(1) the data from a single EPA test;
(2) the average of the manufacturer's tests; or
(3) the average of all the EPA tests.
The decision as to which value to accept is determined by engineering judgment
using fuel economy trend data based on such items as axle ratio, horsepower,
etc., and the history of correlation between the particular manufacturer's
and EPA's test results. The value determined to be more reasonable in light
of the trends and correlations is then established as final for that vehicle.
G.2.5 EPA Develops FE Values on All Emissions-Approved EDVs
In order to meet the guideline established by the Energy Conservation
Act to rely as much as possible on the certification program for data for
fuel economy, a policy of use of Emission Data Vehicles (EDVs)3 for genera-
tion of fuel economy data was established. That is, vehicles representing
^¦Conversations with the Fuel Economy Group indicate that of all vehicles
tested for the 1976 model year, approximately 7.5 percent required at
least one rerun for the city test, and 10 percent for the highway test.
In the period 5/1/76 - 11/27/76, the city test was rerun for 5.4 percent
of all EDVs and 23.8 percent of all FEDVs. In the same period, 6.9 per-
cent of all highway tests were rerun.
2It is VRI's understanding that EPA may request another vehicle to be
supplied by the manufacturer if test data from the initial vehicle are
erratic.
3An Emission Data Vehicle (EDV) is a 4,000-mile vehicle selected and tested
by EPA as part of the emissions certification process.
-------
217
determined base levels are chosen from the EDVs selected by the certification
process to the greatest extent possible.
Thus, all EDVs are run through the highway fuel economy test after they
complete the emissions test, which is the same as the urban fuel economy
test. If it is determined, after applying the deterioration factor, that
a vehicle passes the emissions test when the fuel economy results are
used from both the urban and highway cycles. This reliance on certification
testing and selection processes forms the greatest interface of the fuel
economy program with the certification program, Any changes in the selection
procedure for EDVs can therefore potentially impact the degree to which the
additional vehicles must be tested to cover all fuel economy base levels,
That is, any unrepresented base levels must have a fuel economy data vehicle
whose test results are submitted by the manufacturers and, depending on the
percentage of confirmatory tests required, may affect the total number of
FEDVs by EPA,
G.2.6 EPA Stores Manufacturer Data and EPA Data, Checks Reasonableness
As data from manufacturer's FEDV packages and EPA-generated test data
are assembled, a complete file is created representing all base levels,
These data are added to, corrected, deleted, etc., throughout the year,
and form the base for all initial and final calculations of the AFEs. As
each piece of data 1s added to a base level's file, it is checked for
"reasonableness." This check is basically a comparison of the submitted
values with what could be expected given the specifications of the vehicle,
and in this sense is similar to the judgment used to decide on confirmatory
testing. If the data 1s determined unreasonable, further justification or
confirmation is required.
-------
218
G .2.7 Manufacturer May Request Specific Label Values, EPA Must Approve Before
Use
The labeling process, which involves developing the fuel economy values
that must be posted on all vehicles sold in the US, is carried on as an
adjunct to the preparation of the average fuel economy for the manufacturers.
The sales-weighted values prepared in the fuel economy process for the model
types and vehicle configurations are those yalues which must be posted on new
vehicle windows, A general label is related to an entire model type average
(consisting of several vehicle configurations), whereas a specific label shows,
the average for only one vehicle configuration.
If, for whatever reason, a manufacturer decides to sell vehicles before
determination of general label values, it must request approval of specific
labels for each configuration to be sold. EPA reviews each request and makes
a determination as to whether to approve such use. It should be noted that,
regardless of when specific labels are used, all vehicle configurations within
a model type must use specific labels if any one configuration uses them.
G.2.8 Manufacturer Requests General Label Determination, Provides Initial
Sales Projections, Submits Car Line Classification Information
Manufacturers must formally request a general label value determination
for the labeling process to be initiated. At this time, approximately July-
August prior to the introduction of the new models, the initial sales pro-
jections are provided for all vehicles to be manufactured throughout the year.
In addition, each automobile manufacturer submits to EPA information
necessary for determining the appropriate car line class for each proposed
vehicle. The major determinants of these classes for passenger vehicles are
-------
219
interior volume and the number of passengers which the vehicle can transport.
Pickup truck classes are separated from each other on the basis of gross
vehicle weight rating (GVWR), and other non-passenger automobiles which are
not pickup trucks are classified as vans/special purpose trucks.
In addition to the car line information, the manufacturers submit
information on vehicle weights, engines, transmissions, use of catalytic
converters, and axle ratios which will be available in each car line. Some
of this information is used by EPA in determining model types, and the balance
for determining base levels and vehicle configurations, discussed later.
G.2.9 If Any Base Levels Are Not Represented, Manufacturer Must Submit
FEDV Package
At this point, EPA checks to see that all base levels are represented
by either an EDV or an FEDV. If any levels are not included, the manufacturers
are notified of this lack and required to submit an FEDV for each such base
level, The FEDV packages submitted as a result of this check are reviewed
for confirmatory testing in the same manner as previously discussed,
G.2.10 EPA Determines Test Results To Be Used in Fuel Economy Calculations
EPA performs a final reasonableness check in determining which test
results in the data file for each base level are to be used 1n the calcula-
tions to produce general and specific label values and Initial average fuel
economy CAFE) levels for the manufacturers.
-------
220
G.2.11 Manufacturer Submits Revised Sales Projections
The sales projections may be revised in September-October to the final
1
values to be used for calculating label values and initial AFEs.
G.2.12 EPA Calculates FE Values and Ranges
With all test results from EDVs, required and voluntary FEDVs, and
running changes made early in the year available, and any revised sales projec-
tions made, EPA calculates fuel economy values for each vehicle configuration,
base level, and model type. If enough of these values are available at this
time, ranges for the values (overall similar classes for all manufacturers)
are established also. Calculations of the fuel economy range values are
always made according to the sales-weighted, harmonic average of the urban
and highway test results.
G.2.13 Manufacturer Reviews Data, EPA Corrects
EPA allows the manufacturers a review of the calculated FE values and
of the data used to produce the values. If corrections are deemed necessary
by EPA after careful review of manufacturer's objections, these are made and
the affected FE values recalculated.
i
For compliance the actual production values are used.
-------
221
6.2.14 General Label Values Are Established, Booklet Is Published
The final general label values are calculated after the review-
correction cycle and are used for consumer information during the model
year. The values for all vehicles of all manufacturers certified by the
fall cut-off date are published in the Gas Mileage Guide which must be
made available to consumers in every dealer's showroom throughout the
country. A second guide is compiled in January to cover any late submittals.
6.2.15 Manufacturer May Elect to Use Specific Labels
If all vehicle configurations within a given model type have specific
label values available, then EPA will approve the use of these labels pro-
vided that all yehlcle configurations within the model type are labeled
concurrently. This has been a source of some controversy, since some
manufacturers asked to be allowed to use specific labels for certain
configurations and general labels for others (all within the same model
type). This policy was established to assure equity and eliminate the
possibility of misrepresentation (by using the general labels on those
configurations which had fuel economy values below the average and specific
labels on those above).
6.2.16 EPA Compares Preliminary Calculation of Average Fuel Economy to
Assurance ievei
The decision rule as to whether fuel economy data must be collected
from running change vehicles is based on a comparison of the preliminary
calculated average fuel economy with an asswanae level yet to be established
by EPA. At the least, this level will be the minimum acceptable value for AFE
-------
222
for the year following the model year being studied (i.e., the 1978 model
year value must exceed the 1979 standard of 19 MPG), or will follow the pro-
visions of G.2.18 which follows. It is expected that some safety or assurance
margin will be added to that following year's value to produce the minimum
acceptable AFE for this exemption to be granted.
If the calculated AFE is above the asswcanae level, then the manufacturer
may apply for an exemption of further testing. If it is not, then a vehicle
must be tested for fuel economy for each approved running change. Current
policy1 is that fuel economy results from a 11 EPA tests for emission levels
of running change vehicles (even though the fuel econoiny testing requirement
exemption has been granted) will be used in calculation of the final AFE.
This is possible since the standard practice is to run the highway cycle test
for every vehicle tested on which an urban cycle test is performed.
G.2.17 EPA Tests Vehicles for Approved Running Changes
If a manufacturer's preliminary AFE is below the assurance level, and a
running change is made which does not result in the addition of a base level,
then EPA must test the highest-selling vehicle configuration in each affected
base level. Regardless of whether the preliminary AFE is above or below the
assurance level, if the manufacturer adds a significant base level,2 then
vehicle configurations representing 90 percent of the projected sales of that
aEPA personnel anticipate the creation of another policy which would allow
exemption for testing running changes for the fuel econoniy (as well as
emissions) if.the proposed change is a simple modification within certain
calibration limits.
2Recall that a significant base level is one which represents one percent or
more of total production volume.
-------
223
significant base level must be tested, the preliminary AFE is recalculated
and a new determination is made as to whether running change testing is re-
quired. If this recalculation results in the AFE moving from above to below
the assurance level, then the manufacturer must submit for testing any run-
ning change vehicles which were previously exempted.
If the manufacturer adds a non-significant base level (less than one
percent of production), regardless of the preliminary AFE, then only one
vehicle representing only the highest selling configuration is tested.
However, if during the course of the year, non-significant base levels are
added that accumulate to 3 percent or more of production, then the prelimin-
ary AFE is recalculated and the running change retroactive test rule applies
if the AFE is lowered from above to below the assurance level.
This possibility is most prevalent for the large domestic manufacturers,
since their present averages are closest to the accept/reject AFE level.
If it should occur, then a heavy additional burden will be placed on the
test facilities for these required tests.
The fuel economy testing of running changes is often in conflict
with the emissions certification of running changes since the EDVs normally
selected for these tests are the "highest emitter" cases, which often are
not high-volume vehicles. Thus, if a running change 1s emissions-approved
and the EDV was not a properly representative vehicle, additional fuel
economy data must be collected to satisfy the above requirements. When the
data from any of this testing have been approved, they are added to the FE
data base.
-------
224
6.2.18 Manufacturer May Waive Credit Based on Preliminary Calculations
The statute includes credit carryover provisions such that if a
manufacturer's final AFE (including running change data) is above the
year's AFE minimum level, then the amount by which it is above that level
may be used as a credit in the following year, if in that following year
his AFE is below standard. If the manufacturer's AFE is above the applicable
year's standard by the margin, but below the next year's standard plus
assurance margin, then the manufacturer has the option of waiving any credit
for future years' AFEs in order to be relieved of the necessity for running
change testing. Waiver of this carryover credit is the final requirement
for exemption from running change testing in those cases. Thus, a manufac-
turer may waive the next year's credit if he feels that he will have no trouble
in meeting the next year's AFE requirement and wishes to avoid the cost and
inconvenience of additional testing.
G.2.19 Manufacturer Provides Year-End Production Figures, EPA Calculates
Final AFE for Manufacturer
At the end of production of the model year, each manufacturer is
required to submit final production figures for all vehicles he produces
and/or imports. These figures are used as the final update for the sales-
weighted harmonic averaging, along with updates on any fuel economy data
from voluntary and required FEDVs added since the initial calculation. The
production figures are subject to audit checks for veracity.
This final AFE is the value which must exceed the applicable standard
(the standard in the Act or a standard set by the Department of Transportation
as a result of a manufacturer's request) for that model year (taking into
-------
225
account any credits from the previous year) in order to avoid being assessed
a penalty. However, the statute provides that a "carryback" credit may also
be allowed in certain cases,, so that passage by a margin in a particular year
may produce a credit for a previous year's failing.
It should be noted that each domestic manufacturer which offers imported
vehicles (or vehicles which contain a high percentage of imported parts) have
separate AFE's calculated for both the domestic and imported vehicles (i.e.,
a domestic AFE and an imported AFE) both of which must pass the standards.
-------
EXHIBIT
6-1: FUEL ECONOMY
PROCESS AND TEST PROCEDURES
1ES1
EM TESTS VEHICLES FOR
APPftJYEO ftUMlNS OMNVS
fuel cconflwr WMCTSS
mm. swum wised (S£pr.-ocr.) s«.es projectiohs
ff? ESS11*145 f£ fM CONFiqftATIfiN. USE LEVEL.
m Noea Jm. also re urns. i*ucs. if available
*1. KVICUS OATA. EN ***S KCESSAflV COMtECHQHS
tfUEML LABEL VALUES WE ESTABLISHES: CESU.TS ME
PUBLISHES IN TIC ttKLCT
NFS. ELECT TO USE SPECIFIC LAH.ES PttVlOCO ALL
VEHICLE COOFiajaATKMS WITH!* TWE NDOCL TVK USE
SPECIFIC LABELS
EM COMPARES AFC TO
ASSURANCE LEVEL AFE. IS
AFE » ASSUftMCE LEVEL?
DOES MFR. NAIVE ANT CftCOlT
BASED OH THE PftELlNIMAT
CALCULATION?
It* ASOS DATA FROM DUNNING
CMM£ FiOft TO DATA BASE
wra HKWIOES *£AA £ttt
SALES FIGURES TO EPA
EPA CALCULATES FINAL
AFE FOR *R.
ro
ro
cn
-------
Ill
APPENDIX H
REFERENCES
[Abramowitz and Stegun, 1970]
Abramowitz, Milton, and Stegun, Irene A., eds., Handbook of Mathematical
Functions, National Bureau of Standards, November 1970.
[Box and Tiao, 1973]
Box, G.E.P., and Tiao, G.C., Bayesian Inference in Statistical
Analysis, Addison-Wesley, Reading, Massachusetts, 1973.
[Ford, 1976]
Ford Motor Company, Automotive Emissions Office, Environmental and
Safety Engineering Staff, Pocket Reference, September 1976.
[Ford, undated a]
"Appendix A-7: Total Emissions Variability," material provided
to VRI by Ford Motor Company (exact title unknown).
[Ford, undated b]
"Appendix A-8: Emissions Test Variability," material provided to
VRI by Ford Motor Company (exact title unknown).
[Jeffreys, 1961]
Jeffreys, Harold, Theory of Probability, 3rd edition, Oxford
University Press, London, England, 1961.
[Juneja et al.s 1976]
Juneja, W.K., Horchler, D.D., and Haskew, H.M., "Exhaust Emission
Test Variability," Air Pollution Measurement Accuracy as it Relates
to Regulation Compliance, Air Pollution Control Association
Specialty Conference Proceedings, 1976.
[Juneja et al.t 1977]
Juneja, W.K., Horchler, D.D., and Haskew, H.M., "A Treatise on
Exhaust Emission Test Variability," draft of a paper to be presented
at the International Conference of the Society of Automotive Engineers,
March 1977.
[Lindley, 1965]
Lindley, D.V., Introduction to Probability and Statistics from a
Bayesian Viewpoint - Part 2: Inferencet Cambridge University Press,
Cambridge, England, 1965.
-------
228
[Marzen, 1976]
Marzen, James M., "1977 Model Year Emission-Data Vehicle Selection,"
EPA Internal Memorandum dated 8 January 1976.
[Matula, 1974]
Matula, Richard A., Emissions and Fuel-Economy Test Methods and
Procedures, Consultant Report to the Committee on Motor Vehicle
Emissions, Commission on Sociotechnical Systems, National Research
Council, September 1974.
[Ostle, 1963]
Ostle, Bernard, Statist-Los in Research, Iowa State University Press,
Ames, Iowa, 1963.
[Patterson and Henein, 1972]
Patterson, D.J., and Henein, N.A., Emissions from Conbuat-ion Engines
and Their Control, Ann Arbor Science Publishers, Inc., 1972.
[Paulsell and Kruse, 1974]
Paulsell, C.D., and Kruse, R.E., "Test Variability of Emission
and Fuel Economy Measurements Using the 1975 Federal Test Procedure,"
SAE Paper No. 741035, Society of Automotive Engineers.
[Savage, 1954]
Savage, L. J., Foundations of Statistics, John Wiley & Sons, Inc.
New York, 1954 .
[Scheffe, 1963]
Scheffe, Henry, The Analysis of Variance, John Wiley & Sons, Inc.,
New York, 1963.
[Searle, 1971]
Searle, S.R., Linear Models3 John Wiley & Sons, Inc., New York, 1971*
[US Congress, Senate, Committee of Conference, 1975]
US Congress, Senate, Committee of Conference, Energy Policy
and Conservation Act: Conference Report to Accompany S. 622,
94th Congress, 1st session, S. Report 94-516. 1975.
[US Congress, Senate and House, 1975]
US Congress, Senate and House, An Act: Energy Policy and
Conservation Act, Public Law 94-1633 94th Congress, S. 622,
22 December 1975.
-------
229
[US Department of Transportation, and US Environmental Protection
Agency, 1975]
US Department of Transportation, and US Environmental Protection
Agency, Fuel Economy Teat Procedures Panel Report3 no. 6,
10 January 1975.
[US Environmental Protection Agency, 1977]
US Environmental Protection Agency, personal communication from
Janet Lane Auerbach, Staff Assistant, Mobile Source Air Pollu-
tion Control, 1977.
[US Enviromental Protection Agency, 1976a]
US Environmental Protection Agency, "Review of Fuel Economy Data
Vehicle Packages - 1877 Model Year," laboratory working paper, 1976.
[US Environmental Protection Agency, 1976b]
US Environmental Protection Agency, Part V, "Fuel Economy
Testing, Labeling and Information Disclosure Procedures and
Requirements," Federal Register's Vol. 41, No. 218,
10 November 1976.
[US Environmental Protection Agency, 1976c]
US Environmental Protection Agency, Part IV, "Fuel Economy
Testing; Calculation and Exhaust Emissions Test Procedures
for 1977-1979 Model Year Automobiles," Federal Register,
Vol. 41, No. 177, 10 September 1976.
[US Environmental Protection Agency, 1926d]
US Environmental Protection Agency, MSAPC Advisory Circular,
Supplements A/C No. 58, No. 58-1, 8 September 1976.
[US Environmental Protection Agency, ?976e]
US Environmental Protection Agency, MSAPC Advisory Circular*
A/C No. 56, H July 1976.
[US Environmental Protection Agency, 1976f]
US Environmental Protection Agency, Position Paper on Fuel
Economy Testing and Calculation Procedures, Office of Mobile
Source A1r Pollution Control, 28 June 1976.
[US.Environmental Protection Agency, T976g]
US Environmental Protection Agency, Appendix I from position
paper on statistical considerations 1n calculating the manufac-
turers-4 average fuel economy (exact title not known), 8 June 1976.
-------
230
[US Environmental Protection Agency, 1976h]
US Environmental Protection Agency, MSAPC Advisory Circular A/C No.
49A, 19 January 1976.
[US Environmental Protection Agency, 1976i]
US Environmental Protection Agency, MSAPC Advisory Circular, A/C
No. 54, 8 April 1976.
[US Environmental Protection Agency, 1976j]
US Environmental Protection Agency, Allocation of Light-Duty Certifi-
cation Manpower, EPA memorandum, February 1976.
[US Environmental Protection Agency, 1975]
US Environmental Protection Agency, Fectors Affecting Automotive
Fuel Economy: Third EPA Report, Office of Air and Waste Management
Mobile Source Air Pollution Control, Emission Control Technology
Division, September 1975.
[US Environmental Protection Agency, udated]
US Environmental Protection Agency, Office of Mobile Source Air
Pollution Control, Certification Division, F177 Program Plans,
undated.
[US General Accounting Office, 1974]
US General Accounting Office, Report to the Subcommittee on Con-
servation and Natural Resourcest Committee on Government Operations
House of Representatives, by the Controller General of the United
Stated, B-166506, 15 August 1974.
[VRI, 1976]
Anderson, R., Doyle, T. Farrell, R. and Kinley, W., An Analysis of
EPA'a Motor Vehicle Emission Certification Procedures, Report Number
VRI EPA-1 FR76-1, Vector Research, Incorporated, 30 July 1976.
[Wagner, 1969]
Wagner, Harvey M., Principles of Operations Research with Applica-
tions to Managerial Decisions, Prentice-Hall, Inc., New Jersey, 1969.
------- |