Regulatory Support Document
-Section 207(b) NPRM
by
The Emission Control Technology Division
Office of Mobile Source Air Pollution Control
United States Environmental Protection Agency
November 30,1976
-------
Table of Contents
Pag
Introduction
Summary of the 207(b) Project and Related Issues 5
r;
2. l Summary
2.1.1 Availability of Test Methods and Procedures ^
2.1.2 Good Engineering Practice °
2.1.3 Correlation with the FTP ^
2.2 Summary of Conclusions 13
2.2.1 The short tests
2.2.2 Manufacturer, vehicle, engine size cutpoints 1^
2.2.3 Real World Effects 1"
2.2.4 Analysis
2.2.5 Inspection Equipment, its Maintenance and Calibration 19
2.2.6 207(b) Effectiveness of Short Tests -0
2.2.7 Policy 21
2.3 Summary of Recommendations 22
2.3.1 The procedure for data analysis 23
2.3.2 Optional procedures for setting short test cutpoints— 23
2.3.3 The procedure for determining short test cutpoints 24
2.4 Summary of Open Technical Issues 2a
Ongoing Test Program 29
3.1 Overview 29
3.2 Tests 29
3.2.1 Federal Short Cycle 30
3.2.2 Composite New York/New Jersey Test 30
3.2.3 Federal Three Model Test 30
3'2.4 Clayton Key Mode Test 30
3.2.5 Idle Test 31
3.2.6 2500 rpm Unloaded Test — 31
3.3.-*' Instruments 31
3.4 Vehicle Testing Projects -? J~
3.4.1 The Experimental Catalyst Fleet : — 33
3.4.2 The 1974 Model Year Fleet 34
3.4.3 The Defects Test Fleet 3-|
3.4.4 Four Cities Pilot Project ¦ 35
3.4.5 The State/EPA PiloC Project — 3°
X 7
3.5 Instrument Testing
o o
Analytical Techniques
4.1 Correlation Methodology J>°
4.1.1 Regression Approach
-------
Table of Contents (cont'd)
Page
^.1.2 Ranking Approach ^
4.1.3 Contingency Table Approach ^
4.2 Applicability of the Analytical Techniques ^
4.3 Applicability of the Short Tests ^
4.4 Determination of the Short Test Cutpoints
5. Projected Emissions- Reductions ^5
5.1 Projected Emissions Redactions ^6
5.1.1 Knowns 67
5.1.2 Unknowns ^
5-1.3 Assumptions ^
5.1.4 Methodology for Computing the Effectiveness of a
207(b)/I/M Program
5.1.5 Effectiveness-Results Projections
6. Discussion of Issues and Future Work ^
6.1 Expected Technical Objections to the NPRM by
Industry and Environmentalists ^
6.2 Technical Issues Associated with a Cutpoint
Program ^6
6.3 Other Technical Issues
6.4 Policy Issues
Appendix A
-------
Section 207(b) - NPRM Support Document
1. Introduction
Section 207(b) of the Clean Air Act provides the Administrator with
authority to impose certain warranty conditions upon the motor vehicle
and engine manufacturers when he determines that (1) there are available
testing methods and procedures that can be used to determine whether in-
use vehicles are in compliance with section 202 throughout their useful
lives, (2) such testing procedures and methods are in accordance with
good engineering practices, and (3) such testing procedures (hereinafter
called 207(b) short test (ST)) are reasonably capable of being correlated
with tests conducted under section 206(a) (hereinafter called the FTP).
Once these three criteria have been fulfilled, the Administrator
can establish the procedure by regulation, and upon the availability of
testing facilities, vehicle and engine manufacturers will be required to
warrant their emission control devices or systems to the ownerS"*of these
products. The warranty under these regulations requires the'manu-
facturer to correct, without cost to the owner, any faults in the power-
plant and/or emission control systems which caused the vehicle or engine
to fail the applicable emission standards, provided that (1) the vehicle
has been operated and maintained in accordance with the instructions
furnished by the manufacturer at the time of new vehicle purchase and
-------
2
(2) the failure of the vehicle to pass the applicable emission standards
results in a penalty or sanction against its use.
The warranty provisions provide protection for vehicle owners
against bearing the costs of vehicle repairs resulting from manu-
facturer's error when that error results in penalty or other sanction to
the vehicle owners.
The determination of test availability, good engineering practice,
and correlation with the FTP have been the three major objectives of the
Office of Mobile Source Air Pollution Control (OMSAPC) 207(b) program.
The other requirements of section 207(b), including prescribing of
warranty regulation and determination of the availability of inspection
facilities, are being addressed by other EPA program offices.
Prior to FY75, sufficient data to evaluate test availability and
correlation were not available. While some vehicles were tested using
the FTP and selected short tests, the vehicles tested did not have
advanced" emission control systems- Since Section 207(b) app'lies to
future models only, it is critical that the test or tests selected as
207(b) tests continue to fulfill the correlation requirement as new
control systems are introduced.
In early FY75, a testing and analysis program was developed to
address this need. The program collected emission test data on all
types of available production vehicles over a range of operating condi-
-------
3
tions. Analytically, several methods were examined for defining correla-
tion.. The information presented and analysed in this report is based on
these studies, which have been ongoing since FY75.
The objective of this document is to provide answers to the following
broad questions and to adequately address the issues contained therein -
1. Based on all vehicle testing to date, have short tests been
identified that fulfill the three requirements for a 207(b) test?
2. How should "reasonable correlation" be defined?
3. How can the benefits of a 207(b) test be evaluated?
4. What future work would be needed to continue to support im-
plementation of Section 207(b)?
<«r.
Seer'ion 2 presents a short summary of the project along with a
>
summary of conclusions, recommendations and open issues. "Following the
conclusions, section 3 of the report discusses the design of the ongoing
test programs. Included are detailed discussions of test procedures and
instrumentation. Section 4 discusses the analytical techniques used to
interpret the data. The selection of a technique is influenced by the
definition of reasonable correlation. Section 5 addresses the
benefits associated with 207(b) implementation. The difficulty of
isolating 207(b) benefits from inspection/maintenance benefits is of
-------
key importance. The last section
objections to the NPRM as well as
issues,
4
discusses anticipated technical
unresolved technical and policy
-------
5
2. Summary of the 207(b) Project and Related Issues
This document has been prepared in support of the NPRM for the
implementation o£ Section 207(b) of the Clean Air Ace- The document
deals primarily with the technological issues involved but also identifies
key policy issues which need to be resolved.
2.1 Summary
The technological test procedures for determining emissions by
short tests and the methodologies to establish 207(b) short test cutpoints
(the numerical values used to predict whether vehicles would pass or
fail the Federal Test Procedure) have been identified and have been
determined to meet the requirements of the Clean Air Act on the basis of
data acquired through experimental programs conducted under well-controlled
laboratory conditions. The determination of specific short test cutpoints
~
obtained under real world as compared with laboratory conditions still
needs to be performed. A program to examine this issue, to be carried
out in the State of Oregon, is in the procurement phase with the proposals
having been submitted. The second week of December, 1976, is projected
for contract signing. The data obtained from this program will be
applied to the generation of future 207(b), cutpoints.
Potential obstacles to implementing 207(b) relate to the extensive
resources that may be required annually by the Agency to establish
-------
6
207(b) cutpoints for new models, and the possibility that adequate
correlation may not exist when the short tests are run under real world
conditions.
2.1.1 Availability of Test Methods and Procedures
Since the 207(b) test must be capable of being carried out at
minimal cost, the available tests are limited to short tests that are
simple and quick to perform with minimum instrumentation and equipment
requirements. The tests evaluated for availability included:
Idle test - The raw exhaust gas is measured with simple
instrumentation to determine HC and CO concentration in the raw
exhaust gases with the engine in an unloaded condition. Procedural
variations of the idle test studied included running the test with
the automatic transmission in neutral vs. drive, and idle speed at
normal RPM speed vs. high RPM with the transmission in neutral.
NOx concentration cannot be meaningfully measured at idle since N0:<
emissions are insignificant under idle conditions - an important
limitation of the idle test. The idle teat has been the predominant
inspection test procedure utilized in currently operational state
programs.
-------
7
(2) Steady State Modal Testis. (Federal 3 Mode and Clayton
Key Mode)* - The raw exhaust gas is measured with simple instru-
mentation to determine HC, CO, and NOx concentrations while the
vehicle is driven under prescribed load conditions at two different
speeds and at idle. A dynamometer is used in this test to simulate
non-varyins speed and load conditions. Procedural variations of
the tests involve the severity of the loads aud speeds simulated.
The Clayton Key Mode test is currently being evaluated by the
States of Arizona and California.
(3) Transient tests. (Federal Short Cycle and NY/NJ Composite) -
A sample of the dilute exhaust gas (a CVS unit is used) is col-
lected in a bag while the vehicle is driven on a dynamometer to
simulate a. driving cycle that includes acceleration, deceleration,
and cruise modes. In theory a large number of different transient
tests are possible, to correspond with different driving cycles;
however, in practice only two cycles have been studied: (1) a
composite of the N.J. ACID test and the N.Y. Short Test and (2) the
EPA Short Cycle developed by OMSAPC. The sample accumulated in the
bag is analyzed for HC, CO, and NOx concentration. The concentration
values measured are converted to a mass (gm/mile) basis. At present
no state is using or contemplating the use of such a test.
* The Clayton Manufacturing Company has applied for patents on the Key
Mode emissions test procedure and has stated its intent to charge fees
for use of the diagnostic information provided -with the test results-
The status of the patent and the extent to which it applies to other
steady state modal tests is unknown at this time.
-------
8
While all three types of tests are available in the laboratory
setting, the idle test and the steady state modal tests have been used
in state I/M programs and would thus be able to trigger the 207(b)
warranty as soon as it is promulgated. Because of the differences in
equipment, manpower requirements and time required to conduct steady
state modal tests over the idle test the choice between these two tests
is dependent upon the incremental improvement in accuracy and correla-
tion derived by the steady state modal tests. The transient tests
although defined and available, have never been implemented in a state
I/M setting, presumably because of the significantly greater demands
upon personnel and the sophisticated equipment needed. To the extent
that significantly greater correlation and/or benefits can be shown for
these tests when compared with simpler tests, states may choose to
implement these tests either as l/M tests or as a follow-up test cor a
vehicle failing the I/M test which is covered by the 207(b) warranty.
2.1.2 Good Engineering Practice
The criterion of good engineering practice must be resolved in
terms of whether the tests can (1) be conducted with reasonable demands
upon test personnel and equipment and (2) can yield reasonably accurate
and reproducible results when the test is performed as specified. On
the basis of the results of laboratory tests of all five of the test
procedures evaluated to date and since under reasonable quality control
conditions test result variations can be held within acceptable limits,
these five tests can be judged to conform to good engineering practice.
-------
9
OMSAPC is evaluating commercially available HC, CO, and N'Ox instru-
mentation, working with an instrument trade association to make available
calibration gases to state authorities, and consulting with dynamometer
manufacturers to assure the availability of dynamometers at reasonable
cost. The results of these efforts will augment the information currently
available for determining that a short test meets the requirements of
good engineering practice.
2.1.3 Correlation with the FTP
The requirement that the 207(b) test reasonably correlate with
the FTP test has proven to be the most difficult and controversial task
in the OMSAPC 207(b) program. A short test must be performed in a short
time on a fully-warmed up vehicle, is limited at best to a very few
vehicle speed/load conditions, and will be performed under a wide range
of environmental conditions (i.e., temperature, humidity, human factors
/
and instrument factors which are either uncontrollable or under limited
control)..^ Conversely, the FTP is performed with a cold start, includes
a very large number of vehicle speed and load conditions simulating a
typical stop and go urban commutation, and is performed undfer closely
controlled laboratory environmental conditions (temperature and humidity
are rigidly controlled and test personnel are under engineering quality
control supervision).
It was anticipated that the short tests would not correlate with
the FTP in a classical statistical sense, i.e., no short test is capable
-------
of a reliable and consistent prediction of the FTP mass emissions. The
extensive fleet tests by OMSAPC verified that the Pearson statisitcal
correlation coefficients were low (generally less than 0.8), indicating
unacceptable correlation.
However, there is consensus within EPA that the Clean Air Act
requirement of "reasonable correlation" is met if the short test is
capable of reliably and consistently predicting whether the vehicle
would pass or fail the FTP even if it can not give the magnitude of the
passing or failing margin. The reasonableness of such correlation
depends on the ratio of incorrect predictions to correct predictions.
In the case of the 207(b) test the incorrect predictions are of two
types, each with significantly different consequences. An error of
commission (i.e. the shore test incorrectly predicts failure for a
vehicle that really passes the FTP) would cause a vehicle which conforms
to all Federal emission requirements to be repaired under warranty at some
cost Fc the manufacturer. It is possible, however, that air quality
benefit may s.till be obtained from vehicles which are in conformance
with the emission standards at the time of failure; vehicles in a good
state of tune can often emit at levels substantially below the emission
standards, especially at low mileage. An error of omission (i.e. the
short test incorrectly passes a vehicle that would fail the FTP) has no
cost impact on the manufacturer but represents a lost opportunity for
air quality improvement.
-------
11
By varying the severity of the short test cutpoint (i.e., the
numerical value used to predict passing or failing on the FTP) it is
possible to reduce errors of commission to any desired level, bat always
at the cost of increasing errors of omission.
Although no specific policy guidance regarding what would be an
acceptable error of commission rate was provided to OMSAPC, the original
concensus within EPA was that a primary goal had to be the minimizing of
errors of commission, since the costs associated with such errors would
lead to strong objections to I/M programs by both manufacturers and
consumers. Conversely, it was generally recognized that errors of
omission have to be kept reasonable to avoid the impact of the I/M
program on air quality being marginal and the cost/benefit ratios being
unacceptably low. In the absence of policy guidance, OMSAPC determined
that an overall error of commission rate of no more than 5Z of the total
vehicle population would be reasonable based upon known test variability.^
Similarly, it was assumed that high omission error rates could be tolerated
as long as-' total vehicle test failure rates no less than these experienced
— Setting the short test cutpoint on the basis of a fixed error of
commission rate does not, however, guarantee that all manufacturers
of groups within a manufacturers product line are treated equally, i.e.
the same error of omission rate does not exist for all products for
a fixed error of commission rate. This situation was brought to light
with the very latest data available from the 300 car test fleet and
indicates an inequity. This inequity is discussed fully in Section VI
and alternative procedures are proposed.
-------
12
in current I/H program could be maintained (10-30%). It must be noted,
however, Chat error of commission rates of 5% based upon commissions in
the total population could result in a much higher percentage if ex-
pressed in terms of the population of vehicles which are failed by the
short test.
The major technical effort within QMSAPC for the past two years has
involved extensive fleet testing to acquire test data for the purpose o£
developing the above correlation concept¦ The fleets have involved 150
vehicles of 1974 vintage, a 50 car fleet of identical 1973 model year
catalyst-equipped prototypes, a 5 car fleet of catalyst vehicles for
It
defect testing, and 300 in-use 1975 catalyst and non catalyst vehicles
from the FY74 emission factor program. Each vehicle in each fleet was
subjected to a full spectrum of short tests as well as the FTP — .
With the exception of high altitude vehicles in Denver, testing and
analysis of all of the data from these projects has been completed.
—^ The purpose of the test and analysis program was to develop
correlation data as a Eunction of: FTP vs each short test for*.
(a) each pollutant separately and combined
(b) general population vs. individual manufacturer
(c) laboratory instruments vs. garage type instruments
(d) diff erent weight classes of vehicles
(e) different engine displacements
2/ Referred to elsewhere in this document as "the 300-car test fleet" or
"300-car test data". The actual number of vehicles selected and tested
in 4 cities exceeded 400, but approximately 100 of these were tested at
high altitude. Consequently, their test results are not included in the
analysis presented here.
-------
13
2.2 Summary of Conclusions
2.2.1 The short tests
1. No short, test provides perfect correlation with the FTP;
that is, no short test is able to consistently predict the
exact FTP emission levels for the majority of vehicles.
2. For purposes of the NPRM the agency can specify a number
of short tests and methodologies for establishing short test
cutpoints that will meet the availability, good engineering
practice, and correlation requirements of the Clean Air Act.
3. Under laboratory conditions, short tests which follow a
non-steady state driving trace (the Federal Short Cycle and
the New York/New Jersey composite test) show a higher decree
of correlation than do the steady state modal tests (the
Federal 3 Mode and the Clayton Key Mode) or the idle test.
The idle and steady state tests, however, require much simpler
and less costly equipment to implement and are substantially
less sensitive to errors made by inspection station personnel
and to small discrepancies in instrument accuracy.
-------
14
4. The most practical —'^ short test at low elevations for
hydrocarhon and carbon monoxide emissions is the id.le test
while the most practical test for oxides of nitrogen is the
high speed mode of either the Federal 3 Mode or the Clayton
Key Mode.
5. The short tests which can be considered to meet the Clean
Air Act requirements for correlation with the FTP are the
Federal Short Cycle, the Federal 3 Mode, the New York/New
Jersey composite, the Clayton Key Mode and the idle tests.
If the data trends continue as in the past, the Federal 3 Mode
and the Clayton Key Hode tests may be capable of bein>; shortened
to two mode tests, i.e. idle and high speed modes.
2.2.2 Manufacturer, vehicle, engine size outpoints
6. The limited amount of data available indicate that short
test correlation may vary with the manufacturer of the vehicle
being tested. It must therefore be concluded that the short
tests may have different response characteristics to dif-
ferences in technology than does the Federal Test Procedure.
An example of this is given below frora the 300 car fleet where
a 5% error of commission cutpoint was established based
— Practicality is judged on the relative ability of these tests to
correctly identify failing vehicles while controlling implementation
costs and minimizing the effects of testing errors. The asscs^r.ent of
» *«» vehicle's
-------
15
on the entire fleet within one engine size class. The short
test cutpoint was then used to examine manufacturer specific
error races.
Carbon Monoxide
A Comparison of the Effects on E , E . FF, PP and
c o
(E +FF)/(E +FF) of individua.1 manufacturers as
c o
a result of grouping by engine size.
Data used was from 260 CID and larger engines,
idle mode test, Ec = 5%
Manufacturer
FF
PP
E + FF
c
E + FF
o
1/
All
GM
Ford
Chrysle^
5.0
3.0
13.0
8.6
17.8
20.1
9.4
5. 2
42.1
34.4
52.1
77.8
35.1
42.5
25.5
3.4
.79
.69
1.06
1.04
Where:
E - percent of total sample which fail the short test
C but not the FTP
E - percent of total sample which fail the FTP but not the
o
short test
FF - percent of total sample which fail both the FTP and
the short test
PP - percent of total sample which pass both the FTP and the
short test
— Short test rejection ratio = (E + FF)/(E + FF) and denotes the
number of vehicles failed by the siort test,°(E + FF) , as a fraction
of the number of vehicles which failed the FTP, (E + FF). When this
ratio is greater than unity, the manufacturer is being forced to repair
more vehicles than he should.
-------
16
The conclusion which is drawn in this example is that the short
test outpoints as determined are not stringent enough for CM and
are too stringent for Ford and Chrysler.
7. Data have shown that short test emissions levels are
dependent on the vehicle and engine size. A single short test
standard for all vehicles by model year is, therefore, inappropriate
if the optimum air quality benefit is to be realized through I/M
and if all manufacturers are to be equitably treated under
Section 207(b). An example of this is given below from the 300
car fleet. HC and CO cutpoints were determined for the idle
test while NOx cutpoints were determined for the high speed mode
of the Federal 3 Mode. A 5% E level was used for each pollutant.
c
Engine Size Short Test Cutpoints for 57. E for each
Engine Size. Idle for HC & CO, High
Speed Mode for NOx
CID HC (ppm) CO (%) NOx (ppra)
£150 219 1.84 2305
151-259 213 0.71 1395
>260 151 0.27 1936
With the exception of HC cutpoints for the £150 and 151-259 CID
groups, the great disparity in cutpoints which are required is
clearly seen even when vehicles are grouped into three engine
size classes.
-------
17
8. When the conclusions given in it 6 and -H above are taken
together, it becomes apparent that the agency may have to
consider setting outpoints by examining all of the engine and
vehicle combinations which are in production in order that any
grouping of vehicles would ensure that manufacturers are
treated equitably and that the greatest improvement in air
quality occurs.
9. Short test pass/fail levels which are applicable at low
elevations for the 49 state standards will not be applicable
in all cases at high elevations — .
10. Short test pass/fail levels, although not able to predict
precise FTP levels, are still dependent upon the specified FT?
standards. As the FTP standards are changed for new model
year vehicles, comparable short test pass/fail levels must,
therefore, be introduced.
11. Conclusions 6 through 10 above point out the potential
need for the setting of yearly short test pass/fail levels,
— The exceptions could be those manufacturers who include pressure
compensating devices in their carburetors or fuel injection systems and
in their ignition ciming devices (distributors).
-------
18
which would account for changes in technology, vehicle and
engine size, and emissions standards.
2.2.3 Real World Effects
12. A greater degree of variability will occur in results
collected under real world conditions than under closely
controlled laboratory conditions. The magnitude of this
increased variability and its effect on the accuracy of short
test predictions is not presently known. The Portland project
is directed, in part, toward the resolution of this unknown.
Once the effects of increased variability have been quantified,
adjustments may have to be made in the criteria for establish-
ing the cutpoints (eg: = 4% rather than 5% or an upper
bound 90% confidence interval selected as the 207(b) cutpoint
rather than the best estimate measured value.)
2.2.4 Analysis
13. The conventional correlation analytical technique (Pearson
regression analysis) is not appropriate for the evaluation of
FTP/ST correlation or for the selection of 207(b) cutpoints.
14. An analytical technique for defining correlation, the
contingency table technique, has been developed and is totally
applicable where the data base includes a mixture of FTP
passing and failing vehicles.
-------
19
15. All samples of vehicles which have been tested Co date
have included vehicles which passed the FTP and vehicles which
failed the FTP. It is possible however, that cases will occur
where all the vehicles sampled in one group could either all
pass the FTP or all fail the FTP. A methodology for handling
these eventualities has been developed and has been verified
on a limited data set.
2.2.5 Inspection Equipment, its Maintenance and Calibration
16. Procedures will be required but have not as yet been
developed to ensure that inspection station equipment will
be of sufficient quality and will be maintained and operated
so as to provide accurate short test results.
17. Procedures have not as yet been developed to ensure that
vehicle repair businesses use accurate instrumentation, maintain
this equipment in good operating order and properly train
their personnel in the repair of vehicles.
18. Facilities are not presently available on a national
basis for implementing all-encompassing vehicle inspection
programs. Manufacturers of dynamometers and emissions measuring
equipment will require several years to produce a sufficient
quantity of equipment to meet the requirements of a nation-
wide vehicle inspection program.
-------
20
2.2.6 207(b) Effectiveness of Short Tests
19. Because of the intimate relationship between i/M and
207(b) programs, it is very difficult to separate their
respective effectivenesses. If the implementation of 207(b)
results in states or regions implementing I/M programs which
would not otherwise have been implemented or if the implementa-
tion of 207(b) results in states selecting more stringent
pass/fail cutpoints than would otherwise have been adopted,
then 207(b) implementation ".an be credited with air quality-
effectiveness. On the other hand I/M programs can be implemented
without 20;7(b) and can achieve all of the air quality benefit
independent of 207(b) implementations- However, analysis per-
formed to dace indicates that if states were to implement I/M
programs with the 207(b) cutpoints rather than the 332 failure
rate cutpoints typical of current programs, significant air
quality benefit could be obtained. The predicted effectiveness,
of an I/M program in terms of the first Five years or 50,000
miles of a vehicle's life, using the 207(b) short test pass/fail
levels as computed with a 5% error of commission rate (vehicles
grouped by engine size) is between 22.1 and 34.61 reduction
for HC, 24.6 and 42.8% reduction for CO and 0.33 and 2.0%
reduction for NOx. These predictions are based on the use of
the idle test and the assumptions as detailed in Section 5 of
this document.
-------
Comparable benefits from an l/M program where 1/3 of the
vehicles are failed by the short test, using a single cut-point
for all vehicles, are: HC, between 15.7 and 27.0%; CO,
between 16.8 and 34.1%; and NOx, between 0.34 and 1.73%.
2.2.7 Policy
20. While the basic methodology for the technical imple-
mentation of 207(b) has been identified, key policy and
institutional issues remain to be resolved. These include (a)
definition of acceptable error rates (both E and E rates) ;
c o
(b) an interpretation of what criteria for selection of cut-
points is most equitable (c) a decision on how to include
manufacturer differences in test variability—'' (d) whether
state I/M standards will be pre-empted by the federal 207(b)
cutpoints, (e.g. can a state set lower or higher l/M standards
for purposes of general vehicle compliance and still trigger
warranties?); (e) definition of vehicle owner obligations to
properly maintain and operate their vehicles (e.g. what type
— This is a complex technical issue with major implications. It
errors of commission are to be held to any given level for each
type of vehicle, then the more variable the short-test to FTP
relationship for a type of car the higher the error of omission
rate. This gives manufacturers an incentive to create random
variation of emission levels on their cars. This problem does not
exist, however, with a fixed Short Test Rejection Ratio.
-------
22
of maintenance records must be maintained); (£) impact of
207(b) on aftermarket parts and non-dealer associated service
industries and (g) potential revisions to the present 5 year
or 50,000 mile warranty requirements. Some of these issues
have been addressed by other elements in EPA.
2.3 Summary of Recommendations
1. The Agency should proceed with the issuance of an NPKM
which identifies five short tests. These short tests are the
Federal Short Cycle, the NY/KJ Composite, the Federal 3 Mode,
the Clayton Key Mode and the Idle. Any of these tests can be
adopted by a state or smaller government body for the imple-
mentation of an I/M program which can trigger section 207(b).
The state can also have the option of adopting part or parts
of either of the steady state short tests, e.g. idle mode for
measuring 1IC and CO and the high speed mode for measuring NOx,
if this simplified test meets their particular regional re-
quirements.
2. The NPRM should detail the methodology for the selection
of short test pass/fail levels for each short test. This
methodology would consist of the following segments:
-------
23
2.3.1 The procedure for data analysis
1. The contingency table analytical technique should be
applied.
2.3.2 Optional procedures for setting the short test cutpoints
1. The short test cutpoints are set at the level which
corresponds to a 5% error of commission rate (based on the
total number of vehicles tested) within each group.
2. The short test levels are set using the Short Test Re-
jection Ratio approach. The Short Test Rejection Ratio, (E _
+ FF)/(E 4- FF), would be set at unity based on the total
o
number of vehicles tested within each group.
3. The short test levels are set using a fixed level of
E /(E +FF) for each group,
c c
A. The short test levels are set so that equivalent FTP
pass/fail levels are enacted for each vehicle group, i.e. each
group is allowed to have the same negative effect on air
quality.
5. The short test levels are set so that the Ec rate or
E /(E + FF) rate is a function of test variability,
c c
-------
24
2,3.3 The procedure £or determining short test cutpoints
This procedure includes the grouping of vehicles and the setting of
as many short test pass/fail levels per pollutant as are necessary to
ensure that all vehicles are exposed to either the same error of com-
mission rate, the same short test rejection ratio or one of the other
options proposed for the selection of the short test cutpoint.
1. The Agency should, on a yearly basis, set the short test
pass/ fail levels for the most recent model year vehicles,
4nd these pass/fail levels should remain in effect through
the useful life of the vehicle. This recommendation is necessary
because of the dependence of the short test pass/fail levels
on the emissions standard to which the vehicles were manu-
factured and because these standards are changing with time.
2. The Agency should recommend the continued use by the '
states of the short test cutpoints beyond the useful life of
the vehicle.
3. The short test pass/fail levels which are developed on a
yearly basis should be established through the testing of
approximately 50 privately owned production vehicles from each
vehicle - engine group. This number is judged to be the
smallest sample size which will provide a statistically meaning-
-------
2S
ful sample of vehicles. The total number of vehicles to be
tested on a yearly basis is expected to be between 3000 and
5000, but is dependent upon the manufacturers' production mix,
the effects of altitude, and the number of state specific
standards which are in effect. Two other options, the testing
of a smaller number of vehicle - engine groups and the testing
of assembly line vehicles rather than privately owned production
vehicles are being considered to see if significant cost re-
ductions are possible without loss of acceptable correlation.
4. The agency testing program for short test pass/fail
development should include vehicles which will be inspected
at high altitude.
5. If a vehicle has to be inspected prior to the development
of its short test cutpoints (e.g. owner moves) use the previous
year's cutpoints for that vehicle, provided it is not a new
model and provided there has not been significant technology
changes between the two model years. If the vehicle is signif-
icantly different, use the cutpoints from the previous year
for the same manufacturer's product and technology but in the
next heavier weight class.
6. Small sales volume vehicles can be handled in more than one
way. If the powerplant is from a large volume manufacturer.
-------
26
then the large volume manufacturer's cutpoint at the appropriate
weight would be used. If the product is dissimilar to all
others, then the EPA will set the cutpoint based on previous
year's data and engineering judgment until a statistical
sample has been tested.
7. A manufacturer may appeal the EPA cutpoints if he can
show that his vehicle cannot meet the short test cutpoint
selected.
8. Any vehicle which can not be tested by the ST or the FTP
or which is not required to be certified (e.g. electric car)
should be exempted from section 207(b).
9. Any vehicle for which the short test does not correlate
should be exempt from Section 207(b) until a suitable short
test is developed- For this reason, the present NPRI-! excludes
Diesel powered vehicles.
2.4 Summary of Open Technical Issues
1. The exact effects of real world testing on short test to
FTP correlation is unknown. The expected effect is one of
lowering correlation, not eliminating it. The Portland project
is designed to quantify this issue for light duty vehicles.
-------
27
Completion of the Portland program can not be accomplished
before June of 1978. An additional two to three months will
be required after the completion of testing for all analyses
to be completed.
2. The agency needs to make a policy decision on equity
between manufacturers and within a manufacturer's product
line. The decision on equity centers on the use of a fixed
error of commission rate, on a fixed short test rejection
ratio, on a fixed ratio of E^/(S^+FF), on allowing each group
to have the same negative effect on air quality or on setting
E or E /(E +FF) as a function of variability. This decision
c c c
will affect the size of the yearly cutpoint program.
3. The agency needs to make a policy decision as to what are
acceptable levels of errors of commission if a fixed error of
commission rate approach is adopted. A 51 rate has been
recommended. The agency has to specify acceptable levels of
the short test rejection ratio, E^/(E^_ + FF) , and/or acceptable
variability if one of the other cutpoint selectio'n methods is
implemented.
A. The exact number of short test pass/fail levels which
must be accepted to insure uniformity between manufacturers is
not known. The number could go as high as 100 for each pollutant,
dependent upon manufacturers' product mix.
-------
28
5. Changes in technology could obsolete one or more short
tests presently proposed as acceptable with the result that
new short tests would have to be developed.
6. The exact effectiveness of a 207(b) program is impossible
to compute because it is dependent upon the existence of an
inspection and maintenance program. If a state adopts I/M
cutpoints which are less stringent than the 207(b) cutpoints,
then a change in the state cutpoints to the 207(b) cutpoints
could result in an improvement factor and 207(b) could be
credited with the improvement.
7. The proposed mathematical methodology for setting cut-
points whore all of the tested vehicles either pass or fall
the "FTP has been sucessful on one small sample of vehicles.
It is not known whether this methodology will hold up when
different vehicle test data sets are utilized.
8. A determination must be made on the acceptability of using
assembly line test fleet data rather than privately owned
production test fleet data to establish the yearly 207(b)
cutpoints. This determination will depend upon an analysis
of the green engine effect on the ST/FTP relationship along
with any possible biases in the assembly line data.
-------
29
3. Ongoing Test Program
3•1 Overview
The test program has addressed the adequacy of short test correlation
for a variety of short -tests measured with currently available instrumenta-
tion. Since the 207(b) test must be capable of being carried out in
large or small volume inspection facilities, the available tests are
limited to short tests that are simple and quick to perform with minimum
instrumentation and equipment requirements.
3.2 Tests
The tests considered fall into the three broad categories o£ Idle
Test, Steady State Modal Tests and Transient Tests as outlined in
Section 2.
The current test program has evaluated tests in each of these three
groups. Six short tests were evaluated. Four of these tests required
the use of a dynamometer. Two of the four tests which required the use
of a dynamometer also required the use of a constant volume sampling
system (CVS), analytical equipment of sufficient sensitivity to measure
the dilute sample and a driver's aid, i.e. the driver had to follow a
prescribed speed vs. time trace. The other four tests required analytical
equipment which was suitable for measuring undiluted exhaust gases. A
brief description of each short test follows. Appendix A presents
graphical representations of each short test.
-------
30
3.2.1. Federal Short Cycle: This test is a transient test and is
patterned on the Federal Test Procedure (FTP). It is 125 seconds in
duration and requires a dynamometer with inertia weights, a CVS, sensi-
tive analytical equipment and a driver's aid. Vehicle classes for road
load and inertia weight are the same as used in the FTP.
3.2.2. Composite Hew York/New Jersey Test: This test is a transient
test; it was developed by combining the New York Quick Cycle Test and
New Jersey ACID test. It is 75 seconds in duration and requires a
dynamometer with inertia weights, a CVS, sensitive analytical equipment
and a driver's aid. All vehicles are tested at a dynamometer inertia
load of 3000 lb. and a road load of 3.5 hp at 30 mph.
3.2.3. Federal Three Mode Test: This short test is a steady state
modal test and consists of three steady state modes: a 50 mph mode, a 30
mph mode and an idle in neutral mode. All vehicles are grouped intu
four weight classes with the dynamometer loading at 50 mph and 30 mph
reflecting the road load which the vehicles would experience at those
speeds when under acceleration on the FTP. The dynamometer used in this
test does not simulate vehicle inertia and is therefore much simpler
than that used in the two previous tests. Undiluted exhaust emissions
are measured in each mode. Analytical equipment used in this test does
not have to be as sensitive as that used in the two preceding tests.
3. 2.4. Clayton Key Mode Test: This three mode steady state modal test
was developed by the Clayton Manufacturing Company. It consists of a
-------
31
High Speed Mode, a Low Speed Mode and an Idle Mode (automatic trans-
missions in drive). All vehicles are grouped into three weight classes
with dynamometer loadings approximating the accelerations on the Seven
Mode Test. (The Seven. Mode Test was the FTP prior to 1972). Undiluted
exhaust emissions are measured in each mode. The dynamometer and the
analytical equipment used in this test are equivalent to those used in
the Federal Three Mode Test.
3.2.5. Idle Test: Undiluted exhaust emission measurements are made
with the vehicle engine at idle. The transmission may be in either
neutral or drive. Analytical equipment is equivalent to that used in
the Federal Three Mode and Clayton Key Mode tests. A dynamometer is not
used in this test.
3.2.6. 2500 rpm Unloaded Test: The vehicle engine is operated at
2500 rpm with the transmission in aeutral. Undiluted exhaust emissions
are measured using equipment equivalent to that used in the Federal
/
Three Mode test. A dynamometer is not used in this test.
3.3 Instruments
Two broad classes of instruments were available for use in the
vehicle testing projects. These classes of instruments are: (1) labora-
tory instruments and (2) garage instruments. The laboratory instrument
class consists of those instruments which meet the requirements detailed
in the Federal Register for instrumentation to be used in the vehicle
-------
certification process. The garage instrument class consists of those
instruments which are built and sold for use in vehicle inspection
stations aud vehicle repair facilities.
In the current test programs, laboratory class instruments were
used in the determination of FTP and short test emissions. Garage class
instruments were used in the determination of HC and CO during steady
state modal tests and idle tests only. As garage class NOx instruments
were not available, this pollutant was measured using laboratory class
instruments only.
3.4 Vehicle Testing Projects
The vehicle testing project is divided into six separate elements:
1. Fifty car experimental catalyst fleet. (completed)
~
2. One hundred and fifty, 19 74 model year production light duty
vehicles. (completed)
3. Five car (catalyst equipped), one hundred defects test project,
(completed)
4. Three hundred 1975 model year production light duty
vehicles.
-------
5. State/EPA pilot project. (In procurement phase)
6. Instrument Evaluation. (Phase I is in progress, Phase II will
be initiated with FY77 funds)
3.4.1 The Experimental Catalyst Fleet
This fleet consisted of approximately fifty 1973 model year Ford
Galaxie 4 door sedans, equipped with 400 CIU engines, 2 venturi carburetion
automatic transmissions, 3.0 to 1 rear axle, HR78xl5 tires, air condition-
ing, power steering and power brakes to which catalysts and secondary
air injection had been added. These vehicles were included in the test
project because their availability significantly preceded the availability
of production catalyst equipped vehicles. The vehicles had been in
general service in California and had accumulated mileages which ranged
between 7000 and 36,000 miles. The vehicles were loaned to EPA by the
Ford Motor Company and were tested by an EPA Contractor, Olson Laboratories
Inc. in Anaheim, California. Each vehicle was inspected by the Ford
Motor Company prior to being picked up by Olson Laboratories. The
vehicles were tested by the Federal Test Procedure, the Federal Short
Cycle, the Federal Three Mode, the High Speed Unloaded tost,.the CLayton
Key Mode and a composite of the New York and New Jersey short tests.
Continuous traces of CO, CO^, HC and NOx were mado during each FTP in
an attempt to provide a data base for future short test development.
-------
34
3.4.2 The 1974 Model Year Fleet
Olson Laboratories, Inc. in Livonia, Michigan procured under EPA
contract approximately one hundred and fifty privately owned 1974 model
year cars and delivered the cars to the EPA, Ann Arbor Laboratory Cor
testing. The vehicles were equally divided into three groups of cars.
Within each group, all cars were mechanically and technically identical.
The groups were (1) Ford Pinto Sedan or Runabout with 2.3 litre engine,
automatic transmission, 3.40 to 1 rear axle and production size tires,
(2) Plymouth Satelite, Dodge Coronet/Charger Sedans with 318 CID engine,
automatic transmission, standard 2.71 to 1 rear axle and production size
tires and (3) Chevrolet Belair, Impala or Caprice Sedans with 400 CID
engine, automatic transmission, standard 2.73 to 1 rear axle and pro-
duction size tires. These vehicles were tested for two primary reasons:
(a) to determine if vehicle weight had any significant impact on short
test correlation and (b) to simulate non-catalyst 1975 technology. The
vehicles were inspected by Olson Laboratories and their state of tune
recorded but not modified prior to delivery to EPA. The vehicles were
tested at SPA by the same tests that had been used in testing, the catalyst
prototype vehicles. Approximately twenty of the vehicles which failed
one or more of the 1974 FTP standards were set to manufacturers' specifica-
tions and retested.
3-4.3 The Defects Test Fleet
Five of the cars in the catalyst fleet were retained by Olson
Laboratories for defect testing. The procedure for incorporating a
-------
35
defect was first, to set all parameters to the manufacturer's specifications
arid then to introduce the defect or defects. Each defect was corrected
before proceeding to the next defect. This procedure ensured that only
the selectedd ef ec t(s) was in effect when the appropriate testing was
being performed. The defects which were introduced were as follows: 1)
Inoperative EGR, 2) Insufficient EGR, 3) Insufficient vacuum advance, 4)
No secondary air injection, 5) Insufficient secondary air injection, 6)
Leaks in vacuum lines, 7) Excessive fuel at idle, 8) Insufficient fuel
at idle, 9) High idle rpm, 10) Low idle rpm, 11) Over-rich main fuel
system, 12) Over-lean main fuel system, 1-3) Low fuel pump pressure, 14)
PCV valve stuck open, 15) PCV valve stuck closed, 16) Clogged air filter,
17) Excessive fuel from carburetor power circuit, 18) Insufficient fuel
from carburetor power circuit, 19) Defective intake valve, 20) Defective
exhaust valve, 21) Intermittent misfires, 22) Bridged spark plug, 23)
Reduced efficiency of the catalyst, 24) Advanced ignition timing, 25)
Retarded ignition timing, 26) Excessive centrifugal advance, 27) Insufficient
centrifugal advance, and 28) Excessive vacuum advance.
/
3.4.4 3Q0~Car Test Fleet
Three cities with different climatic conditions were selected for
testing of 1975 production vehicles. The cities were: Chicago, Houston
and Phoenix. Nominally one hundred vehicles were tested in each city.
The vehicles chosen for testing were 1975 model-year privateiy-uwned,
production cars. The vehicle mix was based on expected sales
-------
36
volumes but was modified to include all major technologies used in
meeting the 1975 emission standards, e.g., catalyst (with and without
secondary air injection), Wankel, rich thermal reactor with fuel in-
jection, etc. All vehicles were tested by the Federal Test Procedure,
the Federal Short Cycle and the Federal Three Mode.
3.4.5 The State/EPA Pilot Project
This project is presently in the procurement phase. It will be
performed in Portland, Oregon where there is mandatory vehicle inspection
and repair. The objective of the project is the determination of the
effects on correlation of variables associated with real world testing
of vehicles. Examples of real world variables are (a) human errors due
to on-the-job routine (b) ambient temperature changes (c) the length of
time that the vehicle waited in line for testing, etc. Approximately
2400, 1975/76 model year light duty vehicles will be tested by the state
inspection test (idle test), the Federal Short Cycle and the Federal
Three Mode test at the state inspection lane and by the FTP and the same
short tests at the contractor's laboratory. Approximately 86 engine-
vehicle groups will be represented in the vehicle sample. A target of
50 vehicles in each engine-vehicle group was selected to furnish data on
-------
distribution characteristics within each group. The target number of
vehicles will riot be attainable in some low sales volume groups because
of the unavailability of units. The data will be analyzed by techniques
incorporating the 207(b) methodology described in the 207(b) NPRM, I/M
cost-effectiveness methodology, and other pertinent analytical methods.
3, 5 Instrument Testing
The instrument testing evaluation consists of two parts. At the
present time EPA has contracted with Olson Laboratories to prepare and
distribute a report detailing the qualifications of approximately twenty
commerically available inspection type HC, CO, and NOx instruments. The
instruments will be evaluated in terms of accuracy, repeatability,
response time, deficiencies, and cost. Accuracy is to be determined in
laboratory tests by comparison with known standards. Hang-up time, zero
drift and other deficiencies which could reduce correlation under real
world conditions will be evaluated as will instrument durability. The
second part of this project involves an evaluation of instrument Co
instrument variability within one model line of a manufacturer's product
as well as manufacturer to manufacturer product variability. Phase two
is anticipated to start in FY77. Both of these projects will provide
valuable insight into the effect of instrument variation, on correlation.
-------
38
4. Analytical Techniques
One of the three requirements tor implementation of 207(b) is that
short test(s) can be reasonably correlated with the FTP. The Clean Air
Act did not define correlation nor did it define reasonable correlation.
Thus, two issues are apparent. First, what is the proper method for
determining correlation and second, what value of correlation is reasonable.
4.1 Correlation Methodology
The dictionary defines correlation as "a close or mutual relation"
or "the degree of relative correspondence". This definition does not
indicate a method for quantifying correlation. Statistically, three
different methodologies are associated with the concept of correlation:
regression analysis, ranking analysis and contingency table analysis.
4.1.1 , Regression Approach
Regression analysis is a technique by which one parameter of interest
is predicted by one or more other variables of interest. Thus, a func-
tional relationship is defined between the variables, and for every value
or set of values for the predictor variables there is a value for the
predicted variable. In the case of two variables, the relationship can
easily be shown graphically as is presented in Figure 1. When there is
more than one predictor or independent variable, the relationship must
be plotted in more than two-dimensional space. Figure 2 shows a case
-------
39
where there are two predictor variables. Cases with more than, two
independent variables must be treated mathematically since they cannot
be graphed in a simple form-
Regression analysis can be performed with one or more predictor
variables (independent variables) and with one or more predicted variables
(dependent variables). When there is one dependent variable and one
independent variable, the technique is usually called simple regression.
When there is one dependent variable and more than one independent
variable, the technique is called multiple regression. When there is
more than one dependent variable and any number of independent variables,
the technique is called multi-variate regression.
A regression analysis is based upon the assumption of a relationship
between the dependent and independent variables. This relationship can
take any functional form. A generalized expression is given below:
Y1 = Val W + a2 f2(xl) + ---an W + -
+ b f (x )+. . .+b f (:0 + . . .+Z £ (x )
L 1 2 nnz nnm
+...k f (x ) f. (x )
j 1 m in
The functions f may logarithms, exponentials, powers of x, or any
other functions. A special case which is often considered is simple
-------
AO
linear regression or multiple linear regression. Linear regression
assumes that the independent variables take the following form:
Y = a„ + a, x, + a„ +...+a x
0 11 2 2 n n
There are no interaction terras (such as x^ among the independent
2
variables nor are there any higher power terms {such as x^ ).
Once a functional relationship has been hypothesized, regression
theory develops estimates for the coefficients (a ^, b^. ... z..) which
minimize the error. Most regression is based upon the "least squares"
approach although other approaches could be used. Least squares is
the preferred approach since it provides estimates of the coefficients
which have the minimum possible variance. Thus, the coefficients
are selected to minimize the sum of squares of the actual points minus
the predicted values. Mathematically, this is expressed as
r* , x 2
t (y± - Vj) ¦
X
One of the measures of the quality of the regression rfelationsip
is expressed by the correlation coefficient. The correlation coefficient,
2
r , expresses the percentage of the total variation in the dependent
variable that is explained by the independent variables, assuming the
functional relationship which was hypothesized. This last point is
critical. Figure 3 sho\%is a hypothetical plot of the FTP as the
-------
41
dependent variable and a short test as the independent variable. If the
hypothesized functional relationship is linear, FTP = a + b (ST), the
correlation coefficient will be quite low. That occurs because there is
no single straight line that can be drawn through the points which does
a good job of predicting the FTP. On the other hand, if the hypothesized
2
functional relationship is quadratic, FTP = a + b(ST) + c(ST) , the
correlation coefficient will be quite high. Thus, regression analysis
evaluates correlation in terras of the hypothesized predictive relationship.
The predictive relationship must be a continuous one. For every
value of the independent variable, a specific value is predicted for the
dependent variable. The errors in the predicted values of the dependent
variable determine the correlation coefficient.
For the case of 207(b) correlation, the dependent variable is the
FTP emission for a specified pollutant. The independent variable is the
short test emission for the same pollutant. In cases where the short
test has more than one mode, there are multiple independent variables.
(It would also be possible to have independent variables which are short
test emission levels for pollutants other than the one of interest).
The disadvantage of using a regression methodology to evaluate cor-
relation is that a determination of the exact form of the functional
relationship is necessary. Also, it is equally imporuint to predict
-------
42
each and every data point of the dependent variable with the same absolute
degree of accuracy. In Figure 4, assume data points P^, P^, and are
equidistant from the curve as measured_ver tic ally. The combined lack of
fits of any of these data points will reduce the regression correlation
coefficient by the same amount. However, in considering che degree of correla-
tion between the independent and dependent variables in this case (Short Test vs.
FTP), only the lack of fit of point P2 will affect the number of vehicles
passing or failing the FTP, whereas the lack of fits of points and P3 have
no bearing on this.
4.1.2 Ranking Approach
Ranking analysis is a technique by which two variables are compared
without any assumption as to a predictable relationship between the
variables. Each variable is compared and ordered within itself. For
example, assume there are N pairs of observations (X^, Y^), (X?, Y^),
(X^, Y^j). The X values are ranked with the largest value assigned a
rank of 1; the Y values are ranked with the largest value assigned a
rank of 1. Then the ranks of each (X, Y) pair are subtracted and the
differences are summed. The rank correlation coefficient is defined as
one minus a function of the summed differences so that it will be low
when the variables are independent and high when they are correlated.
A rank correlation value near 1 indicates that two variables, when
normalized, tend to be high or low together. A rank correlation near
zero indicates that the relative rank oE one variable cannot predict the
relative rank of the second variable. A high but non-perfect rank cor-
relation, however, does not allow for an assessment of what magnitude of
errors would occur with the prediction process.
-------
43
Predictec
Variable
(FTP)
X.
Predictor Variable (Shore Test)
FIGURii 1
FTP
Short Test
Mode 2
Short Test - Modc> 1
FIGURE 2
-------
44
A
F.T.P.
>
N
* *
A
A
SHORT TEST
FIGURE 3
F.T.P.
SHORT TEST
FIGURE 4
-------
4S
4,1,3 Contingency Table Approach
Contingency table analysis assumes that there are a discrete number
of groups of individual measurements of a given dependent variable.
There are also one or more independent variables. The analysis attempts
to find one or more independent variable(s) which can be used to discriminate
between the different groups of the dependent variable. In other
words, the contingency table approach says that a predictive relationship
between the dependent and independent variables is not important.
Rather, the only thing of interest is whether the dependent variable
belongs to one of the available group categories. Figure 5 is a two-
dimensional illustration, of the concept of clustering or discrimination.
Figure 6 is a three-dimensional illustration of the same concept.
Contingency table analysis is based upon the assumption that outpoints
can be found for one or more independent variables which will discriminate
on the groups of the dependent variable. The selection of the cutpoints
results in the development of a contingency table as shown in Figure 7.
The predictot variable may be a single variable or the combination of
several independent variables.
Contingency tables are considerably simpler to deal with than
regression analyses since it is not necessary to determine the exact
predictive relationship. Rather, it is sufficient to identify the
number of important groups into which the dependent variable needs to be
-------
46
X
X
X X
X
F.T.P.
F.T.P. STD.
* X
X
X
SHORT TEST
FIGURE 5
F.T P.
f SHORT TEST-MOD!
SHORT TEST -
MODE 2 /\ /
, J /
/ s y
FIGURE 6
-------
~ 7
classified. For the purposes of 207(b) implementation, there are only
two such groups: vehicles which pass the FTP and vehicles which fail
the FTP. Thus, Figure 8 shows the 207(b) application of a contingency
table. This type of table is called a 2x2 contingency table. (A failure
on any of several independent variables would result in an overall
predictor variable failure).
Predicted Variable
Predictor Variable(s)
Group 1
Group 1
Group 2
Group 3
Group N
all
a.l2
a13
. . .
ai
In
Group 2
a21
a22
a23
2n
Group N
anl
an2
an3
a
nn
Figure 7
Predicted Variable
(FTP)
Pass
Fail
Predictor Variable (ST)
Pass
a
b
i
1
Fail
c
d
Figure 8
-------
48
The definition of correlation in a 2x2 contingency table can be
addressed in several ways (unlike regression analysis correlation or
rank correlation). First, a measure of independence between the two
variables can be determined. This indicator will fall between -1 and
+1 just like the other measures of correlation. Because of the simple
form of the predictive relationship inherent in a 2x2 table, other more
meaningful measurement quantities can be computed. These quantities
express the ability of the short test to correctly identify passing and
failing vehicles on the FTP. The following notation can be defined:
Ec = c = percent of sample which is incorrectly failed by the ST
(commission errors)
Eq = b = percent of sample which is incorrectly passed by the ST
(omission errors)
/
FF = d = percent of sample which is correctly failed by the ST
PP = a = percent of sample which is correctly passed bv the ST
Then, the following quantities can be considered in addition to the
rates listed above.
-------
49
? -|_ -pp
"c = the ratio of the cars failing the short test to the cars
Eq + FF which failed the FTP, i.e. the Short Test Rejection Ratio.
£
c = the ratio of the commission error rate to the cars failing
E + FF the ST.
c
E
¦ ¦ -• = the ratio of the commission error rate to the cars passing
fc* « r F ,
c the FTP.
E
o
——- the ratio of the omission error rate to the cars passing
o ' the ST.
FF
——+ ¦ = the fraction of the FTP failing cars which are cars
'o correctly failed by the short test, i.e. che Short
Test Effectiveness.
PP
PP + E = the fraction of the FTP passing cars which are correctly
, passed by the ST.
A selection as to which of these quantities besc expresses cor-
relation depends upon the specific application and the relatrive im-
portance of the types of errors.
4. 2 Applicability ojf the Analytical Techniques
The Senate report language associated with the 1970 Clenri Air Act
states that:
-------
5f)
The [Administrator] would be required to develop a test which
could be quickly and uniformly applied to individual vehicles
on the production line and on the road to determine whether or
not those vehicles comply or continue to comply with the
standards for which they were certified. The quick test would
have to be correlated with the pre-certification test procedure.
(Senate Report No. 91-116)
From this excerpt, it is obvious that the intent of Congress is to
determine whether or not the vehicles cocply with the standards—not
by how much they comply. Thus, the contingency table approach best
fulfills the intent of Congress in establishing whether correlatability
exists between the FTP and one or more short tests.
The standard contingency table correlation coefficient is defined
as:
(PP)(FF)-(E )(E )
£ 2 -j/2
[£PP+Eo)(PP+Ec)(FF+Eq)(FF+Ec)] 1 *
This correlation value indicates whether the two tests are related.
However, it has the disadvantage of weighting errors of omission and
errors of commission equally. Thus, two tests- which have 15-% error:? of
commission and 2% errors of omission are equally well correlated as two
tests which have 2% errors of commission and 15% errors of omission.
With respect to the implementation of 207(b), there is a difference
in importance between the two types of errors. An error of omisnion
-------
51
results in the loss of potential air quality benefits. An error of
commission results in a cost to the manufacturer (or consumer) without a
known air quality benefit. (It is not known whether vehicles whose FTP
emissions are below Federal standards could have their FTP emissions
further reduced as a result of maintenance performed with the intention
of lowering short test emissions. It is possible that these vehicles
could have their FTP emissions increased as a result of such maintenance).
The intent of 207(b) is to improve the emissions performance of vehicles
in the field without requiring consumers to pay additional penalties for
properly maintained and operated vehicles. Thus, commission errors are
extremely important since they increase the cost of 207(b) implementation
without any known benefit to air quality. Omission errors are important
to the degree that the percentage of correctly failed vehicles is lowered,
thereby reducing the air quality benefits. The limiting case occurs
where there are so many errors of omission that no significant air
quality benefit is realized.
The definitions of correlation which consider the weighting of
these error factors are E , errors of commission; E , errprs of omission;
c o
E
c , the commission error rate as a fraction of the cars failing
E + FF
c
the ST; (E + FF)/(E + FF), the Short Test Rejection Ratio; and FF/(E +•
co o
FF) , the Short Test Effectiveness. E^ must be low so that the unnecessary
costs of 207(b) implementation are as low as possible. E must be low
so that the maximum air quality benefit is realized. E^/(E^ + FF) must
-------
52
be low if the program is to minimize the cost associated with unnecessary
maintenance and obtain significant air quality benefit. (It is assumed
that E is not zero. If E is zero. FF must be large rather than E /(E
c c ' c c
+ FF) small.) (E + FF)/(E + FF) must not exceed unity. If this
c o
occurs, the manufacturer is being required to repair more vehicles than
had actually failed the FTP - an inequitable requirement. FF/(Eq 4- FF)
must be high. The higher the value of this ratio, the more effective
the short test is in correctly failing vehicles.
4.3 Applicability of short tests
Specific contingency tabLe correlation coefficients drawn from the
available data taken by the instruments which are most appropriate to
each short test are presented in Tables I, II and III for HC, CO and N0:<
respectively. Tables IV, V, VI, VII, VIII and IX present the results
from the 300 1975 model year car fleet for E , E , FF, PP, (E + FF)/(E
co c o
+ FF), E /(E + FF), E /(E + FF) and FF/(E 4- FF) based upon the assumption
c o c c o
that E is 5%. There are no specific values for these correlation
c
.
measures which arc by definition acceptable or not acceptable. It has
been recommended that either errors of commission do not exceed 5 percent
or that the short test rejection ratio (E + FF)/(E 4- FF) does not exceed
c o
unity. This is a policy decision and is subject to change. It is also
suggested that FF) be less than 15% or that FF/(Eq + FF) be
greater than 50%.
It is important to examine how well the FTP correlates with itself
on the correlation measures that arc tabled. The variability which
-------
SPECIFIC CONCLUSIONS - HYDROCARBON EMISSIONS
Ranking of Short Tests by Contingency Table Correlation Indices 6 5% Er _!/
1974 MY Fleet
Catalyst Equipped. Experimental Fleet
Defects Fleet
Short
Correlation
Short
Correlation
Short
Corral;
Test
Index
Test
Index
Test
Index
Federal Short Cycle
.77
NJ-/NY Composite
.73
Federal Short Cycle
.77
NJ/NY Composite
.73
Clayton Key Mode-Idle
.68
NJ/NY Composite
. 71
2500rpm Unloaded
.58
Federal Short Cycle
.66
Clayton Key Mode-Hi
. 71
Federal 3 Mode-Idle
.43
Federal 3 Mode-Idle
.56
Federal 3 Mode-Hi
. 71
Clayton Key Mode-Low
.32
Federal 3 Mode-Hi
.51
Federal 3 Mode-Hi
. 66
Federal 3 Mode-Low
.32
Clayton Key Mode-Hi
.51
Clayton Key Mode-Low
.65
Clayton Key Mode-lli
.30
Federal 3 Mode-Low
.51
Federal 3 Mode-Low
.65
Federal 3 Mode-Hi
.29
2500rpm Unloaded
.29
2500rpm Unloeaded
.64
Clayton Key Mode-Idle
.26
Clayton Key Mode-Idle
.44
Ui
w
1/ Laboratory Instruments for Federal Short Cycle and NY/NJ Composite, Garage instruments for all other
tests.
TABLE I
-------
SPECIFIC CONCLUSIONS - CARBON MONOXIDE EMISSIONS
1/
Ranking of Short Tests by Contingency Table Correlation Indices @ 5°» '
1974 MY Fleet
Short
Test
Federal Short Cycle
NJ/'WY Composite
Clayton Key Mode-Low
Clayton Key Mode-Hi
Federal 3 Mode-Idle
Clayton Key Mode-Idle
2500rpm Unloaded
Federal 3 Mode-Low
Federal 3 Mode-IIi
Correlation
Index
. 70
.69
.29
.28
.23
.27
.24
.24
.22
Catalyst Equipped Experimental Fleet
Short
Tests
Federal Short Cycle
NJ/NY Composite
Clayton Key Mode-Idle
Clayton Key Mode-Hi
Clayton Key Mode-Hi
Federal 3 Mode-Low
Federal 3 Mode-Lo
Federal 3 Mode-Idle
2500rpm Unloaded
Correlation
Index
Defects Fleet
Short
Test
Correlation
Index
.54
Federal Short Cycle
.71
.53
Federal 3 Mode-Idle
.63
.30
NJ/NY Composite
.58
.21
Clayton Key Mode-Idle
.56
.17
Clayton Key Mode-Hi
.23
.13
Federal 3 Mode-IIi
.22
.13
2500rpm Unloaded
.12
.13
Federal 3 Mode-Low
.06
.05
Clayton Key Mode-Low
.00
!_/ Laboratory Instruments for Federal Short Cycle and NY/NJ Composite. Garage Instruments for all other tests.
Ul
¦C-
TABLE II
-------
.SPECIFIC CONCLUSIONS-OXIDES OF NITROGEN EMISSIONS
Ranking of Short Tests by Contingency Table Correlation Indices 05% Ec —^
1974 MY Fleet
Shor t
Test
Federal 3 Mode-Hi
Federal 3 Mode-Low
Clayton Key Mode-Hi
Clayton Key Mode-Idle
Federal 3 Mode-Idle
Clayton Key Mode-Low
Federal Short Cycle
NJ/NY Composite
Correlation
Index
.28
.28
.28
,28
,26
,26
,15
.08
Catalyst Equipped Experimental Fleet
Short
Testy
Federal 3 Mode-Hi
Clayton Key Mode-Hi
Federal Short Cycle
NJ/NY Composite
2500rpm Unloaded
Clayton Key Mode-Idle
Clayton Key Mode-Lou
Federal 3 Mode-Low
Correlation
Index
.62
.53
.35
.33
.11
.10
.09
.02
Defects Fleet
Short
Test
Federal 3 Mode-Hi
Clayton Key Mode-Hi
NJ/NY Composite
Federal Short Cycle
Federal 3 Mode-Low
Federal 3 Mode-Idle
2500rpm Unloaded
Clayton Key Mode-Idle
Clayton Key Mode-Low
Correlation
Index
.75
.69
.43
.38
.16
.14
.12
.06
.02
i~n
ll Laboratory instruments for all tests
TABLE III
-------
1V + FF Ec lie ££
Comparison of Ec, l'o, FF, PP, Eo + FF, Eo + FF Ec + FF, and p;0 + FF
from 300 Car 1975 Model Year Fleet
Hydrocarbons A/
Idle Test - Automatic Transmission in Drive
Manual Transmission in Neutral
Engine
No.
Size
of
Ec+FF
Ec
Ec
FF
(CID)
Vehicles
Er
Eo
FF
PP
Eo+FF
En+FF
Er+FF
Eo+FF
>260CID
All Vehicles
151
5.0
13.3
14. G
62.1
0.60
0.15
0.26
0.44
General Motors
74
4.0
14.4
16. 3
65.3
0.66
0.13
0.20
0.53
Ford
42
3.9
20.5
R. 3
66. 7
0.43
0.13
0.31
0.30
Chrysler
29
5.7
29. 2
31.6
33.5
0.61
0.09
0.15
0.52
AMC
6
0.0
0.0
0.0
100.0
—
—
—
—
151-259CID
All Vehicles
54
5.0
13.3
8.4
67.8
0.49
0.18
0.37
0.31
General Motors
13
6.5
9.4
22.3
61.8
0.91
0.20
0.23
0. 70
Ford
13
1.0
15.1
1.0
32.9
0.12
0.06
0.06
0.06
Chrysler
7
6.8
5.6
13.5
74.1
1.06
0.36
0.33
0.71
AMC
12
5.3
27.3
5.6
61.8
0.33
0.16
0.49
0.17
Others
7
1.7
21.1
2.0
75.2
0.16
0.07
0.46
0.09
1150CID
All Vehicles
95
5.0
21.9
10.2
62.9
0.47
0.16
0.33
0.32
General Motors
12
11.3
14.4
19.5
54.3
0.91
0.33
0.37
0.57
Ford
6
3.3
13.3
6.0
77.4
0.43
0.17
0.35
0.31
Datsun
8
$. 4
30.3
18.1
46.2
0.48
0.11
0.23
0.37
Toyota
9
2.8
37.3
5.5
53.9
0.19
0.06
0.34
0.13
Volksuagon
21
4.2
23.4
13.1
59.3
0.4 7
0.11
0.24
0.36
Honda
6
2.9
2.6
5.6
83.9
1.04
0.35
0.34
C. 68
Others
33
4.2
21.9
7.3
66.1
0.40
0.14
0.35
0.26
1/ Garage Instruments
TARLE IV
-------
Ec + FF Eg Ec FF
Comparison of 2C, Eo» FF, PP> E0 + FF„ + FF Ec + FF, and E0 + F!
from 300 Car 1975 Model Year Fleet
Carbon Monoxide 1/
I260CID
All Vehicles
151
5.0
17.8
42.1
35.1
0.79
0.08
0.11
0.70
General Motors
74
3.0
20.1
34.4
42.5
0.69
0.05
0.08
0.63
Ford
42
13.0
9.4
52.1
25.5
1.06
0.21
0.20
0.85
Chrysler
9
8.6
5.2
77.8
8.4
1.04
0.10
0.10
0.94
AMC
6
0.2
0.0
0.0
99.8
—
—
1.0
—
151-259CID
All Vehicles
54
5.0
24.6
34.8
35.6
0.67
0.08
0.13
0.59
General Motors
13
7.4
14.1
50.4
28.1
0.90
0.11
0.13
0.78
Ford
13
1.5
36.2
20.4
41.9
0.39
0.03
0.07
0.36
Chrysler
7
15.3
13.6
55.5
15.6
1.02
0.22
0.22
0.80
AMC '
12
5.3
26.1
24.8
43.8
0.59
0.10
0.18
0.49
Others
7
1.0
25.0
30.7
43.3
0.57
0.02
0.03
0.55
<150CID
All Vehicles
95
5.0
26.4
18.4
50.2
0.52
0.11
0.21
0.41
General Motors
12
6.7
15. 4
64.1
13.8
0.89
0.08
0.09
0.81
Ford
6
0.1
25. 6
0.05
74.25
0.01
0.00
0.67
0.00
Datsun
8
10.9
14. 7
38.2
36.2
0.93
0.21
0.22
0.72
Toyota
9
4.7
21.6
12.6
61.1
0.51
0.14
0.27
0.37
Volkswngon
21
8.4
30.0
18.3
4 3.3
0.55
0.17
0.31
0.38
Honda
6
0.0
0.0
0.0
100.0
—
—
—
—
Others
33
2.8
26.5
15.0
55.7
0.43
0.07
0.16
0.36
1/ Garage instruments
TAlJLIi V
-------
Comparison of Ec,
E0, FF, PP
EC
+ FF
Ec.
——FF and
FF
* L. J
Eo
+ FF, Eo
+ FF, Ec
+ FF,
E0 + FF
f rom
300 Car 1975 Model
Year Fleet
Oxides of Nitrogen
Idle '
rest - .
Automatic Transmission in
Drive,
Manual Transmission in Neutral
Engine
No.
Size
of
Ec+FF
Ec
Ec
FF
(CID)
Vehicles
Ec
Eo
FF
PP
En+FF
Eo+FF
Ec+FF
Eo+FF
1260CID
All Vehicles
151
5.0
16.3
3.5
70.2
0.54
0.20
0.37
0.34
General Motors
74
4.0
15.3
6.2
74.5
0.47
0.19
0.39
0.29
Ford
42
3.0
25.4
12.8
58.8
0.41
0.08
0.19
0.33
Chrysler
29
12.2
6,3
5.8
75.7
1.49
1.01
0.68
0.48 o
AMC
6
0.0
34.9
3.3
61.8
0.09
0.00
0.00
0.09
151-259CID
All Vehicles
54
5.0
15.4
4. 6
75.0
0.48
0.25
0.52
0.23
General Motors
13
5.8
20.1
1.8
72.3
0.35
0.26
0. 76
0.08
Ford
13
2.9
9.0
4.1
84 .0
0.53
0.22
0.41
0.31
Chrysler
7
7.4
20.7
13.5
58.4
0.61
0.22
0.35
0.39
AMC
12
2.0
20.0
7.7
70.3
0.35
0.07
0.21
0.28
Others
7
4.8
7.8
3.7
83. 7
0.74
0.42
0.56
0.32
1150CID
All Vehicles
95
5.0
9.0
6.5
79.5
0.74
0.32
0.43
0.42
General Motors
12
9.9
2. 7
8.3
79.1
1.65
0.90
0.54
0.75
Ford
6
0.8
12.5
0.4
86.3
0.90
0.06
0.67
0.03
Datsun
8
2 7 >2
23.8
2.8
46.2
1.13
1.02
0.91
0.10
Toyota
9
0.34
57.0
6.6
36.06
0.11
0.01
0.05
0.10
Volkswngon
2.1
0.6
0.9
0.4
98.1
0.77
0.46
0.60
0.31
Honda
6
0.7
0.1
0.03
99.17
5.62
5.38
0.96
0.23
Others
33
2.4
9.3
2.8
85.0
0.41
0.19
0.46
0.22
1/ Laboratory instruments
TABLE VI
-------
Federal Shore Cycle
« 7* <"* " ¦* \ 2
Size
(CD)
i% 0 ,
of
Vehicles
'-'C
I'F
Kr
E o
H-
pi)
P0-rFP
V.c'^'v
!Ir v VF
260C3
All Vehicles
General Motors
Toi'c
Cii;.*ySiOV
AV:C
151
74
42
29
6
5.0
4.0
2.8
4.0
0.0
12.5
8.9
11.4
20.3
0.0
20.4
21.8
18.0
40.5
0.0
62,1
65.3
67.8
33. 2
100.0
0.77
t). 84
0. 71
0.73,.,
0.00 '
0.15
0,13
0.10
- 0.07
0.00
0.20
0,16
0.13
0,09
o.-oo
0.62
0.71
0 • 61
0.67 ^
0.00
151-259CID
All Vehicles
General Xorors .
I-'orc".
Chrysler
/j:c "
- u /•> •*-
s*. _ . » -3
54
13
13
7
12
7
5.0
1.8
6.0
1.5
1.9
4.9
14.0
13.1
5.3
14,5
16.9
0.7
13.3
18.6
10.8
4.6
21.0
9.1
67.7
66.5
77.9
79.4
60.2
85.3
0.67
0,64
1.04
0.32
0.60
1.43
0.18
0.06
0.37
0.08
0.05
0.50
0.27
0.09
0,36
0.25
o.oa
0.35
0.49
0.59
0.67
0.24
0.55
0,93
All VchicL
Gcncr^.2
Voce
-at sun
Tcyo'.a
Vol
Honda
¦:Otor:
'.or.
95
5.0
13.0
19.1
62.9
0.75
0.16
0.21
0.60
12
2.9
13.4
20.5
63.2
0.69
0.09
0.12
0.60
6
2,1
5.4
13.9
78.6
0.83
0.11
0,13
0.72
3
' 3.9
6.7
41.7
47.7
0,94
0.08
0.09
0.86
9
0.3
31.5
11.8
56.4
0.28
0.01
0.02
0.27
21
2.7
IS.9
17,7
60.7
0.56
0.07
0.13
0.48
6
4.9
7.4
0.8
86.9
0.70
0.60
0.06
0.10
33
7.2
10.3
19.4
63.1
0.90
0.24
0.27
0,65
TABLE VII
-------
• 1" • T-tr* V T,« TTp
3> I- r ,• _ -C *1
Comparison of Ec, l-0> FF, FP, Eg 4 FF, Hq + FF Ec * FF, and E0 -h FF
from 300 Car 1.975, Model Yaar Fleet
Carbon Monoxide
Federal Short Cycle
lysine
Ko.
FF
Size
of
Ee-fFF
Ec
Ec
(era)
Vehicles
Er
E0
FF
PP
Eo+Fl?
Hn+FF
Er+FF
Ho+FF
-duuLi.i/
*
All Vehicles
151
5,0
13.0
44.2
37.8
0.86
0.09
0.10
0.77
General Motors
74
5,2
12.2
3B.1
44.5
0.86
0.10
0.12
0.76
Ford
42
10.1
8.4
52.0
29.5
1.03
0.17
0.16
0.86o>
Chrysler
29
2.0
6.6
76.4
15.0
0.94U
- 0.02
0.03
0.92°
AMC
ft
0.4
o.n '
0.0
99.6
.
• 1.B0
0.00
I51-259CID
All Vehicles
54
5.0
14.6
41.0
39.4
0.8.3
0.09
0.11
0.74
General Motors .
13
0.5
20.4
44.1
35.0
0.69
0.01
0.01
0.6ts
Ford
13
8.7
6.4
38.6
46.3
1.05
0.19
0.18
0.86
Chrysler
7
2.6
30.3
38.3
23.3
0.59
0.04
0.06
0.55
,'wMC "
12
4.1
16.4
34.5
45.0
0.76
0.08
0.11
0.6b
Ot hers
7
4.6
5.6
48.4
41.4
0.98
0.09
0.09
0.90
130CID
All Vehicles
95
5.0
19.4
'25.4
50.2
0.68
0.11
0.16
0.57
General Xctors
12
0.1
36.9
40.5
22.5
0.52
0.00
0.00
0.52
Ford
6
G4>
23.8
1.3
74.4
0.07
0.00
U.00
0.07
Datsun
8
0.4*
33.1
19.8
46.7
0.33
0.01
0.02
0.37
Toyota
9
3.7
25.6
8.6
62.1
0.36
0.11
0,30
0.25
Voj.k'jv/r.^or.
21
4.9
24.5
23.4
47.2
0.59
0.10
0.17
0.49
? J A 'Vi ^
6
0.0
0.0
0.0
100.0
—
—
—
—
t .u.u.vt
Others
33
6.8
10.7
30.8
51.7
0.91
0.16
0.18
0,74
TABLE VIII
-------
Oxides of Xltrogen
Federal Short Cycle
" ? n »* '"a
IsO <
Ec
Ec-rF?
£c
A' il
Si-o
,• fj~n)
or.
Vehicles
!v~
Eo
vy
??
Eo+FF
V
~.r+FF
HoH-Fl
26GCI3
All Vchiclas
151
5.0
11.5
13.3
70,2
0.74
0.20
0.27
0.54
General Motors
74
1.1
10*. 9
10.7
77.3
0.55
D. 05
0.09
0.50
Ford
42
0.6
23.5
14.7
61.2
0.40
0.02
0.04
0.38
C i* r y 3 i. c l
29
20.9
2.1
10.0
67.0
2.55'-'
' 1.73
0.63
0.83
,u:c
6
9.0
2.4
38.9
49.7
1.16
0.22
0/19
0.94
All Vehicles
54
5.0
7.5
12.5
75.0
0.83
0.25
0.29
0.62
Gc-ieri:- Xo:ors .
13
0.5
10.8
11.0
77.7
0.53
0.02
0.04
0.50
Ford
13
6.2
8.4
4.7
80.7
0.83
0.47
0.57
0.36
Chrysler
7
5.0
6.9
27.it
60.7
0.94
0.15
0.15
0.80
i
12
13.6
1.7
26.0
58.7
1.43
0.49
0.34
0.94
w _ . i O .. •¦>
7
0.6
7.5
3.0
83.9
0.34
0.06
0.17
0.29
All Vcl.icier.
95
5.0
5.8
9.a
79.4
0.95
0.32
0.34
0.63
Giw-zvzl •••ctor2
12
3.9
2.2
8.8
85.1
1.15
0.35
0.31
0.80
r c v c
6
iy.2
5.8
7.1
67.9
2.04
1.49
0.73
0.55
.j r. c s• ¦
8
2.9
10.8
14.7
71.6
0.69
0.11
0.16
0.58
T.'vo
9
0.7
35.9
28.6
34.8
0.45
0.01
0.02
0.44
VslUs-.vT.sor.
21
1.2
1.1
0.2
97.5
1.08
0.92
0.86
0.15
6
0.0
0.1
0.0
99.9
0.00
0.00
0.00
0.00
r ••c. >• ••
33
1.7
3.8
8.8
85.7
fl.H3
0.13
6
-JO-JS-
Table ix
-------
oz
results from repeated FTP tests on the same vehicle results in a potential
for errors even if the FTP itself were the 207(b) test. A contingency
table example of this is given in Table X.
Ec and Eq from two samples of randomly generated FTP variables
The first generated variable is the assumed FTP and the second generated
variable is the assumed short test. The variables were generated assuming
a normal distribution with the mean equal to the measured FTP value for
each 1975 MY vehicle in the FY74 EFP and a coefficient of variability
of 15% for HC and 20% for CO.
E + FF
c
E
c
E
c
FF
Pollutant
E
c
E
o
FF
PP
E + FF
o
E + FF
o
E 4- FF
c
E + FF
o
HC
6
12
21
62
0. 55
0.18
0.22
0.64
CO
5
1
4 7
47
1.08
0. 10
0.10
0.98
TABLE X
Based on the model used to develop Table X, a comparison of the
results given in Tables IV, V, VII, and VIII with those in Table X
indicates that the short tests arc doing an acceptable job of categoriz-
ing vehicles into the four classes of the contingency table.
4.4 Determination of short test cutpoints
The implementation of section 207(b) will require that short test
cutpoints be determined for each vehicle size/engine/technology, hereinafter
called a group for each model year of vehicles which is produced after
207(b) final rulemaking. The cutpoints need to be determined in such a
-------
63
way as to continue to assure chat che short cescs promulgated correlate
with the FTP. Thus, the analytical methods used to define acceptable
short tests are the same analytical methods which will be used to determine
short test cutpoints; that is, the contingency table correlation technique.
The cutpoint analysis can only be performed after an acceptable
data base has been collected. The collection of this data base must
ensure the following:
a. That emissions be measured by the FTP and by each applicable
short test.
b. That emissions be measured from every group which is in
production so as to establish group to group differences.
c. That a statistically viable number of vehicles be tested in
e^ch group so as to quantify within each group the differences in
emissions caused by production tolerance stack-up.
d. That emissions be measured from vehicles which pass and from
vehicles which fail the standards.
e. That the effects caused by significant differences in altitude
must be resolvable from the data.
-------
64
It appears logical Co expect the short test pass/fail levels to
change in response to changes in the FTP emission standards and also to
be affected by modifications in technology. For the immediate future,
therefore, the Agency should plan on having a yearly program for the
collection of data from new model year vehicles. This data will be
analyzed to determine model year and group outpoints. The exact size of
such a test program is not known at the present time because it is
dependent upon the manufacturers' model mix and changes in vehicle
variability. A preliminary estimate of the program would allow for the
testing of 50 vehicles from each group in production and could be
expected to result in the testing of between 3000 and 5000 vehicles
annually. (Although a program of this size would be expensive, the
outputs of such a program, if performed under strict quality control
conditions on properly maintained vehicles, could also be used to
implement section 207(c) of the Clean Air Act). Three methods of
collecting these data are under consideration at this time. One is
based" upon additional testing during certification of new vehicles,
another involves assembly line testing and the third is based-»upon field
testing of in-use vehicles. Thtjse approaches are described in section
6.2.1.
A pilot program is being performed in Portland, Oregon on 1975/76
model vehicles. This program may allow a decision to be made on whether
fewer numbers of outpoints and, therefore, a smaller test program, could
be implemented. Issues pertaining to the implementation of a yearly
cutpoint program are discussed in section 6.
-------
65
5. Projected Emissions Reductions
The projections given in this section are estimates which are
based on very limited data, i.e. the 300 1975 model year cars tested
in three cities, and must be treated as such. The methodology used
in the development of each projection is given in detail so that
the reader can fully understand the assumptions which are required
to be made in the development of the projections.
Before going into the discussion of each projection, the question
must be addressed as to the benefits which can be credited to section
207(b) versus what benefits can be credited to an Inspection and
Maintenance program. The issuance of 207(b) short test outpoints
without a vehicle inspection and maintenance program will result
in zero reduction in vehicle emissions. An inspection and maintenance
program can exist, however, without the implementation of section 207(b)
through the selection of the 207(b) cutpoints or based on other criteria
such as fixed failure rates. If the implementation of 207(b) results in
states cfr regions implementing I/i-l programs which would not otherwise
have been implemented or if the implementation of 207(h) results in
states selecting more stringent pass/fail cutpoints than would otherwise
have been adopted, then 207(b) implementation can be credited with
obtaining air quality benefits. On the other hand, I/M programs can be
implemented without 207(b) and can achieve all of the air quality benefit
independent of 207(b) implementation.
-------
66
It was judged to be reasonable for the purposes of this document to
lump the effectiveness of I/M and 207(b) into one projection which is
predicated on the use of EPA determined short test cutpoints in viable
state or local government operated I/M programs. Cutpoints were determined
based upon a 5% error of commission rate, ft. companion analysis for an
I/M program utilizing a failure rate of approximately 33% of the vehicles
tested is also presented for comparison purposes. The 33% failure rate
corresponded to approximately a 2% rate and is typical of many existing
I/M programs. The failure rates for each pollutant were: 7% for HC,
22% for CO and 4% NOx, giving a total failure rate of 33%. The period
of time for which the effectiveness is computed corresponds to the
useful life of the vehicle as defined in the Clean Air Act. Thus, the
difference in effectiveness between the two cases provides an estimate
of potential benefits which could accrue if the 207(b) cutpoints were
adopted in an I/M program which is typical of current programs. Benefits
which can be expected to accrue through the continuation of inspections
throdghout the actual life of the vehicle should be credited solely to
I/M and are not presented in this document.
5.1 Projected Emissions Reductions
Since a multitude of factors must be considered in the computation
of emissions reductions projections for the implementation of section
207(b), these factors are grouped below into categories titled
Knowns , Unknown.5; and Assumptions,
-------
67
5,1,1 Knowns
1. Vehicle emissions tend to increase with age and Che
accumulation of raileage. This rate of increase referred to
as the deterioration factor is know for vehicles which are
operated and maintained under the vehicle certification program.
The deterioration rates for both catalyst equipped and non-catalyst
equipped vehicles are known. The deterioration rates which were
derived from the 1975 model year certification fleet (not sales
weighted) for use in this analysis were:
HC - 0.014 gm/mile/1000 miles
CO - 0.084 gw/mile/1000 miles
NOx - 0.0 gm/mile/1000 miles
2. Some data are available on the deterioration rates of in-use
'Vehicles from the Emissions Factors Programs (EFP)A. These
deterioration rates do not cover extensive periods of time for
new model years.
3. The emissions levels of in-use vehicles tend, on the average,
to be significantly higher than would be predicted if these vehicles
deteriorated at the same rate as certification vehicles and had
been in compliance at the time of manufacture.
* In-house correspondence dated August 6, 1976 and September 16, 1976 on
deterioration rates of various aged vehicles.
-------
68
*
A. The relationship between short test emissions arid FTP emissions
is known for the 300 car fleet. The emission standards to which
the vehicles were manufactured is also known.
5. Within the 300 car fleet, those cars that failed the FTP
tended to do so by a substantial margin, thereby implying a very
high rate of deterioration. The majority of those cars which
passed the FTP did so by a relatively safe margin, thereby showing
a low rate of deterioration. There is a strong indication, there-
fore, that there is a significant difference in deterioration rates
between those cars which pass the FTP and those cars which fail the
FTP.
1975 Model Year Cars Tested In Chicago in FY74 EFP
HC
co
NOx
FTP Standards
1.5 gm/mile
15.0 gm/mile
3.1 gm/raile
Average of all cars
1.13
22.47
2.41
Average of FTP
passing cars
0.76 "
9.52
2.11
Average of FTP
fai-lrins cars
2.36 "
34.03
3.79
6. Statistical tests oi both the short test data and the FTP
data from the 300 car fleet indicated that the assumption that
each exhibited log normal distribution characteristics could not
be rejected.
7. A positive correlation (Pearson regression) was observed
between HC and CO emissions of the 300 car test fleet as measured
by the FTP.
-------
69
5.1.2 Unknowns
1. The emissions levels of Che cars that were tested in this
project at the time of manufacture are not known. It is,
therefore, not possible to determine exactly the deterioration
rate of the vehicles from the time of manufacture to the time of
first inspection.
2. The overall deterioration rates of in-use cars in either an
I/M or a non I/M environment are not known. The shape of the
deterioration curve is also not known, i.e., whether the deteriora-
tion is linear or is rapid or slow immediately following maintenance.
3. The reasons why emissions from many vehicles in the EFP arc
higher than the standards to which the vehicles were manufactured
are not known. Possible reasons for the observed elevated emis-
~
sions axe:
a. The vehicle exceeded the standard at the ti,4e of manu-
facture and shipping from the production facility.
b. The vehicle was readjusted (incorrectly) by the dealer
prior to delivery to the owner.
-------
70
c. The vehicle was readjusted (incorrectly) after delivery
to the owner either by the owner, a mechanic or the dealer, ar.
a consequence of owner dissatisfaction with the vehicle's
operation (driveability, hard starting, fuel economy, etc.).
d. The vehicle deteriorated at a much higher rate t'aan would
be predicted by the certification process either because of
defective parts or methods of operation, e.g., use of leaded
fuel in a catalyst equipped vehicle.
4. The effectiveness of maintenance performed by the service
industry, with respect to reducing emissions under real world
conditions, is not known. Some laboratory data are available but
these are rather incomplete.
5. The period of effectiveness of maintenance (i.e., the point in
mileage accumulation following maintenance at which the vehicle
emissions have returned to their levels before maintenance) is
unknown.
6. With the implementation of section 207(b), it is not known
whether the manufacturers will attempt to incorporate a larger
margin of safety between the emissions standards and their design
goals.
7. The frequency of inspection which will be adopted by each
state is unknown.
-------
71
5.1.3 Assumptions
General Assumptions
The premise used in the development of assumptions was to err on
the side of producing low program effectiveness predictions rather than
high program effectiveness predictions.
1. Because the 300, 1975 model year fleet does not contain a
sufficient number of identical vehicle/engine/technology specific
vehicles, vehicles are grouped into three engine sixes for purposes
of analysis as well as into a single fleet. The engine size group-
ings are: (1) small, less than 150 CID, and mainly 4 cylinder
engines; (2) medium, 151 through 259 CID, and mainly 6 cylinder
engines; and (3) large, greater than 260 CID engines.
2. Vehicle inspections are equally distributed throughout
/
the year and as a consequence, each vehicle in the population
receives its inspection when it is approximately 1 year old, 2
years old, etc.
3. The rate of vehicle mileage accumulation does not vary
significantly from the rates reported in the Department of Trans-
portation Vehicle Usage Survey and as a consequence, vehicle
inspections will occur when vehicles have accumulated 17,500 miles
(1 year of age), 33,600 miles (2 years of age) and 46,300 miles (3
years of age).
-------
72
4. Vehicles which are not in a 207(b)/Inspection and Maintenance
program basically exhibit; two deterioration rates based on their
classification at the first inspection. One deterioration rate
applies to PP and Ec vehicles, i.e., low emitters, and is the same
as that determined in the certification process. The second deteriora-
tion rate applies- to high emitters, FF and Eq vehicles, and is
represented by the levels as determined in the EFP. (Figure 9)
5. The deterioration rates derived from the 1968 through 1974 MY
cars in the EFP are applicable to -newer model year cars which fail
the FTP, i.e. catalyst equipped vehicles which fail the FTP. The
deterioration rates which were developed from the EFP data and
which were used in this analysis were:
HC - 0.072 giu/mile/1000 miles
CO - 0.80 gm/mile/1000 miles
NOx 0.0 gm/mile/1000 miles
6.-'t Deterioration rates are assumed to be linear with mileage
between any two points on the mileage-accumulation-versus-emissions
plot. This assumption is used irrespective of the assumed period
of effectiveness of maintenance.
7. Three periods for the effectivenss of maintenance are assumed
for cars that are FF vehicles at the first inspection: 6 months,
-------
73
9 months and 12 months, and Che effectiveness of a 207(b)/lnspection
and Maintenance program is computed for each case. These effectiveness
of maintenance periods become applicable to Eq vehicles once they
become FF vehicles.
8. The effectiveness of maintenance in an inspection program is
such that the emissions immediately following inspection and
maintenance are assumed to just meet the emissions standards for
the pollutant that the vehicle failed. Because of the positive
correlation which was observed between HC and CO emissions (Known
/'7), three methods for computing the level of the short test passing
pollutant, following maintenance, were employed.
These methods were:
a. That the pollutant level was unaffected by the maintenance
performed to correct the other pollutant.
b. That the pollutant level was affected by the maintenance,
and was reduced by the same percentage as was the pollutant
that caused the vehicle to fail the short test.
c. That a lower limiting value was applied to (b) above
where no short test passing pollutant was reduced to less
thaai one half of the stautory standard.
-------
74
9. A second set of assumptions W2s also used in the computation
of the effectiveness of maintenance. This assumption sets the
FTP level following maintenance at the level which is projected
to correspond to the short test cutpoint. There is assumed to
be no effect on the short test passing pollutant as a result of
maintenance, i.e. the sane as case 8.a., above.
Specific Assumptions for PP. FF, E and F. Vehicles
o c
10. The correctly passing vehicles, i.e., PP vehicles, are assumed
to perform similarly to certification vehicles and the certification
deterioration factor is assumed applicable to these cars. (Figure
10) .
11. Error of commission vehicles, E vehicles, are assumed to be
c
tuned after inspection so that their emissions just correspond to
the standards, i.e., their emissions are increased. Because these
vehicles are assumed to be inherently "good" vehicles, the cercifica-
tion deterioration factor is applied following maintenance (Figura
11).
12. Error of omission vehicles, Eq vehicles, because they are
truly high emitters but do not receive forced maintenance, are
assumed to deteriorate at the same rate as vehicles which are
-------
73
DETERIORATION CHARACTERISTICS OF VEHICLES IN A NQM I/M ENVIRONMENT
1 Time(years) 2 3
17500 Miles 33600 46800 50000
FIGURE 9
DETERIORATION CHARACTERISTICS OF VEHICLES IN A 207(b)/l/H ENVIRONMENT
VEHICLES WHICH PASS THE FIRST INSPECTION
I .
r
! I ' : i . !
| Certification. Q^t:eriqrajtipn Rar6_^ort all
-TT"" . ' V" ¦ , "I ¦ : ' f
PP vehicles^"¦ r r* ;
iliihiiniD ffii
Tt?
'.yehicle id>i .overj the.iFjTP' Jsfe'to'ridi i6^pe :tic>n[:jIf fjriled;" iy~ rr [
- short: xe^C; folljows-'-sctl-id-lipe-. it-rdt»es
0toliows .dashticL [line.: .f...
17500
Time (years) 2
Miles 33600
50000
FIGURE 10
-------
76
DETERIORATION CHARACTERISTICS OF VEHICLES IN A 2Q7(b)/I/M ENVIRONMENT
ERROR OF COMMISSION VEHICLES AT FIRST INSPECTION
J ;
. !
; i '
¦ ; Cercif idation iDetdcioirdtiipn : Ra'te t., '
!¦¦ ¦¦-;-]¦ 4 M rlH ! r j-i-Hi -H !-I i j-H--
Ll-.l-iJ ¦[ i'.".! lit " I iOasnp^..tlji
• I • • r
±;uxt
. \ -
i i
j.T
I
iT
Lt: 'isk^'d.'hjikeLJ iatiicaties pajtk; falUpweldi ..i 1 J
fir | ! \U 1--}. f-1« | •
, :i : ; , r rl! ' I : V i II ^ H i i I i I i • ! - rf-TT :
i LL p
-Ai |.L r^-'r
|tni;|qi[ipw«|oj.'
-i-1.1 I.! -i.Lj.i-1—
i
50000
FIGURE II
DETERIORATION CHARACTERISTICS OF VEHICLES IN A 2Q7(b)/I/M ENVIRONMENT
ERROR OF OMISSION VEHICLES AT FIRST INSPECTION
17 500 mileage 33600 46800 50000
TTTfMi,.'L' 1 O
-------
77
riot in an I/M program, i.e., at a race determined from the EFP.
The emissions level from which this deterioration begins is Chat
which existed at the time of inspection (Figure 12).
13. Correctly failed vehicles, FF vehicles, are assumed to be
repaired so as just to bring them into compliance with the amissions
standards or standards set in assumption //9. Because the primary
reason for their not being in compliance in the first case is not
known, it is assumed that their emissions levels will tend to
increase more rapidly than the fleet average following maintenance,
and three models are applied to these vehicles so as to bracket the
potential eventualities. The three deterioration models which are
applied to FF vehicles are:
a. That these vehicles deteriorate to their emissions
levels which existed prior to inspection in 12 months (one
inspection period). (Figure 13)
b. That these vehicles deteriorate to their emissions levels
which existed prior to inspection in 9 months an4 then continue
to deteriorate at a rate which is equal to vehicles in a non-
I/M program (the EFP Rate) until the next inspection. (Figure
13)
c. That these vehicles deteriorate to their emisssions
levels which existed prior to inspection in 6 months and then
continue to deteriorate at a rate which is equal to vehicles
-------
/ o
DETERIORATION CHARACTERISTICS OF VEHICLES IN A 207(b)/I/M ENVIRONMENT
VEHICLES WHICH FAIL THE FIRST INSPECTION
FIGURE 13
-------
79
in a non-I/K program (the EFP rate) until the next inspection.
(Figure 13).
5.1.4 Methodology for Computing the Effectiveness of a 207(b)/
Inspection and Maintenance Program.
The following procedure was utilized in the generation of the results
given in section 5.1.5.
1. Since the idle test (normal idle speed) is currently being
selected by most states and regions for in-use vehicle inspections
for HC and CO, and as the high speed mode of the Federal 3 mode is
the most adaptable test for NOx, the projected program effectiveness
was computed on the basis of these tests.
2. Vehicles within the 300 car sample were grouped according
to- engine size. The size groupings followed those specified
in section 5.1.3, number 1.
3. The 300 car data set was used to establish emissions distri-
bution characteristics as measured by both the short tests and by
the FTP. Distribution characteristics were established based
on the total fleet and for each of the three groups within the fleet.
In all cases, the distribution characteristics were determined to
be log normal.
-------
80
4. The short test pass/fail levels were determined from the 300
car test fleet data using a 5% error of commission rate. Short
test cutpoints were determined for the whole fleet and for each
group within the fleet. Also, a set of cut points were determined
for the I/M case where a 33% overall failure rate was modeled.
5. A 3000 car fleet was modeled based on the distribution charac-
teristics of the 300 car test fleet. This fleet contained the same
relative number of small, intermediate and large engine sizes as
was contained in the test fleet. This modeling was necessary to
guard against a distortion in results which would have occurred
with the small number of vehicles (300) in the test fleet.
6. The emissions as measured by the FTP from each vehicle in the
3000 vehicle model was tracked from its first inspection point,
i.e., 17,500 miles, to its 50,000 mile point.
7. The total mass of emissions emitted by each vehicle from its
firs-tr'inspection through 50,000 miles was computed using the ap-
propriate deterioration characteristic for that vehicle as shown in
Figures 10 through 13, i.e., the area under each curve was integrated
to determine the total mass of emissions emitted. This period was
used in the computation of program effectiveness because the vehicle's
emissions will not be affected by the program prior to the first
inspection. A bivariate log-normal model for distribution was used
to establish the short test emissions levels for each vehicle at
its second and third inspections. The original, within-group,
-------
81
short test outpoints were applied to each vehicle to establish its
new classification in the contingency table at the second and third
inspections. Because of the assumptions on vehicle deterioration
characteristics, the categories into which each vehicle can be
placed in any following inspection is dependent upon the categoriza-
tion of a vehicle in the previous insepction. These possible new
categories, together with the results on emissions with time, are
given below and are depicted in Figures 10 through 13.
a. If the vehicle was classified as a correctly passing
(PP) vehicle in the previous inspection, then it can be
classified as either a PP, FF, E or an error of com-
o
mission (E ) vehicle in the following Inspection.
b. If the vehicle was classified as correctly failing (FF)
in a previous inspection, then it can only be classified as
either an FF or an error of omission (E ) vehicle in the next
inspection. That is, since the FTP emissions are assumed to
just meet standards after vehicle repair, once a/vehicle fails
the FTP, it will always fail the FTP, although it will not
necessarily always fail the short test,
c. If the vehicle was classified as an error of commission
(Ec) vehicle in a previous inspection, then it can only
be classified as an FF or an E vehicle in the next inspection.
-------
82
d. If Che vehicle was an error of omission (E ) vehicle
o
in the previous inspection, then it can only be classified as
an FF or an E, vehicle,
o
8. The total mass of emissions from each vehicle in a non 207(b)
/I/M environment was computed by classing the vehicles as cither
FTP passing or failing and deteriorating the vehicles as shown in
Figure 9.
9. The effectiveness, expressed as a percentage reduction in
emissions for each pollutant of a 207(b)/I/M program, was determined
for each pollutant by determining the total mass of emissions fol-
lowing the first inspection which would be emitted by the fleet
under study in both the non-inspection and inspection cases and
expressing the reduction which occurred with inspection as a
percentage of the non-inspection case, i.e.,
/
207( b ) /1/n (Total emissions without inspection)-(Total emissions with Inspection)
Effective- = (Total emissions without inspection)
ness
5.1.5 Effeetiveness-Results Projections
The resulting projections for the reductions in light duty
vehicle emissions associated with an I/M program which uses 207(b) cutpoints
are given below.
-------
83
Case a. The level of the short test passing pollutant was un-
affected by maintenance Co correct the short test failing pollutant
and maintenance lowered the level of the failing pollutant to the
FTP standards.
Maintenance
Effectiveness
Period
(Months)
Number of
years since
first in-
spection and
final mileage
Estimated Overall
Program Effectiveness in
%. Cumulative not yearly
HC
CO
NOx
12
1
22.7
29. 7
2.05
2
31.6
35.9
1.81
50,000 miles
34.6
39.1
2.00
9
1
19.5
24.2
1.25
2
26. 7
29.4
1.04
50,000 miles
30.0
33.0
1. 29
6
1
14.6
17. 3
0.46
2
19.3
21. 2
0. 22
50,000 miles
23. 3
25. 1
0. 50*
TABLE XI
-------
34
Case b. The level of the short test passing pollutant was un-
affected by maintenance and the FTP value of the failing pollutant was
corrected to correspond to the short test outpoint.*
Maintenance
Effectiveness
Period
(Months)
Number of
years since
first in-
spection and
final mileage
Estimated Overall
Program Effectiveness in
%. Cumulative not yearly.
HC
CO
NOx
12
1
21.1
29.0
1.78
2
30.0
35.2
1.55
50,000 miles
32.9
38.3
1. 70
9
1
18.4
23.6
1.05
2
25.4
28.9
0.82
50,000 miles
28. 5
37.3
1.05
6
1
13.8
17.0
0. 32
2
18.9
20. B
0.08
50,000 miles
22.1
24.6
0.33
TABLE XII
" The FTP values which correspond to the short test values and
which were used in this model are:
Engine Size
HC
CO
NOx
(CID)
(gm/mile)
(gm/mile)
(gm/mile)
<150
I. 57
18.08
3.10
151-259
1.63
16.66
3.10
>260
1.81
15.0
3.26
TABLE XIII
-------
35
Case c. The level of the short test passing pollutant was
affected by maintenance and was reduced by the same percentage as
was the failing pollutant as a result o£ maintenance. The failing
pollutant was lowered to the FTP Standard.
Maintenance
Ef fectiveness
Period
(Months)
Number of
years since
first in-
spection and
final mileage
Estimated Overall
Pro&ratn Effectiveness in
%. Cumulative not yearly.
HC
CO
NOx
12
1
22.7
32.6
2.05
2
31.6
39. 2
1.81
50,000 miles
34.6
42.8
2.00
9
1
19. 5
26.4
1.25
2
26. 7
32.1
1.04
50,000 miles
30.0
36.4
1.29
6
1
14.6
1.8.8
0.46
2
19.8
23.2
0.22
50,000 miles
23.3
27.9
0.50
TABLE XIV
-------
86
Case d. The level of Che short test passing pollutant was affected
fay maintenance and was reduced by the same percentage as the failing
pollutant except a lower bound for improvement of one half of the statutory
standard was imposed on the short test passing pollutant. The failing
pollutant was lowered to the FTP standard.
Maintenance
Effectiveness
Period
(Months)
Number of
years since
first in-
spection and
final mileage
Estimated Overall
Program Effectiveness in
Z. Cumulative not yearly.
HC
CO
NOx
12
1
22.7
32.4
2.05
2
31.6
39.0
1.81
50,000 miles
34.6
42.5
2.00
9
1
19. 5
26.2
1. 25
2
26. 7
31.9
1.04
50,000 miles
30.0
36.1
I. 29
6
1
14.6
18.7
0.46
2
19.8
23.0
0.22
50,000 miles
23. 3
27.6
0. 5(1
TABLE XV
The short test outpoints which were used in each of the above projections
were based on a 5% error of commission rate for each engine size group.
These cutpoints are given below:
Engine Group
Short test outpoints
(CID)
HC (ppm)
C0%
• NOx (ppm)
<150
219
1.84
2805
151-259
213
0.71
1395
>260
151
0.27
1986
TART.F. XVT
-------
87
The resales of Che companion analyses which used a fixed overall failure rate
of approximately 33% and grouped all vehicles into the same class* is
given below. Results from case (a) in this group compare to results from
case (a) in Table XI where the 207(b) cutpoints are used, etc.
Case a.
Maintenance
Effectiveness
Period
(Months)
Number of
years since
first in-
spection and
final mileage
Estimated Overall
Program Effectiveness in
%. Cumulative not yearly-
tic
CO
NOx
12
1
16.4
22.2
1.59
2
24.0
27.7
1.58
50,000 miles
27.0
30.8
1.78
9
1
14.1
18.0
1.04
2
20.1
22. 5
1.03
50.000 miles
23.2
25.9
1.28
6
1
10.5
12.8
o
o
2
00
16.2
0.45,
50,000 miles
1S.1
19.7
0.73
7
TABLE XVII
* This approach was taken because most existing I/M prograirs
group all vehicles into one class per model year.
-------
88
Case b,
Maintenance
Effectiveness
Period
(Months)
Number of
years since
first in-
spection and
Estimated Overall
Program Effectiveness in
%. Cumulative not yearly,
final mileage
HC
CO
NOx
12
• 1
14.3
18.4
1.04
2 •
21.2
23.6
0.99
50,000 miles
23.7
26.1
1.13
9
1
12.5
15.1
0.63
2
17.9
19.3
0.58
50,000 miles
20.3
22.0
0.76
6
1
9.4
10.9
0.23
2
13. 3
14.0
0.15
50,000 miles
15.7
16.8
0.34
TABLE XVIII
The FTP values which corresponded to
were used in this model are*
/
the short
test values
and vtv
Engine
(CID)
Size HC
(gm/mile)
CO
(gm/mile
NOx
) (gm/mile)
<150
1. 71
17.66
3.
10
/
•151-
250 1.84
21.82
3.
79
'
>260
2.69
32.51
4.
06
TABLE XIX
* Note the significant differences in FTP values used in these
two case comparisons as shewn in Tables XIII and XIX:
-------
89
Case c.
Maintenance
Effectiveness
Period
(Months)
Number of
years since
first in-
spection and
final mileage
Estimated Overall
Program Effectiveness in
%. Cumulative not yearly.
HC
CO
NO
12
1
16. 4
2.15
1.59
2
24.0
30.5
1.58
50,000 miles
27.0
34.1
1.78
9
1
14.1
19.6
1.04
2
20.1
24.7
1.03
50,000 miles
23.2
28.8
1.28
6
1
10.5
13.9
0.50
2
14.8
17.8
0.45
50,000 miles
18.1
22.1
0.73
TABLE XX
~
-------
90
Case d.
Maintenace
Ef fectiveness
Period
(Months)
Number of
years since
first in-
spection and
final mileage
Estimated Overall
Program Effectiveness in
%. Cumulative not yearly
HC
CO
NOx
12
1
16.4
24.3
1.59
2
24.0
30.3
1.58
50,000 miles
27.0
33.8
1. 78
9
1
14.1
19.5
1.04
2
20.1
24.5
1.03
50,000 miles
23.2
28.5
1.28
6
1
10.5
13.8
0.50
2
14.8
17.6
0.4")
50,000 miles
18.1
21.8
0. 73
TABLE XXI
The short test outpoints which were used in each of the cases where all vehic
were treated as a single group and which produced approximately a 332
failure rate for all pollutant are given below:
Vehicle Class
Short Test Cutpoiwts'"
HC (ppm) CO (%) NOx (pom)
All
269
1.71
2S33
TABLE XXII
The substantial differences in cutpoints between Tables XVI
and XXII must be noted.
-------
91
6. Discussion of Issues and Future Work
6.1 Expected Technical Objections to the NPRM by Industry and
Environmentalists.
6.1.1 Objection: The data upon which the NPRM are based were all
collected under laboratory conditions and do not, therefore, include the
effects of real world variables.
Response: This is true and the agency is proceeding with t^e
collection of data under real world conditions to evaluate the effects.
The short test pass/fail levels will be determined by a set error of
commission rate, a set short test rejection ratio, a set short: test
effectiveness value and a set air quality level in the Portland study.
As the short test data which will be collected at the state inspection
station will be under real world conditions, and as the same dnta will
be collected in the laboratory, the magnitude of the real world effects
~
will be documented. The magnitude of this factor xvill be incLuded in
future programs for setting short test cutpoints.
6.1.2 Ob.jaction: New vehicle prices will increase because manufacturers
will have to include the anticipated Section 207(b) warranty costs.
Response: Those manufacturers whose products experience very
low failure rates because of good manufacturing and design practices
will experience little, if any, cost increases in their products. Con-
-------
92
versely, those manufacturers whose products experience high failure
rates could find themselves at a competitive disadvantage because of
higher product prices. They can correct this problem by improving
product quality control.
6.1,3 Objection: Two of the five tests which the agency is proposing
will require relatively expensive dynamometers which must be capable of
simulating vehicle inertia loads atid sensitive and costly emissions
measuring instruments. Much of the repair industry may not be finan-
cially capable of purchasing this test equipment so as to ensure that
effective repairs have been performed.
Response: These two tests are included because they exhibit a
higher level of correlation than do the other tests and the states must
be given the opportunity of adopting these tests if they so choose. The
agency does not, however, expect that any state will adopt either of
these tests because of the reasons given in the objection.
6.1.A Election: The quality of instruments used in vehicle inspection
as well as maintenance facilities will critically affect the accuracy of
the short test results and, therefore, the true error rates. How does
the agency propose to handle this problem?
Response: The agency has an ongoing instrument evaluation
program which is designed to identify the better "garage" type instruments
-------
93
which are presently on Che market. This program will be expanded during
FY77 so as to evaluate the degree of quality control which exists in the
manufacture of instruments.
The National Bureau of Standards is in a position to furnish
standard gases to any state or organization which wishes to maintain the
accuracy of its emissions measuring equipment.
The agency has encouraged the states to implement vehicle
inspection and mechanic training programs as part of their implementation
plans.
Prior to the implementation of Section 207(b), the agency will
in conjunction with the states, instrument manufacturers, calibration
gas suppliers, the vehicle manufacturers and any ocher appropriate
organizations, develop guide lines and approved practices to ensure that
accurate results are obtained in vehicle inspection stations and vehicle
repair facilities.
6-1-5 Objection: The agency uses the concept of a deterioration
factor in the vehicle certification process. IJhy hasn't a similar
factor been adopted in Section 207(b)?
Response: The Clean Air Act requires that vehicles be in
compliance with the emissions standards for which they were designed and
manufactured for their useful Life. As the purpose of testing of in-uso
-------
94
vehicles is to determine whether or not the vehicles are in compliance
at the time of testing, and as the short test pass/Eail level is based on
data obtained from vehicles which passed the FTP and from vehicles which
failed the FTP at various mileage accumulation levels, there is no need
for the inclusion of a deterioration factor in the short test pass/fail
level.
In addition, there is no data to show that short test to FTP
correlation is affected by mileage.
6.1.6. Obj ection: The agency is proposing more than one methodology
for the selection of short test outpoints but has only provided data
based on the fixed 5% error of commission rate methodology.
Response: The agency's initial position on this issue was
to attempt to minimize errors of commission so as to control warranty
costs while accepting a less than desirable improvement in air quality.
It was 51so the agency's judgment that a fixed error of commission rate
would be equitable between manufacturers.
Very recently-obtained data has shown that the fixed error of
commission concept is not truely equitable between manufacturers because
it allows varying error of omission rates between manufacturers. The
latest data is being analyzed by the alternative methodologies so as to
improve equitability and to evaluate the potential for raising the
improvement in air quality which results from Section 207(b). The
results from these latest calculations will be published as soon as they
become available. Results are expected by February, 1977.
-------
6.1.7 Objection: The fixed error of commission rate is not equitable
between manufacturers and does not give the maximum attainable air quality
benefit associated with the implementation, of section 207(b).
Response: This is true in that all manufacturers do not have
an equal error of omission rate and some manufacturers are allowed to
repair a lesser percentage of vehicles than others- Additionally, this
concept could cause a manufacturer to repair a greater number of vehicles
failed by the short test than he should be required to because
short test failures can exceed FTP failures. For this reason, the other
approaches are proposed and the comments on all proposed methodologies
will be considered prior to final rule making.
6.1.8 Objection: Setting the short test rejection ratio (E + FF)/
(E + FF) equal to unity will increase the number of errors of commission
and thereby the manufacturers' warranty costs.
Response: This is the only short test selection methodology
which ensures true equitability between manufacturers, i.e. the short
test cannot fail more vehicles than fail the FTP and every manufacturer
is called upon to repair only the maximum number of vehicles which is
his share. As this method will usually set lower short test cutpoints
than other methods, it is expected to provide the greatest improvement
in air quality attainable with the short test selected.
-------
96
6.1.9 Objection: The following ratios, E /(E + FF), Ec/(Ec + PP) ,
E / (E + PP), FF/(£ + FF) and PP/(PP + E ) which can be considered for
o o o c
setting the short test cutpoint all offer potential inequities between
manufacturers and should not be adopted.
Response: This argument is true as is the fixed error o£
commission approach. They are being offered, however, in the NPRJ1 so as
to obtain comments, both pro and con.
6* ^ Technical Issues Associated with A Cutpoint Program
6,2.1 What types of vehicles should be tested in a cutpoint program?
Response; The cutpoint analysis requires a data base which
includes the following:
a. that emissions be measured by the FTP and by each appli-
cable short test
b. that emissions be measured from every group wliich is in
production so as to establish group to group differences
that a statistically viable number of vehicles be tested
in each group so as to quantify within each group the
differences in emissions caused by production tolerances
-------
97
d. that emissions be measured from vehicles which pass and
from vehicles which fail to meet the standard (FTP)
e. chat the effects caused by significant differences in
altitude be resolvable from the data
f. that cutpoint differences caused by significant
differences in standards be resolvable from the data
Three methods are available for the collection of data. The firsL~
is through the addition of the appropriate short tests to the vehicle
certification procedure. The second method is by the collection of
short test data at the end of the assembly line. The third is through
the collection of data from relatively new, production, light-duty
vehicles.
The first method, that of collecting short test data during the
certification process meets requirements a, h, e, and f indicated above.
This method does not meet requirements d and c, because:
(1) the vehicles tested are prototype, not production unite,
and do not account for the effects of production tolerances
(2) even if the vehicles were production units, the number tested
is insufficient to provide a statistically viable sample for the
-------
establishment of the relative effects of production tolerances on
short tests and the FTP
(3) by definition, all of the vehicles passing the certification
process meet the emissions standards
The second method, end of assembly line testing, does not meet
requirements a or e- Requirement a can be met, however, if FTP and
additional short test testing is introduced at the assembly line. There
does not appear to be a method for meeting requirement e, (i.e. high
altitude effects) with this method because most (if not all) assembly
plants are at relatively low elevations. Additionally, this method
introduces one problem which does not apply to either of the other
methods. The problem is that of the new powertrain ("green powertrain")
effect. If this method is adopted, an additional correlation step will
be introduced, i.e., between zero mileage powerplants and broken-in in-
use powerplants. Presently, there are limited statistically-viable data
available for quantifying the relationship.
The third method, that of testing fleets of current model year,
production vehicles by the FTP and the appropriate short tests meets all
of the requirements for acceptability given above. The major weakness
of this approach lies in its expense and its inability to provide short
test pass/fail levels at the start of the model year. In practice, the
short test pass/fnil levels will not be available until almost the end
of the model year. This schedule can be accelerated somewhat by multiple
-------
99
test sites, but the earliest practical data for the availabilitv of the
data is toward the end of the third quarter of the model year. This
schedule is not anticipated to be a problem since new vehicles are not
normally required to be inspected until they are one year old.
6.2.2 Should different types of vehicles have different pass/fail
cutpoints and if so, on what basis are different outpoints determined?
Response: The purpose of a 207(b) test is to fail all vehicles
whose FTP emissions exceed the Federal standards. Data on different
groups of 1974 and 1975 model year vehicles clearly indicate that the
relationship between the FTP and the short test is group dependent.
This can be observed in Tables IV through IX and illustrated graphically
in Figure 14.
F-p
^TP Std
T
Short Test
Figure 14
Thus, if all vehicles whose FTP emissions are above standards are to be
failed with a short test, different short test pass/fail cutpoints will
be needed.
-------
100
The second question concerns the methodology by which short test
cutpoints are selected for different vehicle classes to ensure equit-
ability. The correlation analysis to date has selected short test
cutpoints for each group o£ vehicles studied by setting a five percent
limit on the errors of commission. This method has several advantages.
First, it is easy to implement. Second, each vehicle group manufacturer
will be incorrectly penalized for the same fraction of his sales of that
vehicle as every other vehicle group manufacturer. There are some
disadvantages to this method. This occurs when the number of vehicle
groups is greater than one. First, the mean FTP emissions failed in
vehicle group 1 may be very different from the mean FTP emissions failed
in vehicle group 2 even though both short test pass/fail cutpoints were
set to ensure 5% commission errors. Second, vehicle group 1 may have a
total true failure rate of X% while vehicle group 2 has a total true
failure rate of 2XZ. Thus, the manufacturer of vehicle group 1 is not
credited for having produced vehicles which are in better compliance
with the Federal standards. Third, assume that vehicle group 1 has
t
extremely high short test variability. Then, it is likely that due to
short tesJt variability, cars with low FTP emissions will have high short
/
test levels. This situation will result in a higher short test pass/fail
cutpoint than for a low test variability class, since high test vari-
ability increases commission errors and omission errors. Thus, a manu-
facturer could be rewarded for high short test variability. The same
situation could occur with high FTP variability.
-------
101
Several other options are available and are detailed h&low.
Option 1: Set the short test rejection ratio, (E^ + FF)/(E + FF),
equal to unity.
This option is no raore difficult to implement than the fixed error
of commission approach, it treats all manufacturers equitably and does
not give any advantage to manufacturers who produce vehicles with high
FTP and/or ST variability. Rather, it encourages improvements in quality
control because the manufacturer is never allowed to repair fewer vehicles
than he should, i.e., true FTP failing vehicles. This approach does not
permit the setting of a single ST cutpoint for all vehicles. This is
true for all equitable approaches for setting cutpoints and therefore
cannot be treated as a negative characteristic of the method. Air
quality benefits will tend to improve from the original approach because
the error of omission rate will be lowered. However, mean FTP emissions
of fueled vehicles in two different groups could still be significantly
different. Many manufacturers will have to repair more error of com-
mission vehicles than under the original concept. Also, it would still
be possible that some manufacturers would not be credited with producing
vehicles which were more in conformance with Federal standards than
vehicles of other manufacturers.
Option 2: Set a fixed level of E^/(E^ + FF) for each group.
This option is more difficult to implement since' E /(E + FF) can
c c
be small when FF is large or when E is small. Thus, there is more than
c
-------
102
one way to optimize the function. This option does, however, have the
advantage of crediting manufacturers of vehicle classes which have low
FTP failure rates. The option does not alter the variability considera-
tions of the existing approach.
Option 3: Set the short test outpoints so that equivalent FTP
pass/fail levels are enacted for each vehicle group, i.e., each group is
allowed to have the same negative effect on air quality.
This option would work in conjunction with a fixed acceptable
commission error rate. "For all groups of vehicles, the five percent
error of commission cutpoinC is computed using the contingency table
technique. Then, the FTP levels equivalent to the short test cutpoints
are computed using linear regression estimates with the FTP as the in-
dependent variable and the short test as the dependent variable. A
sample calculation is shown below where there are 3 groups of vehicles
stratified by engine displacement.
Engine S ixe
(CID)
FTP levels equivalent to short test
cutpoint 0 5% E
HC
gin/mile
CO
gro/roile
NOx
gm/mile
<150
151-259
>260
1.57
1.63
1.81
18.08
16.66
15.00
3.10
3.10
3.26
-------
103
In order to treat each group of vehicles equally from an air quality
standpoint, the highest FTP levels would be selected as the equivalent
FTP levels at the short test standards. In the example above, these
levels would be HC - 1.81 gm/mile, CO = 18.08 gut/mile, and NOx = 3.26
gin/mile, Using regression techniques, a set of short test outpoints
equivalent to these FTP levels would be determined for each vehicle group.
The commission error rates would then be adjusted for those group/pollutant
categories that originally had FTP equivalent levels lower than the worst
case levels.
This option credits classes with low FTP emissions and/or low
variability. It does lose some potential air quality benefit over the
straight five percent error of commission case. However, this option
lends itself to a methodology for including the concern over high manu-
facturer test variability. The FTP and short test variability could be
determined for each group. Variability could be defined as the ratio of
the standard deviation to the mean of the short test levels within a
/
narrow range of FTP levels. EPA could then specify an acceptable vari-
ability. ..yOnly those groups with acceptable variability would be used to
establish the five percent error of commission cutpoints, the equivalent
FTP levels, the FTP worst case cutpoint, and the final short'test cutpoints.
Groups with high variability might be penalized and have greater than 5
percent commission errors when they have a cutpoint equivalent to the
worst case FTP level. In the NPRM, manufacturers should be asked to
comment on acceptable variability.
-------
104
Option 4: Set an error of commission rate or E /(E + FF) rate as
—e — C C
a function of test variability. Allow a five percent error of commission
rate when test variability is at the maximum acceptable level-
This option eliminates the variability problem in Option 2 and the
existing approach. A functional relationship between errors of commission
or E^/(FF+E^) and variability could be developed using mathematical
simulation techniques. This could be accomplished by taking a set of
data, and using probability distribution theory and random numbers,
variability could be introduced into the' data base at different levels.
Then, using fixed short test outpoints, commission errors can be computed
and plotted against variability.
The definition of short test variability would be the ratio of
the standard deviation to the mean of short test levels within a narrow
range of FTP levels. Analyses of the variability of existing data will
be undertaken and the NPRM will solicit comments on acceptable variability.
The--^jtions given above have to be analyzed in order tcx determine
how different the projected air quality and error rate results are as
well as how difficult the methods are to implement. A detailed analysis
will require the completion of 207(b) testing in Portland. However, a
preliminary analysis based on the 300 car test data can be undertaken
with the analysis focusing on three different groups. (Groups will be
based on engine displacement.) These preliminary results will be
available at the time of final rulemaking and along with comments from
manufacturers, should provide the basis for a final rulemaking decision.
-------
105
6.2.3 How cart group outpoints be determined in cases where all cars
tested pass the FTP or fail the FTP?
Response: This issue is shown graphically in Figure 15-
FTP
;<
* X
V*
* >¦
ST
ST
Figure 15
There are six opt ions possible for handling this problem. Two are
by mathematical means, two require additional testing and two rely
on engineering judgment.
Option 1: This option 1 would fit a linear regression equation to
the available data and use it to project the short test/FTP relationship
i
beyond the region of available data. The short test cutpoitlt would be
selected to correspond with the FTP standard allowing a margin of safety
to account for correlation. Expressed in equation form, this approach
says:
-------
106
ST
= ST
cutpoint value @ regression
line intercept with FTP
x (£}
correlation
coefficient
In the case where this option has been evaluated, the function, f, for
HC and NOx was determined to be a constant, the value of which was 0.78,
while for CO the function was also a constant of value 0.5. The results
obtained by the 5% Ec method and by this option were:
Short Test Outpoints
HC . CO NOx
(pptn) (%) (ppm)
E - 5% 184 0.71 2265
Model (Option 1) 186 0.75 2365
Option 2: Option 2 would fit a statistical probability distribution
to the available data- Then, data could be mathematically generated in
the region where they do not exist. A standard method of selecting the
cutpoint would then be applied.
/
Option 3: A defect test program can be initiated to obtain data in
the region where th.£ population at vehicles is scarce. In .cases where
there are no data on passing FTP cars, the program would consist of
tune-ups rather than incorporation of defects. Such a program would
require a knowledge of what defects to introduce as well as a large
number of vehicles and tests so as to include vehicle to vehicle vari-
abili ty.
-------
107
Option 4: Test an additional group of vehicles. If this second
group of vehicles included vehicles dissimilar to those originally
tested (i.e. not all either PP or FF vehicles), the ST outpoint would
be set from the second group of vehicles. This option possesses the
weakness that the second group would probably behave the same as the
first group and nothing would have been accomplished.
Option 5: Transfer standards from a similar group. This option
would apply within groups of vehicles produced by the same manufacturer.
Provided the same vehicle size/engine size/technology is utilized in
meeting emissions standards, it can be argued that the products manu-
factured by Division A of a corporation are sufficiently similar to the
products from Division B for the ST outpoints to be transferable in
this eventuality. An example of this approach is the use of Chevrolet
Division intermediate size vehicles short test cutpoints for Pontiac
Division intermediate size vehicles provided the same displacement
engine
-------
108
only can be used to predict cutpoints. These cutpoints can then be
compared with the cutpoints determined from the entire data base. These
analyses show promising results. They will be completed prior to final
rulemaking and if they are successful, there would be no need to implement
additional testing programs or adopt either of the other two options.
6.2.4 Should each pollutant be treated independently in setting
cutpoints?
The main advantage of treating each pollutant independently is that
the computational procedures are relatively simple. Computational procedures
become extremely complex when all pollutants are grouped together and
trade-offs must then be made of the relative health effects of each
pollutant so that it can be given its factor of importance. Moreover,
air quality decisions have historically tended to be made based on
individual control of each pollutant. The disadvantage of considering
each pollutant separately is that the true vehicle commission error
is not immediately known. The true error rate can only be determined
by examining each vehicle separately, i.e. if the vehicle is. a commission
error for HC and fails CO then it is not a true commission error. The
converse is also true. The only way that the true commission error rate
can go higher than the rate set by individual pollutants is when all-
pollutants on the same vehicle are errors of commission or correct
passes. The net effect is that true commission errors are usually
lower than individual pollutant errors because "bad" vehicles tend to
truly fail (fail the ST) on more than one pollutant. While each group
-------
109
could have equivalent commission errors on individual pollutants, the
group- to-group composite error rates could differ significantly. A.
group manufacturer is only concerned with his true commission error rate
taken over all pollutants.
In order to treat the pollutants jointly, there must be a method
for combining errors and failures across pollutants as referenced above.
Several methods could be used. A total percentage of commission errors
could be equally divided between the two or three pollutants and a
commission error on a single pollutant would only be considered a commission
error if it remained a commission error when all pollutants were con-
sidered. A second method would be Co weight the allowable individual
pollutant commission errors according to some other factor such as
failu re rate or ambient air quality level in a specific region divided
by ambient air quality standard. Two examples can be given. First,
assume the true commission errors are to be controlled to five percent and
the test sample of vehicles indicates that k51 of the vehicles fail CO
and 30% fail HC. Then, the commission errors could be divided so that
31 of the commission errors were a result of incorrect CO failures and
27, of the commission errors were a result of incorrect HC errors.
Second, assume a given AQCR has an HC ambient level of . 16 ppm and a CO
ambient level of 13.5 ppm. Thus, HC exceeds the standard by a factor of
2 and CO exceeds the standard by a factor of 1.5. The commission errors
could be apportioned so that 2.9 percent of the errors were HC errors
and 2.1 percent were CO errors. This approach would tend to result in
higher 207(b) failure rates for HC and thus, HC air quality benefits
would be weighted to account for the greater ambient problem.
-------
110
Outpoints used in this paper were selected based on each pollutant
independently- Preliminary analysis based on 1973-197 5 vehicles tested
in the emission factor program indicates that the overall commission
error rate would be expected to be, at most, the same as the individual
pollutant error rates and it could be considerably less. This is shown
in Table XXIII.
Commission Errors Combine Across Pollutants
Combined Commission Errors
Failure rate
1975 Model
1973-1974 Models
HC
CO
Both
HC
CO
Both
HC & CO
HC & CO
20%
1
2
2
1
3
4
30%
4
3
4
2
6
6
40%
7
6
7
3
8
7
50%
13
8
10
5
11
10
TABLE XXIII
Table XXIII shows the results when the respective fleets are treated
as a whole. Because of technological differences in vehicles and because
of varying effects of production tolerances, it is reasonable to expect
that the true error of commission rate will vary between manufacturers
and between groups- This effect needs to be further evaluated based on
existing data.
6.2.5 Is there a need to perform more than one cutpoint program for
a given model year of vehicles?
Response: If the contingency table relationship between the FTP and
the short test is mileage dependent, i.e. the correlation is mileage
-------
Ill
dependent, then the outpoints would have to be changed as the vehicles
accumulated mileage. There is no basis in engineering to support the
argument that the correlation between the short tests and the FTP is
mileage dependent.
As the vehicles age, and their emissions controls deteriorate but
are not repaired, then a greater number of vehicles will fail the short
test, and rightly so- The ability of the short tests to perform Che
function of detecting component failures can be seen in the increased
correlation indices of the defects fleet, vs. the correlation indices of
the CEV fleet as shown in Tables I, II and III. In this case, the short
test cutpoints developed for the CEV fleet were applied to the defects
fleet with the result that more vehicles failed the short tests, thereby
supporting the conclusion that correlation is not mileage dependent.
Additional support for the argument for not changing cutpoints with
mileage was obtained from available 1975 model year data which has been
analyzed. While only two distinct mileage categories have sufficient
data, these-categories indicate no statistical difference in cutpoint
would be obtained. Analysis is ongoing to extend this work./to pre-1975
vehicles since the available data base is larger.
Work also needs to be performed to address the question of what the
confidence interval is around the cutpoint that was determined from the
57o commission error criteria. (If a decision is made to use another
-------
112
cutpoint methodology, that confidence interval will also have to be
developed). The development of the confidence interval is ongoing. It
is a complex statistical problem, depending on the sample size, total
short test failure rate and total FTP failure rate. The work is
expected to be completed prior to final rulemaking so that decisions can
be made as to what magnitude of differences in the two cutpoints is
statistically significant.
6. 3 Other Technical Issues
6.3.1 Are garage type instruments adequate to implement 207(b)?
Response: The answer to this question is clearly yes for HC and CO
instruments when the results given below in Table XXIV are compared.
These results were obtained using Pearson regression analysis on the CEV
fleet. The question cannot be clearly answered for HOx at this time as
data have not been collected using garage caliber NOx instruments since
these instruments were not available at the time of the correlation
investigated.
-------
113
Correlation Coefficient
Short
Test
Mode
HC
CO
Clayton Key Mode
(Laboratory Instruments)
Hi
0.61
0.26
low
0.53
0. 39
idle
0.92
0.54
(Garage Instruments)
Hi
0. 73
0.37
low
0.73
0.21
idle
0.88
0.52
Federal 3 Mode
(Laboratory Instruments)
Hi
0.87
0.08
low
0.79
0. 22
idle
0.80
0.48
(Garage Instruments)
Hi
0. 76
0.24
low
0.73
0. 21
idle
0.78
0.52
TABLE XXIV
The answer to this question is not as clearly yes when the results
given in Table XXV are compared. These results were obtained from
the 1974 MY fleet of vehicles.
-------
114
Correlation Coefficient
Short
Test
Clayton Key Mode (Laboratory)
all vehicles
Chrysler Corp.
Ford Motor Co.
Chevrolet (GM)'
(Garage)
all vehicles
Chrysler Corp.
Ford Motor Co,
Chevrolet: (CM)
Mode
idle
idle
HC
0.793
0.723
0.825
0.460
0.455
0.245
0.692
0.100
CO
0.739
0.704
0.650
0.757
0.470
0.372
0. 560
0.229
Federal 3 Mode (Laboratory)
all vehicles
Chrysler Corp.
Ford Motor Co.
Chevrolet (GM)
(Garage)
all vehicles
Chrysler Corn.
Ford Motor Co.
Chevrolet (GM)
idle
idle
0.803
0. 709
0.851
0.252
0.632
0.660
0.717
-0.060
0.734
0.724
0. 622
0.733
0.476
0. 397
0.550
0. 392
TABLE XXV
Sensitivity to instrumentatioa was noted in the result's from the
1974 roodel year fleet. When ail of the vehicles were treated as a
group, the laboratory instruments show a superiority over the garage
instruments. When the data are analyzed by vehicle group, this supe-
riority o£ laboratory instruments over garage instruments holds for the
Chrysler Corporation cars in the Clayton Key Mode test but almost dis-
-------
115
appears in the Federal 3 Mode Test. In che ease of the Cbevcolots,
where the correlation is very poor for both classes of instruments, the
laboratory instruments are clearly better. In the case of the data
obtained from the Ford Motor Company vehicles, the differences between
laboratory instruments and garage instruments diminishes substantially.
The CEV fleet were also Ford Motor Company vehicles.
The following conclusions are based on all of the data collected to
date:
t. Laboratory instruments show a superiority over garage in-
struments on some vehicles.
2. Garage instruments almost never show a superiority over
laboratory nstruments.
3* Approximate equivalence between garage and laboratory instruments
has been noticed on two different groups of Ford Motor Company
vehicles. These vehicles were the low emitting, catalyst fleet
vehicles, which were Cull size Fords, arul the '' i. vitas in tue lv / ^
model year fleet.
A. The laboratory instruments were clearly superior for the 1974
f-fY Chevrolet vehicles.
-------
) 1 f-
L l O
The magnitude of this problem, if it continues, will be placed
in perspective when the data from the Portland test program becomes
available. Moreover, it will be necessary to recalculate these
correlation coefficients based on the contingency table technique
rather than Pearson correlation.
It is important to realize, however, that the effect of garage
instrumentation will automatically be factored into the short test
pass/fail levels. Thus, although some potential air quality benefit nay
be lost by the non-use of laboratory instrumentation, the commission
errors will not increase,
Approximate costs for one set of test equipment in each instrument
class, including all equipment necessary for measuring exhaust emissions
are $S0 K to $120 K for laboratory instruments and $12 K to S18 K for
garage instruments.
Differences in the costs of dynamometers center on the requirements
of the test to be performed. If the test requires a simple_dynamometer
which only has to simulate steady speed conditions then the cost will
vary between $2,000 and $3,000. If the dynamometer has to simulate
transient vehicle conditions, i.e., inertia, then the cost will vary
between $9,000 and $11,000.
6.3.2 What effect will real world conditions have oit short test
correlation?
-------
117
Response: There are presently no data on the impact of real world
variables such as inspector errors resulting from job routine, ambient
variations, the waiting period oC the vehicle which precedes the short
test, etc. on short test to FTP correlation.
Human errors, such as those caused by a routine type of job,
inaccuracies in following a driving trace if used, misreading at\ instrument
or misidentifying the engine which is in a vehicle, must be expected, and
accounted for, in a vehicle inspection program- Evaluation of the
magnitude of this problem is not presently possible because of a lack
of data. The data necessary for making this quantification will be
obtained in the Portland project.
Evaluation of the effects of ambient variations and the time that
Che vehicle waits to be tested will be accomplished on the basis of the
Portland project. Ambient effects will be evaluated through the comparison
of data taken during cold, moderate and hot weather conditions. Vehicles
/
participating in the project will be subject to the same variations in the
waiting p#tiod as vehicles not in the project- The waiting period of
I
each vehicle will be recorded and short test results will be evaluated,
if possible, to determine the effects on emissions of delays which occur
before testing.
-------
118
Since the short test pass/fail levels are determined by a set error
of commission rate (or possibly some other set efficiency measurements) and
as the short test data which will be used for setting the actual pass/fail
levels will be collected under real world conditions, the effects of
these variables will automatically be factored into the short test
pass/fail values.
6,3.3 Will change.s in technology obsolete one or more of the currently
correlatable short tests?
Response: As manufacturers develop sophisticated computerized
engine control systems, it is possible that current correlatable short
cycles would become less correlatable. That is, the potential always
exists for a manufacturer to design around a specific emission test. A
similar problem could occur if EPA were to go to a non-methane hydrocarbon
standard. The methane hydrocarbon fraction may be mode dependent and
may lessen the correlation presently achieved based on total hydrocarbons
for the FTP and the equivalent as measured by the garage instruments.
In order to keep abreast of potential changes in short test cor-
relatabilxty, all prototype vehicles obtained in the laboratory for
technical evaluation will be tested on the 207(b) short tests. Large
numbers of these vehicles will also be tested using continuous measure-
ment techniques. Thus, data will be available to indicate whether
potential, problems exist with current short tests and, if necessary, these
data will form a base for developing new short tests.
-------
119
An additional source of data could come £ram the prototype certifica-
tion vehicles. Manufacturers will be asked to comment in the NPRM as
to any problems which would be incurred by including 207(b) short tests
on all certification tests performed in the EPA lab and/or at manufacturer
facilities.
fc.3.4 Will the implementation of 207(b) affect manufacturer's recommended
service intervals?
Response: Manufacturers recommended service intervals are presently
based on "average" vehicle use. When manufacturers have to warrant
emission levels for all vehicles up to their useful life mileage levels,
they may want no reduce the scheduled service intervals. A decision on
how to set these service intervals may depend on an assessment of the
difference in emissions/maintenance between the average car and the cat"
used in severe service. Manufacturers will be asked to comment on this
issue in the NPRM. Also, it may be possible to obtain service related
data from some severe service fleets such as taxi cabs, rental fleets,
etc. This. Information will be obtained, to the extent possible, before
final rulemaking.
6.4 Policy Issues
6.4.1
What is an acceptable level for reasonable correlation?
-------
120
Response: This is strictly a policy issue although the acceptable
level of commission errors could be expected to be related to the total
failure, rate. After the August 19, 1976 briefing for Mr. Quarles, he
indicated that a level of equal to 5 percent was not unreasonable
given that the total failures were between 30 and 40 percent. Also, the
level established during the first year of vehicle use may decrease as
vehicles accumulate mileage. The decrease would not occur because of a
change in the FTP/short test relationship. Rather, it would occur
because larger percentages of vehicles fail and fail at higher FTP
levels.
The vehicles which are identified as commission errors can be
examined and their FTP emissions compared with vehicles which are
identified as passed vehicles. In one small sample of vehicles, at a 3"
E level, vehicles in the E cell have mean FTP emissions significant Lv
c ' e
above those which are classified into the PP cell, as given in Table
XXVI,* Thus, the cars identified as F. cars could be expected to exceed
c
Federal FTP standards within a year if they were not maintained and
would fail at the second inspection. Therefore, they may not be real
errors in the sense of a once a year inspection program. It must be
stressed here, however, that all cars were examined by the same eutpoiut,
which is known not to be equitable. The picture will probably change
extensively onc.e vehicles are properly grouped and their appropriate
cutpoints applied.
6.4.2
the FTP?
What is acceptable vehicle variability on the short tests and
-------
121
Mean and Percent of 1975 Model Year Vehicles
Classified Pass/Fail by the FTP and the Chicago Idle Test-
FTP
Pass Fail
Chicago Test
Chicago Test
Pass 45% 28%
Fail 3% 24%
FTP
Pass Fail
Pass HC = 0.67 HC = 1.44
CO = 9.30 CO =28.11
Fail HC = 1.14 HC = 2.01
CO =12.96 CO =44.79
TABLF. XXVI
* based on 137 vehicles tested in the FY74 emission factor program.
-------
122
Response: This is a difficult question to answer. It was brought
up in the fuel economy regulations and was not resolved chere. The data
from the Portland experiment would permit an assessment of variability
as a function of FTP level or average commission error level or average
omission error level. It is improbable that the 3 cities data can
provide a partial answer to these, relationships. However, a decision on
acceptable variability would require inputs from the manufacturers.
6.4.3 Should the annual cutpoint program be performed by the states
or by EPA?
Response: The Clear Air Act directs the Agency to implement section
207(b). It docs not specify, however, what the Agency's role is to be
in the yearly selection o£ short test pass/fail levels, variations in
the applicability of short, tests with modifications in technology, etc.
In light of the continuing changes in the vehicle emissions standards
and modifications in the technology used to meet these standards, it is
reasonable to expect that changes in the short test pass/fail levels
must be made on a model year basis. These changes can be made by the
agency from data collected from a single or at most two fleets of
vehicles representing the national standards at high and low
altitudes. If the states were to perform the work necessary
to execute the changes, then each state would have to test a fleet of
vehicles—a costly and inefficient procedure. It appears logical,
therefore, for the agency to furnish each state with a yearly set of
-------
123
shore test pass/fail levels so as to maintain national uniformity as
well as to hold costs at an acceptable level.
In the August 19, 1976 Quarles briefing, Mr. Quarles commented that
he did not believe that the states will have the resources to establish
outpoints and that EPA should plan to undertake the task with the under-
standing that states could develop more stringent cutpoints if they
should desire.
6. A. 4 Mill the Federal 207 (b) cutpoints pre-empt state I/.M standards?
Response: States would have the option of setting higher or lower
I/M pass/fail levels. The question of concern is whether states would
be able to trigger the 207(b) warranty with more stringent l/M standards.
This issue would seem to require a state to run their own cut point,
program to prove that reasonable correlation still exists. Or, the data
collected by EPA in their cutpoint program could be analyzed for the
proposed state cutpoint levels. The decision on reasonable correlation
would be difficult to determine on a state by state basis.
6.4.5 Will final rulemaking include the results obtained.in the
Portland study?
Response: The present schedule for the Portland study is to have
the contract signed by December, 1976. The 207(b) testing phase of chid
-------
124
program would not be completed 'until June 1978, although preliminary
results would be available throughout the year. The Portland study will
provide insight into the number of outpoint levels needed, the size of
ttie yearly outpoint program, the effect of real world variables on short
test correlation and the effect of variability on correlation. The
program is not expected to change the recommendations on which short
tests are correlatable nor the methodology for selecting a outpoint for
a given vehicle group. Thus, final rulemaking can proceed prior to
completion of the Portland study.
6.A.6 Are there other uses for the yearly outpoint data?
Response: The yearly cutpoi.nt program has the potential to be u.^ed
to implement section 207(c) of the Clean Air Act, the recall provision.
Sample sizes of individual groups will be relatively large, test vehicles
will be properly maintained and used, testing will be performed early in
the new model year, and testing will be performed using the FTP* as well,
as short tests. By performing a 207 (c) program in conjunction-.vi th the
207 (h) outpoint program, owner:; of defective class vehi.cl.es will he
protected in states with more stringent l/M standards.
Although some modifications may be incorporated in the FTP (ie: elimina-
tion of evaporative emissions tests and minor modification of the pre-
conditioning cycle) the effects of these modifications on the requirement
of reasonable correlation for 207(b) and the requirement of a substantial
number of nonconforming vehicles for 207(c) are expected to be negligible.
A test program will be initiated to investigate the effects of these test
d i f f erenees.
-------
125
Appendix A
Graphical Descriptions of Short Tests
-------
50.
/
Heavy Vehicles
Med ium Vehicles /^/ /*
40
/ / / Light yehicles^
in
n
m
o
3"
Heavy Vehicles
30
Medium Vehicles
20.
•• Light Vehicles
10.
'-y— -
TIM
F3M High
/'
> KM High
"i
m
CL
re
rj
0J
b-*
F3M Low
/
/
KM Low
H.
J-*)
ui
X)
a>
ro
a.
M
G-
O
D-
Q
n
V_i
CJ
ft
o
p
n>
V;
re
o
a.
Q
,-'1/
/
. *.
•c-qui red
Idle in neutral - F3M
Idle in Drive - KM
2500 rpm - High Speed Idle
-------
------- |