Portland Study: First Preliminary Data Analysis by Janet Becker May, 1978 Characterization and Applications Branch Emission Control Technology Division Office of Mobile Source Air Pollution Control Office of Air and Waste Management U.S. Environmental Protection Agency ------- I. Background The EPA is currently conducting the Short Test Correlation and Effectiveness Study, which is a large scale study of Oregon's motor Vehicle inspection program. Through this effort, EPA is' gathering information to quantify the emission reduction and fuel economy effects that are achieved in an ongoing inspection and maintenance (I/M} program. Much emphasis is placed on obtaining information on the practical aspects of I/M, including consideration of short test effectiveness in identifying vehicles whose exhaust emissions are above federal certification standards, costs of repair, and air quality impact. A major goal of the study is to determine methods of optimizing the effectiveness of I/M. There are two basic study areas in the Short Test Correlation and Effectiveness Study (Portland Study). The first area, designated Element I, is intended to address questions regarding the "corre- latability" of short emission test procedures with the Federal Test Procedure. The motivation for this study area is Section 207(b) of the Clean Air Act. For Element I, 2,400 1975-77 model year light duty vehicles will be tested over three short emission tests at an Oregon state inspection lane, and over these three tests plus the FTP at the EPA contractor's (Hamilton Test Systems') laboratory facility to evaluate qualitative prediction (accurate prediction of FTP passage or failure in conjunction with air quality considerations) approaches to meeting the 207(b) correlation requirement. Special emphasis will be placed on assessing the effects of the real-world Portland I/M situation on correlation. The second study area, designated Element II, is intended to address questions relating to the cost/effectiveness of inspection and maintenance as an in-use vehicle emission control strategy. There will be a total of 600 light duty vehicles tested for Element II, with some overlap between Element I and Element II vehicles. A description of the vehicle groups represented in Element II is contained in Table 1. Vehicles from Eugene, Oregon, are intended to serve as a "control group" in the sense that I/M is not required of these cars and they have been selected to match the Portland vehicles within each model year-pass/fail group with respect to model year, CID, and model type. The development of techniques for making appropriate comparisons of Portland and Eugene data is currently underway. All vehicles in Element II are subjected to several short emission tests, a diagnostic inspection, the FTP, and the Highway Fuel Economy Test. If a Portland area vehicle fails the state test, it returns following maintenance and is retested over the full and hot start FTP, the HFET, and the idle test by both Oregon Department of Environmental Quality (DEQ) and Hamilton Test Systems (HTS) to determine the immediate effect of maintenance. Information on the type and cost of maintenance performed and on diagnostics is also collected. ------- -2- Table 1 Description of Element II Vehicle Groups Group Description Number of Vehicles 1. Portland area 1972-74 model year vehicles which passed the state inspection test, 100 2. Portland area 1975-77 model year vehicles which passed the state inspection test, 100 3. Portland area 1972-74 model year vehicles which failed the state inspection test, 100 4. Portland area 1975-77 model year vehicles which failed the state inspection test, 100 5. Fugene, Oregon 1972-74 model year vehicles, 100 6. Eugene, Oregon 1975-77 model year vehicles. 100 TOTAL ELEMENT II VEHICLES 600 Eugene vehicles are not subject to mandatory I/M, so retest immediately after maintenance does not occur. However, all 600 vehicles are tested at 3-month intervals over the following year to obtain estimates of FTP emission levels, FTP emission deterioration, idle deterioration, fuel economy deterioration, and information on diagnostics and voluntary owner maintenance. II. Data Flow Following vehicle testing/data collection, data packets containing test-related information such as strip charts, calibrations, etc., are manually reviewed by the EPA personnel assigned to Portland for the duration of the study. When EPA approves the data packets, the complete sets of data are placed onto files and are computer- checked for reasonableness by EPA. Necessary changes are made by HTS personnel in coordination with EPA. Once this checking/editing procedure is completed, the data are transferred onto magnetic tape for data analysis. Analysis is carried out in Ann Arbor by CAB personnel. ------- -3- III. Preliminary Analysis of Portland Project Data A. Status of the Data Base The status of the Portland Study data base at the time of this analysis is summarized below in Table 2. This is a status report of vehicles whose data have been checked/edited, and placed onto magnetic tape for analysis. Table Z Status of the Portland Study Data Base Vehicle Group Number of Vehicles Element I 319 Element II, Portland, 1972-74, pass 94 Element II, Portland, 1975-77, pass 89* Element II, Portland, 1972-74, fail 80 Element II, Portland, 1975-77, fail 88* Element II, Eugene, 1972-74 95 Element II, Eugene, 1975-77 93 TOTAL 681 *These vehicles served as both Element I and Element II entries. That is, of the 319 vehicles listed under Element X, 177 served as Element II vehicles also. B. Cost/Effectiveness of I/M 1. Cost and Immediate Effect of Maintenance At the present time, approximately half of the information on FTP emission reduction immediately following maintenance, and on the cost of this maintenance is available. The average cost of maintenance for the 1972-74 model year (pre-catalyst) Portland vehicles is $37. For 1975-77 (post-catalyst) model year Portland vehicles, this figure is $26. However, over half the vehicles have incurred costs of under $15. Both average and per vehicle reductions on HC and CO FTP emissions immediately after maintenance are substantial, with the 1972-74 model year group experiencing an ------- -it- overall 30% reduction for each pollutant, and the 1975-77 model year group experiencing about a 50% reduction for each pollutant.* Except for post-catalyst CO, average HC and CO levels following maintenance are within 3% of certification standards. Average post-catalyst CO follow- ing maintenance is still about 20% above the certification standard. NO emissions are showing a slight, statistically insig- nificant, increase following maintenance. City fuel economy shows a slight increase, and highway fuel economy shows a slight decrease. The information on FTP emissions and fuel economy before and after maintenance is displayed in Table 3. For comparison, information on idle emission reduction is displayed in Table 4. Percent reductions on idle HC and CO are running about two times higher than the corresponding percent reductions on the FTP. 2. Effectiveness after Three Months Back on the Road Information on emission deterioration following maintenance is very important in evaluating the effectiveness of an I/M program. For example, if failed vehicles' post- maintenance emissions are deteriorating to pre-maintenance levels shortly after the retest, then I/M's potential effectiveness will not be realized. Information on FTP deterioration will be obtained over a one-year time period via the quarterly retest aspect of the Element II program. Quarterly retest data on Element II failed vehicles are very limited at this point in time, with 17 1972-74 model year failed vehicles and 10 1975-77 model year failed vehicles having returned for their first quarterly retest. However, a review of the vehicle-specific FTP emissions data leads to two pre- liminary conclusions: * These reductions indicate the experience of the vehicles currently in the Portland Study's data base. More data from the Portland Study, plus information on Portland's mix of failed vehicles are needed before corresponding reductions for the Portland area can be estimated. These reduction estimates should"be available in the next report. ------- -5- 1'able 3 FTP EMISSIONS BEFORE AND AFTER MAINTENANCE 112 1972-1977 MODEL YEAR LIGHT DUTY VEHICLES 57 1972-74 Model Year Cars HC (g/mi) CO (g/mi) NOx (g/mi) City F.E. (mi/gal) Highway F.E. (mi/gal) Before 3.95 50.2 3.19 After 2.74 35.0 3.22 Percent Change -31% -30% +1% 16.3 16.3 -0% 23.4 22.8 -3% (3.0)' (34.0)* 55 1975-77 Model Year Cars HC (g/mi) CO (g/mi) NOx (g/rai) City F.E. (mi/gal) Highway F.E. (mi/gal) Before 2.89 38.9 After 1.55 18.1 Percent Change -46% -53% 2.49 2.51 +1% 16.4 16.6 +1% 23.5 23.1 -2% (1.5)* (15.0)* *1975 FTP certification standards ------- -6- Table 4 IDLE EMISSIONS BEFORE AND AFTER MAINTENANCE 112 1972-1977 MODEL YEAR LIGHT DUTY VEHICLES 57 1972-74 Model Year Cars 1 2 DEQ Results HTS Results Idle HC Idle CO Idle HC Idle CO (ppm) (%) (ppm) (%) Before 353.5 4.12 348.4 3.20 After 152.7 0.81 160.4 0.80 Percent Change -57% -80% -54% -75% 55 1975-77 Model Year Cars DEQ Results HTS Results Before After Percent Change Idle HC Idle CO Idle HC Idle CO (ppm) (%) (ppm) (%) 361.2 3.45 390.0 4.92 91.7 0.33 78.7 0.28 -75% -90% -80% -94% ^"Oregon Department of Environmental Quality Test is performed at a state inspection station by state personnel. ^Hamilton Test Systems. Test is performed at the contractorTs laboratory facility. ------- -7- 1. The potential for sustaining emission levels achieved immediately after maintenance exists, 2. On a substantial percentage of vehicles, rapid emission deterioration is occurring shortly after I/M maintenance is performed. The potential for sustaining emission levels achieved immediately after maintenance is evidenced by the fact that of the vehicles which are reduced to a certain percentage of their initial emission levels, some either stay or go below these post-maintenance levels in the quarterly retest. One way to get a quantitative handle on this fact is to express each vehicle's initial, post-maintenance and quarterly retest FTP emission levels as a fraction of its initial level, thus achieving vehicle-specific normalized emission levels: for each pollutant, a vehicle's initial normalized level is 1.0, with repeat test normalized levels falling below or above 1.0, depending on whether a reduction or increase from the initial level occurs. Since the normalizing factors are vehicle-specific, the average FTP reductions/increases cannot be derived directly from the average of the normalized numbers. The following graphs display the averages of the normalized numbers for all 1972-74 model year vehicles' HC and CO, and 1975-77 model year vehicles' HC and CO, respectively. The average emission levels for these vehicles initially, immediately after maintenance, and at the quarterly retest are displayed in Table 5. Normalized HC FTP Emission Levels of 17 1972-74 Model Year Vehicles Normalized CO FTP Emission Levels of 17 1972-74 Model Year Vehicles 1 A -A 1 1— Initial Post- Quarterly Initial Post- Quarterly Test mainten- retest Test mainten- retest ance ance ------- -8- Normalized HC FTP Emission Levels of 10 1975-77 Model Year Vehicles Normalized CO FTP Emission Levels of 10 1975-77 Model Year Vehicles 1.0 0.64 0.77 1.0 1.61 H » 1 : i t Initial Post- Quarterly Initial Post- Quarterly Test mainten- retest Test mainten- retest ance ance Initial Test Post-Maintenance Quarterly Retest Table 5 Average FTP Emission Levels 27 1972-77 Model Year Vehicles 27 1972-74 Model Year Vehicles HC CO NOx (g/mi) (g/mi) (g/mi) 2.90 42.1 3.52 2.46 28.9 3.48 3.18 32.4 3.56 10 1975-74 Model Year Vehicles HC CO NOx (g/mi) (g/mi) (g/mi) 2.48 33.3 2.28 1.43 16.1 2.09 1.74 25.6 2.10 The above graphs include all 27 of the failed vehicles which have had their first quarterly retest. However, five of these vehicles' FTP levels increased extremely rapidly following maintenance and thus strongly affected the graphs presented above. Omitting these five vehicles, the following graphs result. These five vehicles' FTP levels are shown in Table 6. ------- -9- Normalized HC FTP Emission Normalized CO FTP Emission Levels of 15 1972-74 Model Levels of 15 1972-74 Model Year Vehicles Year Vehicles • -^93 1.0 I-0 0?73~ 0.76 ! 1 1 \ Initial Post- Quarterly Initial Post- Quarterly Test mainten- retest Test mainten- retest ance ance Normalized HC FTP Emission Normalized CO FTP Emission Levels of 7 1975-77 Model Levels of 7 1975-77 Model Year Vehicles Year Vehicles 1.0 1.0 0.64 0.65 0.72 -4 1 1 1 H Initial Post- Quarterly Initial Post- Quarterly Test mainten- retest Test mainten- retest ance ance The information on emission deterioration following maintenance is very limited, but suggests the possibility that relatively small numbers of rapidly deteriorating vehicles could significantly impact I/M's effectiveness. Future analysis will be aimed at identifying specific ------- -lo- calises of such rapid deterioration in. an effort to determine steps which could be taken to optimize I/M's effectiveness. Table 6 FTP Emission Levels 5 1972-77 Model Year Vehicles 1972-74 1975-77 Car 1 Car 2 Car 3 Car 4 Car 5 HC CO HC CO HC CO HC CO HC CO (g/mi) (g/mi) (g/mi) (g/mi) (g/mi) (g/mi) (g/mi) (g/ml) (g/mi) (g/mi) tial Test 1.54 46.8 2.75 43.9 3.97 3.78 3.96 76.0 1.80 35.2 t-Maintenance 1.15 42.1 2.41 27.5 2.36 11.0 2.66 9.53 1.10 17.3 rterly Retest 11.43 56.2 2.88 50.4 2.93 34.9 3.44 38.1 2.76 47.3 3. Identification of High FTP Emitters Portland's I/M program is intentionally designed to give the motorist the advantage of any doubt in the pass/fail decision. First, the idle cutpoints may be considered to be lenient, and, in addition, tolerances are placed on the cutpoints so that, for example, catalyst vehicles are typically required only to stay within a 1% idle CO reading. However, the fact that failed vehicles are on the average emitting at substantially higher levels than certification standards, whereas passed vehicles are on the average emitting at or below certification standards suggests that "gross emitters" of FTP HC and CO are nonetheless being caught. Table 7 displays average FTP emission levels for passed vs. failed Portland area Element II vehicles. 4. Incorrectly Failed Vehicles Of the 112 Portland area Element II vehicles which failed the state inspection test and returned for maintenance, four were failed incorrectly. All four vehicles were 1975-77 model year cars whose average FTP HC, CO, and NO did not change significantly following maintenance. Hie average cost of maintenance was $18, with a minimum cost of $0 and a maximum cost of $37. ------- -11- Table 7 INITIAL FTP EMISSION LEVELS PASSED VS. FAILED PORTLAND AREA ELEMENT II VEHICLES 1972-74 Model Year Cars HC CO NOx N (g/mi) (g/mi) (g/mi) Passed 94 3.08 33.8 3.42 Failed* 80 3.98 52.9 3.18 (3.0)** (34.0)** 1975-77 Model Year Cars HC CO NOx N (g/mi) (g/mi) (g/mi) Passed 89 1.20 11.1 2.63 Failed 88 2.72 38.1 2.43 (1.5)** (15.0)** * The pass/fail ratio reflects the study design: for Element II, equal numbers of passed and failed vehicles were recruited for each Portland area model year group. ** 1975 FTP certification standards. IV. Idle Test/FTP Correlation Short test/FTP correlatability is a major issue in the promulgation of Section 207(b) of the Clean Air Act. EPA's approach to defining correlatability to date has been from the back door: find reasonable ways to relate short test emission measurements and FTP emission measurements in the hope that a good and practical definition of correlation will surface. One of the possible ways of relating short test, and in particular, idle emissions to FTP emissions is via a qualitative prediction approach. Basically such an approach would evaluate the degree to which the idle test is an accurate predictor of FTP passage or failure, and would include considerations of air quality in addition to necessary public policy considerations. ------- -12- The success of the idle test in accurately predicting FTP passage or failure and in realizing potential air quality benefits ultimately depends on the idle test pass/fail outpoints. Therefore, the degree of accurate prediction and the degree to which potential air quality benefits are identified in the Portland Study must be looked at with the perspective that one set of cutpoints produced the pass/fail results. Another set of cutpoints could produce more or less accurate prediction and could identify more or less of the potential for air quality improvement. The analysis of the Portland data with respect to qualitative prediction approaches to correlation will be modified to accommo- date policy decisions which affect the Agency's view of correlation. Currently, Technology Services Corporation, under contract to EPA, is developing theory which will facilitate a thorough investigation of possible relationships between short tests and the FTP. One major development will be to use short test measurements on HC, CO, and NO simultaneously to increase the predictive capabilities of short ?ests. An early look at all Element I data (all 1975-77 model year vehicles combined) indicates that approximately 60% of the vehicles' DEQ idle test pass/fail results concur with their FTP pass/fail results. The inaccuracies fall largely in the "incorrect pass" category. Only 2% of all vehicles were incorrectly failed. The following matrix summarizes this information. Fail 126 (39%) 87 (27%) 213 Pass 100 (31%) 6 (2%) 106 226 93 319 Pass Fail DEQ Idle Test ------- -13- Although a detailed analysis of the extent to which potential air quality benefit is identified by the DEQ idle test has not been performed, the average emission levels per cell of the above matrix suggest that identification is occurring for HC and CO, but perhaps not for NO . The following matrix, which corresponds to the one above, displays the average emission levels per cell. Fail FTP 1.49 HC 16.5 CO 2.84 NO x 2.80 HC 39.6 CO 2.45 NO X 0.93 HC 0.98 HC Pass 8.08 CO 10.3 CO 2.04 NO 2.19 NO X X Pass Fail DEQ Idle Test V. Summary The following tentative conclusions may be appropriately drawn from the analysis which has been performed to date: 1. The immediate effect of maintenance on HC and CO FTP emission levels is substantial. For the current sample of vehicles, percent reductions for pre-catalyst vehicles immediately after maintenance are approximately 30% for each pollutant, and for post-catalyst vehicles are approximately 50% for each. 2. The potential for obtaining and sustaining low FTP emission levels following idle test-related maintenance exists. 3. On a substantial percentage of vehicles, rapid emission deteriora- tion is occurring shortly after I/M maintenance is performed. 4. The average cost of maintenance is $25-35, with over half of the vehicles requiring less than $15 for maintenance. 5. The side effects of reducing HC and CO are minimal with respect to NO^, and both city and highway fuel econony. 6. For the current sample of vehicles, idle emission reductions immediately following maintenance (in percent) are about twice as high as the corresponding FTP emission reductions. ------- -14- 7. Oregon's I/M program is catching gross emitters despite the f^ct that the Oregon pass/fail cutpoints may be considered lenient. 8. Failed vehicles' HC and CO average emission levels immediately after maintenance are close to certification standards, except for post-catalyst CO, where the after maintenance average is 20% above the certification standard of 15.0 g/mi. 9. Incorrect failures are uncommon, and the effect of maintenance on these vehicles' average FTP levels is insignificant. 10. Oregon's I/to program appears to be doing a reasonable job of identifying potential air quality benefit with respect to HC and CO. This is probably not so for NO^. ------- |