EPA-AA-IMS-81-11 Technical Report A Study of the Effectiveness of Mechanic Training For Vehicle Emissions Inspection and Maintenance Programs R. Bruce Michael April, 1981 Notice Technical Reports do not necessarily represent final EPA decisions or positions. They are intended to present technical analysis of issues using data which are currently available. The purpose in the release of such reports is to facilitate the exchange of technical information and to inform the public of technical developments which may form the basis for a final EPA decision, position or regulatory action. Inspection and Maintenance Staff Emission Control Technology Division Office of Air, Noise and Radiation U.S. Environmental Protection Agency Ann Arbor, Michigan ------- 2 TABLE OF CONTENTS / Page 1.0 SUMMARY AND CONCLUSIONS . 1 2.0 BACKGROUND 2 3.0 TRAINING COURSE USED 3 4.0 FACILITY TRAINING 3 5.0 STUDY DESIGN 5 5.1 Vehicle Selection and Recruitment 5 5.2 Vehicle Testing 6 5.3 Repairs by Participating Facilities 7 6.0 RESULTS 7 6.1 Emissions and Fuel Economy 7 6.2 Cost and Amount of Repairs by Commercial 15 Facilities 6.3 Quality of Repairs 16 6.4 Repair Effectiveness by Facility Type 24 7.0 DISCUSSION AND CONSLUSIONS: POSSIBLE REASONS FOR THE OBSERVATIONS IN PORTLAND AND GENERAL IMPLICATIONS FOR MECHANIC TRAINING BENEFITS 7.1 Possible expectations of training, actual 25 results and possible explanations 7.2 General implications for mechanic training 26 benefits and limitations of this study 8.0 MECHANIC TRAINING EMISSION REDUCTION BENEFITS CONTAINED IN MOBILE2 APPENDIX - Vehicle Test Fleet 33 ------- 1.0 SUMMARY . This report describes a study to determine if a short, practical training course for emissions repairs has a supplementary emission reduction and fuel economy benefit to an Inspection and Maintenance programi The study was performed in the early part of 1980 in Portland, Oregon. Two matched samples of 21 vehicles each were sent to ten repair facilities before and after the facilities received training. The experiment was blind in that the ten repair facilities did not know that the study was in progress or that the 42 vehicles were in any way connected with EPA. The training course used was a 16 hour course in repairing vehicles which fail an idle emissions test. Vehicles were tested before and after repairs on the Federal Test Procedure and various short tests for emissions and fuel economy. Contractor laboratory personnel then attempted to restore the vehicles to their specifications and vehicles were retested. Results indicate that: 1. Training in Portland did not produce greater emission reduction benefits from repairs than the emission reduction benefits without training. Similar emission reductions of approximately 41% HC and 53% CO on the Federal Test Procedure occurred from repairs both before and after training. 2. A 0.8% improvement in fuel economy from repairs (average of city and highway driving) was seen after training. This improvement was not statistically significant, however. 3. The average cost of repairs was nearly identical before and after training, about $19.50. 4. Several measures of the quality of repairs indicate a slight improve- ment with training. The quality of carburetor adjustments, which is highly important, did not improve with training, but spark timing adjust- ments and driveability improved slightly. 5. Emission reductions from repairs performed by different types of facilities (gas stations, independent garages and chain stores) did not differ significantly in emission reductions either before or after training. 6. Contractor repairs to the vehicles show that the potential exists for significant further emissions reductions beyond the reductions achieved by commercial facility repairs. This potential may not be realizeable in practice. Significant potential for further fuel economy improvement also exists. ------- There are two major reasons why this study may not have shown significant emission reduction benefits from training. First, the Portland idle emission standards are more stringent than almost any other I/M program will adopt, thus there is not as much "room" for improvement that training could accomplish in Portland, compared to other I/M programs in which the mechanics do not have to adjust the vehicles' emissions.so low. For example, the idle CO standard for most late model vehicles in Portland is 1.0% (including the allowed tolerance) and the average after repair idle CO level is 0.1%; in New Jersey the standard for those vehicles is 3.0% and the average after repair idle CO level is 0.8%. Second, the I/M program had been operational in Portland for over four years prior to the study. It is expected that during that time mechanics would have gained some on-the-job training and "grapevine" information which would lessen the effect of a training course. Also, the mechanics would have formed habits for repairing the emission failure vehicles which a short training course could not greatly change; for example, many facilities in Portland always perform, for a set fee, a "DEQ adjustment", meaning standard adjustments to pass the state idle emissions test. Additional discussion and conclusions are contained in Section 7.0. 2.0 BACKGROUND Numerous studies in the past have indicated that mechanic training is a very important part of an Inspection and Maintenance (I/M) program. In 1973 the National Academy of Sciences[1]* stated its concern that the service industry may not be able to adequately service cars from an emission control stand- point. Since that time many other studies have agreed that mechanic training is essential to an I/M program in order to achieve the full benefits, both in terms of emissions reduction and fuel economy improvement. This need can be seen by observing the effort the auto manufacturers put into training dealer- ship mechanics. A service training manager from one of the major manufactur- ers summarized the general need: "Engineers continually make refinements and improvements so technicians need continual refreshers to keep up with the changes in existing technology; it's a never-ending process."(2] Twenty-nine states will have operating I/M programs in 1987. States will receive "credits", representing expected emission reductions due to I/M, which they will use in 1982 to demonstrate compliance with the National Ambient Air Quality Standards. EPA estimated in 1978 that additional emission reductions could be achieved from a mechanic training program; many states are planning to institute training partly in order to claim more credits. Since training is a relatively low cost strategy, it is very attractive. The 1978 EPA credit estimates were based (in MOBILEl) on a mathematical model using emission results from a large number of in-use vehicles. The credits associated with mechanic training were based on the assumption that trained mechanics would be able to repair failed I/M vehicles to the Federal standards for which they were designed. This assumption had not been tested in a field experiment. Therefore, in 1979 EPA made plans for testing the effectiveness of a training course on emissions in an area which currently had I/M, but whose mechanics had little or no formal training in the area of emissions. Results from that study were to be used in a revision of MOBILEl, called MOBILE2. * Numbers in brackets designate references at the end of the report. ------- The Portland, Oregon area was chosen as the site to test the effectiveness of training. This was a logical choice for EPA since the area had an established I/M program with very little previous formal training, and EPA already had a contractor-operated emissions testing laboratory present. The contractor had experience in recruiting vehicles and testing them using a variety of tests including the Federal Test Procedure (FTP), the same procedure EPA uses to certify new cars. 3.0 TRAINING COURSE USED In order for emissions training to be widespread and accepted by a large number of auto technicians, it would have to be relatively short in length. EPA had a short, practical course designed by the National Center for Vehicle Emissions Control and Safety of Colorado State University (CSU) over the 1978-79 period. Design for this course began more than one year prior to its use in Portland and it had already been field tested, although CSU felt that minor refinements may still have been needed. The course was designed to be a maximum of 16 hours in length, which could, for example, be taught over several evening sessions. In this way mechanics would not have to take leave from work, a design which was considered essential to the course becoming widespread. The course mainly teaches the proper diagnosis of I/M failures; in other words, hov the mechanic should proceed in order to repair a vehicle which has failed the I/M idle test. The titles of each of the six units taught in the course follow. Unit Topics 1. I/M Programs and Vehicle Emissions; Short Tests. 2. Equipment Used (Infrared Analyzer). 3. Types of Emission Failures; General Troubleshooting Information. 4. Correction Procedures for Excessive HC Emissions. 5. Correction Procedures for Excessive CO Emissions. 6. Proper Carburetor Adjustment Procedures. 4.0 FACILITY TRAINING Five categories of repair facilities which perform work for emission repairs were designated. The original design called for the recruitment of three of each type, for a total of 15 facilities to be used in the study. Fifteen facilities were recruited, but five later had to be dropped due to low or no participation in the training, including the only new car dealership which had been signed up. (Dealership mechanics were generally not agreeable to the training, because they already attended many training sessions provided by the manufacturers.) The facility categories and the number of facilities of each that were actually used throughout the study are listed below. ------- Facilities Used No. 1. Gasoline-service stations s 2 2. Independent repair garages 2 3. Chain stores (e.g., Wards, Goodyear) 2 4. New car dealerships 0 5. Facilities advertising I/M emissions 4 repairs (one was a gas station, the other three independent garages). 10 Approximately 100 facilities were originally contacted. Facilities were told that EPA was sponsoring a pilot offering in the Portland area of a short course in emissions repair that had been developed by Colorado State University (CSU). The course was offered at no charge to mechanics. Incentives to the mechanics were free training, a certificate of completion, and a monetary incentive of $50 for completing the course. They were also told that we wanted to include only those facilities in which all tune-up mechanics could take the course. (The reason for this was to assure that test vehicles sent to the facilities would be repaired by participating mechanics.) Although this was not strictly adhered to for one facility, results from that facility were not used in the final analysis. All mechanics in the 10 facilities listed above received the training. Many facilities contacted were either not interested or could not get all the tune-up mechanics there to take the training. The EPA Project Officer also asked questions concerning previous training in order to identify facilities which had an unusually high amount of training and would therefore be unrepresentative of the norm. Nearly all facilities reported that their mechanics had at least some previous training, but none were judged as being outside the norm. The mechanics from the facilities used in the final analysis had varied experience and prior training. The median amount of tune-up experience was five years and the median amount of training was one prior course. The range of experience was from one to twenty-nine years and the range of the amount of courses taken was from none to seven (nearly half of the mechanics had previously taken one prior course). The mechanics were trained in February and March of 1980. Five classes at three community colleges were held, averaging 7 mechanics per class. Four of the classes were held in five evening sessions over two-week periods, and the fifth class was given on two consecutive Saturdays in all-day meetings. All but three of the mechanics who started attended all sessions. A total of 23 mechanics were trained from the 10 facilities used in the study. Their average test scores on the same written test at the beginning and end of the training were 71 and 90 respectively. CSU felt that a minimum final score of 80 should be attained by anyone who had successfully learned material in the course. All of the mechanics achieved scores of at least 80 at the end of the training. ------- Prior to the training, a two-day workshop was given by CSU for the instructors who would be teaching the classes. Orientation was given to familiarize the teachers with the course and the CSU instructional materials and to give an idea of how much time to spend on each section. EPA and CSU felt that this was necessary, because the instructors had not seen the course material and at that time there was also no mention of time to be spent on each section of the course - since the course covers a lot of material over a relatively short period of time, mention of the time to be spent on each section was more important than with most courses. Because teaching the course would be a new experience for the teachers, we felt that the workshop was necessary and would not give them more knowledge or expertise than teachers in an actual I/M mechanic training program would have after teaching the course once or twice. Some experience in teaching the course would be the normal situation to evaluate anyway, since it can be expected that in I/M areas which offer a lot of training, each teacher would hold several classes. 5.0 STUDY DESIGN 5.1 VEHICLE SELECTION AND RECRUITMENT Two groups of 1974-77 model year light duty passenger vehicles were used. The Scope of Work called for the two groups to match each other by vehicle type and emission failure type. Vehicle type includes model year, make, engine size, fuel system (1, 2, or 4 venturi carburetor or fuel injection) and transmission type. Emission failure type means the failure of HC, CO or both HC and CO, on the state idle emissions test. The first group of vehicles was sent to the facilities for repairs before training had occurred, the second group after training. Each facility was sent the same types of vehicles after training as had been sent before training. The pre-training group was selected in order to be representative of a failing fleet of vehicles. For the 1975-77 model years this was a simple matter. Another EPA study in Portland had already obtained a representative sample of failing vehicles which were used for this training study. Another approach was needed for the 1974 model year cars. First, a group of vehicles representing high sales was chosen. From this group were recruited enough willing owners of vehicles such that the assumed failure rate of 50 percent for that model year would yield the required number of failing vehicles, and they would be representative of a failing fleet. The post-training group of vehicles was recruited in order to match the first group. The pre-training group consisted of 43 vehicles. Due to several factors including four facilities dropping out of the training program and miscellan- eous vehicle problems, only 21 were used in the final match-up. Matching these vehicles exactly, in order to produce the post-training group, was somewhat difficult. In total, 55 vehicles were recruited for the second group. This number was large mainly for two reasons. First, EPA planned on there being 31 vehicle pairs and was recruiting vehicles accordingly. Second, matching exactly for emission failure type caused more vehicles to be recruited than could be used in the study. ------- 8 One of the 21 pairs used was not an exact match for emission failure type, but was considered acceptable. In that one case-the pre-training vehicle failed the state idle test for CO only, and the postftraining vehicle failed for both HC and CO. The HC level was not extremely high and no misfire was present, however, so that maintenance to reduce the CO level was all that was required in order to also reduce the HC level, the same type of maintenance the pre-training vehicle needed. Nine of the 21 pairs did not match for model year, but matched in everything else. 5.2 Vehicle Testing Owners brought their vehicles to the EPA contractor test laboratory, completed a questionnaire, and had no further involvement. Contractor personnel drove the vehicles to the state test lane for inspection. If a vehicle failed the state inspection test (SIT) for emissions, it entered the program. Otherwise it was returned to the owner. A series of tests was performed on each study vehicle under a minimum of three conditions: as received, after maintenance by the commercial repair facility and after restorative maintenance (RM) by the contractor. Each series of tests consisted of the following: 0 Diagnostic Inspection 0 Federal Test Procedure 0 50 MPH Cruise Test 0 Highway Fuel Economy Test 0 Four-Mode Idle 0 Loaded Two-Mode 0 State Idle Test at Laboratory Separate from the purpose of this study, tire pressures and spark timing at curb idle conditions were recorded during one or more of the series of tests. The purpose of the RM sequence was to return the vehicles to their specifica- tions concerning tune-up parameters and emission controls, within reasonable limits. The sequence was designed to follow diagnosis and repair information that was given in the training course, so that the trained mechanics in the study should have been able to perform them. No extremely expensive repairs were made, but repairs were more extensive than mechanics might usually perform. For example, no major engine work was performed even if diagnosed as probably being needed, however, carburetor repairs and replacements were made if diagnosed as being needed. In some cases, more than one RM sequence was performed. For the final matched list this occurred four times in the pre-training sample and once in the post-training. All of the additional RM sequences in the pre-training sample were performed due to a concern that carburetor idle mixture adjustments may not have been performed according to the specifications of the manufacturers. There had been a misunderstanding on the part of the contractor concerning this adjustment, which was easily cleared up. The one instance of additional repairs to a post-training vehicle was done because of driveability problems. ------- The problem was not serious, but was noticeable, and was solved by further adjustment and repairs. These additional repairs were acceptable to the study design since repair procedures do take driveability into consideration. 5.3 Repairs by Participating Facilities Contractor personnel, rather than the owners, drove the vehicles to the repair facilities. This was done so that the best documentation of vehicle repairs could be achieved, consistent instructions to mechanics could be given, and laboratory scheduling would be smoothest. The drivers acted as if they were the owners, simply requesting that the cars be repaired in order to pass the state emissions test. The state inspection sheet which gives the failure condition(s) was always in the car. No mention of a price limit was made. However, if asked, the driver would instruct the mechanic to perform the least expensive adjustments or repairs in order to simply make the car pass the emissions test. Due to the low average repair costs seen for I/M repairs in other studies, it is assumed that this low-cost instruction is commonly given by owners. 6.0 RESULTS Average emission changes due to repairs before and after training are an obvious highly important result, but is not the only important result. Also of importance in determining the degree of success of the training is the type and quality of repairs, cost of repairs, fuel economy changes, the emission reductions versus the potential reductions of the vehicles, individual facility performances, and more. The motivation of the type and amount of repairs must also be considered when determining the success of training. What may appear as lack of success by the training program may actually be due to a failure to change economic motivations, rather than a failure to impart new skills. For example, if low cost is always the top priority, repairs which are diagnosed as needed to restore the vehicle's emission control system to full function, but not necessary in order to pass the state emissions test, will not be done. 6.1 Emissions and Fuel Economy Average emissions and fuel economies are presented for six categories in Tables 1 and 2. The emissions are from the Federal Test Procedure (FTP) driving cycle and from the idle emissions test (performed at the laboratory). The fuel economies are from the FTP and the Highway Fuel Economy Test (HFET). Table 1 is for the before-training group of vehicles, Table 2 is for the after-training group. The FTP HC, CO and fuel economy levels are shown graphically in Figures 1-3. The percent and absolute changes in FTP emissions and fuel economies are presented in Tables 3 and 4. All changes are from the as-received condition. ------- As-Received After Repairs by Facilities After Repairs by Laboratory 10 Table 1 Before Training Emissions and Fuel Economy Levels (21 Vehicles) Idle Emissions** Federal Test Procedure (Using Garage- Emissions (grams per mile) Type Analyzer) HC CO NOx HC -(ppm) CO (%) 2.94 1.77 1.73 47.4 22.7 16.9 2.57 252 2.39 107 2.53 106 3.02 0.38 0.20 Fuel Economy*** (miles per gallon) FTP HFET 15.15 15.16 20.95 20.59 15.82 21.69 Federal/State* Standards (Average for all vehicles) 1.78 18.6 2.84 258 1.2 15 21 Table 2 After Training Emissions and Fuel Economy Levels (21 Vehicles) Idle Emissions** Federal Test Procedure (Using Garage- Emissions (grams per mile) Type Analyzer) HC CO NOx HC (ppm) CO (%) As-Received 3.33 After Repairs 1.92 by Facilities After Repairs . 1.64 by Laboratory 45.1 2.93 307 4.2 20.2 2.93 64 0.15 13.9 2.65 60 0.07 Fuel Economy*** (miles per gallon) FTP HFET 14.91 15.23 20.58 20.32 15.41 21.03 * Federal emission standards are based on the 1975 FTP - the 1974 Federal emission standards were converted in order to average them with the 1975-77 standards. The fuel economies are based on the new car certification values; fuel economy stan- dards for a few vehicles were estimated. "State standards" refers to the state idle test cutpoints. ** From the second idle portion of the SIT taken at the contractor's laboratory. *** The conclusion stated in Section 1.0 that training produced a 0.8% improvement in fuel economy is based on changes in combined FTP (city) and HFET (highway) fuel economy. ------- 11 u.o 3.5 -3.0 a. 2.5 DC UJ en ac cl.O 0.5 0.0 Figure 1 HTDROCflRBONS MEflN LEVELS BEFORE flND flFTER TRfllNING (21 Vehicles) 2.9^ 3.33 fl/R fl/M R/M BEFORE TRAINING fl/R fl/M R/M flFTER TRfllNING fl/R - flS RECEIVED fl/M - flFTER MflINT, R/M - RESTORflTIVE ------- 12 60 50 Figure 2 CflRBON MONOXIDE HERN VALUES BEFORE RNO RFTER TRfllNING (21 Vehicles) = 30 oc IU ti- er, 20 ac ec cc O 10 47. 22.7 16.9 US. 1 fl/R fl/M R/M BEFORE TRAINING fl/R fl/M R/M flFTER TRfllNING fl/R - flS RECEIVED fl/M - flFTER MfllNT. R/M - RESTORflTIVE ------- 13 21.0 Figure 3 CITY FUEL ECONOMY HERN VflLUES BEFORE flND flFTER TRfUNING (21 Vehicles) 18.0 . 15.0 ; £12.0 t- u. o 9.0 e_ 6.0 3.0 0.0 15.1 15.2 15.8 14. fl/R fl/M R/H fl/R fl/M R/H BEFORE TRfllNING flFTER TRRINING fl/R - RS RECEIVED fl/M - flFTER MfllNT. R/M - RESTORflTIVE ------- BEFORE TRAINING After Repairs by Facilities After Repairs by Laboratory AFTER TRAINING After Repairs by Facilities After Repairs by Laboratory BEFORE TRAINING After Repairs by Facilities After Repairs by Laboratory AFTER TRAINING After Repairs by Facilities After Repairs by Laboratory 14 Table 3 Percent Changes In Emissions and Fuel Economy From Repairs*. (21 Vehicles) Federal Test Procedure HC CO NOx Fuel Economy** FTP HFET -40% -52% -41% -2% +0.1% +4.4% -1.7% +3.5% -42% -55% -51% -69% -10% +2.1% +3.4 -1.3% +2.2% Table 4 Absolute Changes In Emissions and Fuel Economy From Repairs* (21 Vehicles) Federal Test Procedure (grams per mile) HC CO NOx -1.17 -24.7 -0.18 -1.21 -30.5 -0.04 Fuel Economy** (miles per gallon) FTP HFET +0.01 -0.36 +0.67 +0.74 -1.41 -24.9 0.00 -1.69 -31.2 -0.28 +0.32 -0.26 +0.50 +0.45 * All changes are relative to the as-received levels. ** The conclusion stated in Section 1.0 that training produced a 0.8% improve- ment in fuel economy is based on changes in combined FTP (city) and HFET (highway) fuel economy. ------- 15 Several statistical tests were used to analyze the data in Tables 1-4. The standard two-sample t-test and the paired t-test are the two tests most appropriate (the latter pairs the matching vehicles of the two groups before comparison of the groups). These tests all indicate that there is no significant difference (at even the 90% confidence level) between the emission and fuel economy levels of the before and after-training groups at any of the repair stages. In other words, the two groups were so similar in both the as-received condition and the after maintenance condition, whether from the trained or the untrained facilities, that the differences which appear in the tables could easily be due to random variations. The latter statement holds true whether the comparison is made using the after-maintenance levels, or the changes due to maintenance (the ideas of Tables 2 and 4). The as-received emission levels compare favorably to samples of vehicles in other test programs, thus giving credibility to the representativeness of the vehicles used in the training study. Comparisons of 1975-1977 model year vehicles from this training study and the EPA Portland Study (which was an evaluation of the Portland I/M program) are given in Table 5. Both the aatched, paired sample of 34 vehicles (17 vehicles each in the before and after-training samples were 1975-77 model years), and all 75 vehicles from the mechanic training evaluation which are 1975-1977 model years are shown. (The Portland Element III program used only 1975-77 model year vehicles, which is the reason for selecting only these years from the training study for compar- ison.) The slightly lower HC and CO emissions of the Portland Element II and III vehicles is expected, since these vehicles were tested when they were younger and had less mileage than the vehicles in the Mechanic Training study. ------- 16 Table 5 / As Received Status of Mechanic Training Evaluation Vehicles and Portland Study Vehicles 1975-1977 Model Years Only Federal Test Procedure Study Portland Element II & III Mechanic Training Evaluation (Matched) Mechanic 75 44,931 3.13 44.0 2.67 Training Evaluation (All) N 207 34 Odometer 31,100 41,722 HC 2.69 3.17 CO 39.7 47.3 NOx 2.72 2.53 ------- 17 6.2 Cost and Amount of Repairs by Commercial Facilities The average maintenance costs were similar for the two fleets and are listed in Table 6. The types and amounts of repairs for the two fleets were also very similar. The repairs performed are listed in Table 7. Table 6 Average Cost of Repairs by Facilities Before Training $19.14 After Training $19.81 Table 7 Type and Amount of Repairs by Facilities Item Spark Plugs Carburetor Carburetor Idle Speed Timing Air Filter Oil EGR System Choke Vacuum Hoses Type of Repair Amount Before Training Amount After Training Amount By Laboratory* Replace 2 Adjust 20 Overhaul 1 Adjust 20 Adjust 2 Replace 0 Change 0 Repair/Replace 0 Repair/Adjust 0 Reroute/ Replace/ Repair ? 1 20 1 20 3 1 1 0 0 5 17 4 18 5 6 0 8 2 * Average for the before and after training groups. ------- 18 6*3 Quality of Repairs > The contractor performed various inspections ,.of the vehicles after each test sequence. These inspections gave an evaluation of the maintenance primarily in the areas of the carburetor and ignition timing, which are the most frequent maintenance items performed. Three measurements were taken which reflect the quality of maintenance to the carburetion system: (1) the position of the carburetor idle mixture screw(s) relative to the laboratory setting, (2) the maximum amount of engine speed increase at idle due to the addition of propane gas, called "propane gain", and (3) the idle emission levels, particularly idle CO. For commercial repairs, each of these measurements can be compared to the laboratory measure- ments for determining maintenance quality. (However, this method assumes that the laboratory adjusted the carburetors perfectly which may not always have been the case.) The carburetor idle mixture screw positions were measured in terms of the number of 1/4 turns necessary to reach the seated position. Propane gain correlates with the leanness of the air-fuel mixture; the greater the engine speed increase with propane added, the leaner the setting. The idle CO is important, but less so than the two measurements previously stated; this is because nearly all of the vehicles have catalytic converters which substantially decrease HC and CO resulting in readings at the tailpipe that do not reflect well the emissions from the engine itself. Results of these three measurements, shown in Tables 8-10 do not show any trend in the quality of carburetor adjustments after training. Concerning Table 10, training had no noticeable effect on the distribution of idle CO levels. The mean levels and standard deviations are very similar, when leaving out the one noted vehicle in Table 10. Also, 17 of 21 vehicles in each group were adjusted to between 0.0 and 0.3% CO, confirming the similarity in distribution. ------- 19 Table 8 Evaluation of Carburetor Idle Mixture Screw Settings by Commercial Facilities A. Number of 1/4 Turns Different from Contractor Setting* (Positive numbers mean the commercial settings were richer than the contractor settings.) BEFORE TRAINING AFTER TRAINING Mean .45 .24 Standard Deviation 1.7 1.5 B. Absolute Value of Number of 1/4 Turns Different from Contractor Setting** BEFORE TRAINING AFTER TRAINING Mean 0.9 1.3 Standard Deviation 1.5 1.4 C. Frequency of Commercial Settings In Directions Relative to Contractor Settings. BEFORE TRAINING AFTER TRAINING Richer 11 11 Same 15 12 Leaner 7 10 * Example: If one screw were 2 1/2 turns out (from the seated position) after commerical repair and only 2 turns out after contractor repair, the number of 1/4 turns different would be +2. ** Example: If the number of 1/4 turns different for one screw was +2 and another was -2 the simple mean would be zero; the absolute mean would be 2, though, because only the numbers are considered in computing absolute values, not the signs. Using absolute values results in a higher mean for this reason. ------- 20 Table 9 v Evaluation of Carburetor Idle Mixture Settings Using Propane Gain Measurements (Idle Speed Increase Due to Addition of Propane Gas) Standard Deviation Mean* Mean of the Increase Difference Differences (rpm) BEFORE TRAINING Commercial Facilities 144 Contractor 130 -14 81 AFTER TRAINING Commercial Facilities 104 Contractor 112 8 82 * Note that although the mean increases from the commercial facilities and the contractor are very similar, the standard deviation is large, showing that for individual vehicles the propane gains from the commercial facilities were often quite different from the contractor setting. The differences were positive one-half of the time and negative the other half, both before and after training. Table 10 Idle CO Settings (%} Standard Mean Deviation BEFORE TRAINING Commercial Facilities 0.38 (.18)* 0.92 (.29)* Contractor 0.20 0.39 AFTER TRAINING Commercial Facilities 0.15 0.24 Contractor 0.07 0.17 * Excluding one vehicle in the group which had a very high CO reading results in the amounts shown in parentheses. The facility apparently adjusted this vehicle to pass the state I/M test, got the certificate of compliance at the state I/M lane, and then readjusted the vehicle before returning it to the "owner". The assumed reason was that the vehicle had poor driveability when adjusted to pass the DEQ. With more thorough repairs the contractor achieved acceptable driveability and low emissions, however. ------- 21 The contractor measured three types of ignition timing which are useful in evaluating the quality of repairs to this system by the commercial facili- ties. The three types are basic, mechanical, and total advance. Basic timing is the initial setting, usually made at idle. Mechanical (or centrifugal) advance is the increase in advance when the engine is operated at higher speed. Total advance measures the first two types plus engine vacuum advance, and is also measured at a moderately high engine speed. Since the basic timing is the only one which affects idle emissions, that will be the main type of timing noted here; it also affects fuel economy significantly and, is an important measure of the quality of repairs for that reason. Table 11 shows the number of vehicles with basic timing retarded before and after repair. Table 11 Vehicles With Basic Timing Retarded By More Than 2° BEFORE TRAINING AFTER TRAINING As-Received 2 1 After Repair by Commercial Facilities 5 1 In the before training group, the two vehicles in Table 11 with retarded timing as-received were unchanged by repair. Therefore, a total of three vehicles were retarded by repair. In the after training group, the one vehicle with retarded timing as-received was corrected by repair and one other vehicle which had timing properly set as-received was retarded by the repair facility. Considering both mechanical and total advance together, two vehicles less in each group were retarded (by more than 4°) after repair, thus some correct repairs were mada in each group. It was not apparent what repairs were made to the before-training cars to correct the mechanical and total retard, but in the after-training group it was evident that two cars had their vacuum advance mechanisms fixed. A fourth measure of the quality of repairs can be made by looking at the type of repairs performed for the different types of failures. A previous concern with I/M repairs was that mechanics did not know what the possible causes of HC and CO failures were and would perform some needless repairs. One particular problem cited was that of performing repairs to the ignition system, such as spark plugs and wires, distributor cap, rotor, etc., on a vehicle which failed for CO only. The repairs just mentioned would be unnecessary because they do not have an effect on CO. Earlier EPA studies of the Portland I/M program showed that there was a significant amount of the'se unnecessary repairs performed. In this study almost no unnecessary repairs were noted. In the pre^training group, one vehicle which failed for CO-only received a timing adjustment. This was unnecessary, but is a reasonable and often recommended adjustment to make when doing carburetor adjustments, and therefore is of almost no ------- 22 concern. No other adjustments or repairs to any part of the ignition system were performed on any of the other vehicles which failed for CO-only, showing good diagnosis even without training. Similar results were seen in the post-training sample. Only one unnecessary, repair item was performed, the replacement of spark plugs to a vehicle which failed for CO-only. The majority of the vehicles failed for both HC and CO and therefore no judgements could be made as to the types of repairs performed except to say that they appeared to be correct. The above refers to only the paired vehicles used in the analysis. Inspecting the repairs of all other vehicles in the study, 'there were two instances in the pre-training sample of expensive misdiagnoses, but none in the post-training sample. One pre-training vehicle was diagnosed by a mechanic as needing a new timing chain and gears with an estimated repair cost of $220. This vehicle was sent to another facility and received a carburetor overhaul which apparently was what it needed, although this cost $111. A second vehicle was adjusted to pass the idle test, but the mechanic thought it might have a bad catalytic converter. The laboratory found that the real problem was leaking intake manifold gaskets; in all fairness, however, this was a difficult repair case with the laboratory reportedly spending 6 hours in repairs and noting that one cylinder had low power which could have been a valve problem. One expensive repair that the faciities often did not diagnose as necessary, but which the laboratory found necessary, was carburetor overhauling. Of all 98 cars in the study, 25 received carburetor overhauling or replacements. Of these 25, only 8 were diagnosed as being needed by the facilities (in 3 cases outside of the matched, paired groups, EPA did not let them do the work even though the laboratory later agreed that the carburetor needed to be over- hauled). Apparently, repair facilities do not generally diagnose carburetor overhauling unless they cannot get the vehicle to pass the idle test. This is expected with the assumption that owners usually try to obtain the lowest cost repairs which will allow their vehicles to pass the test. As expected, vehicles which need carburetor repairs, but do not receive them, still have high FTP emissions after repairs even though they pass the idle test. Table 12 shows the FTP and idle emission levels at the different repair stages of the 17 vehicles which needed carburetor work (as determined by the contractor), but did not receive work by the repair facilities. Only 8 of the 17 were from the final matched samples (4 from each group in A and B). As can be seen, FTP HC and CO remain quite high after repairs by the facilities. One vehicle (not from the matched pairs) also had very high emissions after laboratory repairs. The carburetor was overhauled, but the repair had no effect on FTP HC or CO emissions. The contractor should have had further repairs performed, because CO emissions remained high at the high speed portion of the state idle test, which is an indication that there is still a carburetor problem. This is a diagnostic check that was used in other cases, but was not used on this vehicle after the carburetor overhaul, for unexplained reasons. Without this one vehicle the mean FTP HC and CO for the "combined" group (after laboratory repairs) would have been similar to the mean for all the other vehicles after repairs by the laboratory (see Tables 1 and 2). ------- 23 Table 12 Emissions and Fuel Economy of Vehicles Needing Carburetor Repairs But Not Receiving Them By The Commercial Facilities* Idle Emissions** Federal Test Procedure (Using Garage- Fuel Economy Emissions (grams per mile) Type Analyzer) (miles per gallon) BEFORE TRAINING (n=7) As-Received After Repairs by Facilities After Repairs by Laboratory AFTER TRAINING (n=10) As -Received After Repairs by Facilities After Repairs by Laboratory COMBINED (n=17) As-Received After Repairs by Facilities After Repairs by Laboratory HC 4.08 (3.88) 2.97 (2.44) 2.26 (1.75) 3.78 2.80 1.72 3.91 (3.83) 2.87 (2.71) 1.94 (1.73) CO 72.1 (65.0) 57.5 (45.4) 32.7 (19.0) 63.5 46.4 17.5 67.0 (64.0) 51.0 (46.1) 23.8 (18.1) NOx 2.37 (2.55) 2.11 (2.29) 2.46 (2.63) 2.37 2.29 3.22 2.37 (2.44) 2.22 (2.29) 2.91 (3.00) HC (ppm) CO (%) 334 (366) 169 (162) 120 (118) 365 73 75 352 (365) 112 (106) 94 (92) 2.8 (3.1) 0.70 (0.72) 0.33 (0.24) 2.7 0.20 0.09 2.7 (2.8) 0.41 (0.40) 0.19 (0.15) FTP 15.58 (16.46) 15.66 (16.89) 15.98 (16.98) 14.96 13.90 15.53 15.21 (15.49) 14.57 (14.88) 15.71 (16.04) HFET 20.69 (21.52) 20.66 (22.05) 21.55 (22.62) 20.30 18.63 21.64 20.46 (20.74) 19.42 (19.78) 21.60 (21.99) * Emissions and fuel economies without the one vehicle that still had a carburetor problem are shown in parentheses. ** From the second idle portion of the SIT taken at the contractor's laboratory. ------- 24 A final measurement which would reflect the quality of repairs was a drive- ability evaluation at each repair stage. The main problems looked for were engine surging, stumbling, backfiring, stretchiness, misfiring and run-on ("dieseling"). The number of vehicles having at least one problem was greater before training than after training, as shown in Table 13. The reader should be aware of the fact that this evaluation was quite subjective and that only a trend can be deduced from the results, i.e., that driveability remained good after repairs by trained mechanics, whereas it was worse in some cases after repairs by untrained mechanics. The after repair frequencies of problems are not statistically different. Table 13 Number of Vehicles With Driveability Problems After Repairs After Repairs As-Received by Facilities by Laboratory BEFORE TRAINING (n=21) 2 63 AFTER TRAINING (n=21) 2 1 2 ------- 25 6.4 Repair Effectiveness by Facility Type As was mentioned in Section III,, four types of repair facilities were used: gasoline-service stations, independent repair garages, chain stores, and facilities advertising I/M repairs. In order to increase the number of vehicles in some of the categories for statistical comparison purposes, the facilities in the "advertising" category will be put into the appropriate other three sections. This will particularly help the "independent" category which formerly had only two facilities and three vehicles sent to it, because three of the four facilities which advertised were independents (the fourth advertiser was a gas station). The FTP HC and CO emissions, and fuel economies are shown for each of the categories at the different repair stages in Table 14. The two-sample t-test and the paired t-test show that there is no significant difference between any of the different facility types' repairs before and after training. The tests were performed for both the after maintenance levels and the changes due to maintenance. Table 14 Effectiveness of Different Repair Facility Types BEFORE TRAINING As After % Received Repair Change AFTER TRAINING As After % Received Repair Change Gas Stations (n=7) FTP HC CO mpg* Independents (n=8) FTP HC CO mpg* Chain Stores (n=6) FTP HC CO mpg* 2.68 47.0 15.30 3.17 44.1 14.65 2.93 52.4 15.68 1.65 24.5 15.35 2.06 20.1 14.65 1.51 24.1 15.67 -38% -48% 0.3% -35% -54% 0 -48% -54% 0 2.85 35.7 15.72 3.79 47.2 14.32 3.26 53.3 14.83 1.25 9.38 15.88 2.16 22.1 14.76 2.38 30.2 15.14 -56% -74% + 1.0% -43% -53% +3.1% -27% -43% +2.1% * City fuel economy. Highway fuel economy is not shown by facility type in this table but is shown in Tables 1-4. The conclusion in Section 1,0 that training produced a 0.8% improvement in fuel economy is based on combined city/highway fuel economy. ------- 26 7.0 DISCUSSION AND CONCLUSIONS: POSSIBLE REASONS FOR THE OBSERVATIONS IN PORTLAND AND GENERAL IMPLICATIONS FOR MECHANIC TRAINING BENEFITS The results of this study showed small and statistically insignificant incremental benefits due to training in the important areas of emissions and fuel economy. This conflicts with the higher expectations of EPA, as well as the expectations of many people in state agencies, industry, and the commercial repair industry. This section will attempt to resolve this apparent conflict. Part A will discuss the optimum results of training, the actual results, and the possible explanations for the differences between the expectations and the results. These three topics will cover several important areas in the training, e.g., diagnosis of the cause of the I/M failure. Part B will discuss the implications that this study has for full-scale mechanic training programs conducted as part of an operating I/M program and the limitations of this study in determining the benefits of training. 7.1 Possible expectations of training, actual results and possible explanations 7.1.1 Maximum expectations of what the training should accomplish The ideal situation following an I/M mechanic training program would be that all diagnoses and repairs which affect emissions and fuel economy are completely accurate, no vehicles are left unrepaired to such an extent that they are unable to pass the I/M test, and vehicle driveability is good. The only repairs which are performed are ones which are in fact necessary to correct an emissions or fuel economy problem; wasteful replacement of parts that are still good would be eliminated. All repairable emissions problems would be corrected, not just those which are directly causing the I/M test failure. The entire vehicle emission control system would ideally be working at its full function after repair. The emission levels of individual repaired vehicles would therefore be close to the Federal Test Procedure certification standards, and fuel economy would be optimal. Most I/M areas will have cost waivers. In areas which do allow cost waivers, the number of waivers would ideally be less with trained mechanics due to proper diagnoses and repairs resulting in lower repair costs (assuming that the repairs necessary to pass the I/M test are performed first). Also, fewer I/M-avoidance measures (registering a vehicle in a non-I/M area, fraud, cheating) would occur, contributing to lower fleet emission levels. The acceptable driveability of vehicles after repair would result in fewer vehicles receiving readjustments after repair, thus further contributing to low fleet emission levels. ------- 27 7.1.2 Actual results The repair facilities appeared to perform nearly identical proper diagnoses both before and after training. In the pre-training sample there were only two cases of mis-diagnoses and none in the post-training sample; however since problems which are difficult to diagnosis are rare, the mechanics were not tested rigorously on their diagnosis abilities. Repairs were generally correct, but not entirely accurate, resulting in similar emissions and fuel economies before and after training. Judging from the emissions and fuel economies achieved by the contractor personnel, significant improvement could be made in the accuracy and/or quantity of repairs by commercial facilities. The commercial facilities did not correct all of the emissions problems that were in fact repairable. No vehicles were left unrepaired to such an extent that they would not pass the I/M retest, either before or after training. However, one instance of a facility readjustment occurred in the before-training sample. The readjustment was apparently due to poor driveability. It is not possible to determine if this would have recurred after training if the same circumstances had appeared. 7.1.3 Possible explanations Mechanics in the Portland area had been familiar with I/M repairs for a few years. Through experience on-the-job they seem to have learned to diagnose the common problems of failing vehicles well.. Unusual problems continue to give them difficulty, however, which result in some incorrect diagnoses. Concerning repair procedures, mechanics learned the correct procedures (particularly carburetor adjustment procedures) well in the training, but there is no evidence in the FTP or other data that they took the time to do them precisely at the repair facilities. This is partly due to the mechanics being satisfied with adjustments which enabled the vehicles to pass the I/M test with a margin of safety and fairly good driveability. Although the training course did try to teach mechanics to repair other emissions problems not necessary to pass the I/M test, not enough incentive existed for them to do so. The overriding incentives from owners seem to be low cost and acceptable driveability. Also, due to competition, repair shops want to have low average repair charges to their customers and hope to have a reputation for inexpensive repairs. 7.2 General implications for mechanic training benefits and limitations of this study Little Likelihood of Complete Repair - In order to get the vehicles' emission control system as close as possible to a fully functional condition, the contractor's laboratory personnel often spent more than one hour in diagnoses and repairs. Because of the substantial extra cost necessary for these repairs, it is doubtful that a training program would ------- 28 ever result in commercial mechanics being as thorough. A legal require- ment plus a large enforcement effort might be successful in mandating very thorough repairs of all repairable problems, but this would likely be an unacceptable option to all I/M areas. Learning Curve - There may exist a learning curve such that mechanics will have learned as much after a period of time of on-the-job training as they would have with a formal training course. Consider the hypothetical curves in Figure 4. The top curve may represent the knowledge gained when formal training is given to mechanics in the first year of an I/M program. That curve continues to rise due to continued knowledge gained through experience, i.e., on-the-job type training. The bottom curve may be the learning associated with on-the-job training (and no formal training). In this hypothetical case, the difference between the curves is greatest during the first year and then decreases steadily until the third or fourth year at which time they are approximately equal. The value of formal training is the area between the two curves and may translate, to some extent, into emissions benefits during the first few years. Since the formal training course in Portland occurred after I/M had been in effect for about four years, the value of early training would not have been detected. The implication is that in any I/M program, in the early years formal training will (1) reduce emissions and repair costs, (2) reduce the number of cars which are not successfully repaired at first attempt and must therefore get re-repaired (the ping-pong problem) or obtain waivers, and (3) prevent cases of poor driveability after repairs with subsequent readjustments. The results of (2) and (3) would lead to further emissions benefits during the early years. The effect of training in the later years of an I/M program appears to depend on the stringency of the cutpoints used in the inspections. Effect of Stringent Cutpoints - In mature I/M programs with cutpoints like Portland's, there appears to be no emissions, cost or driveability benefits from training. The mechanics may perform slightly more accurate repairs, but if so, the difference in terms of emission levels after repair appears to be minimal. Effect of Lenient Cutpoints - In I/M programs with more lenient cutpoints than in Portland the implications are less clear. One possibility is that the learning curve mentioned above will occur, with associated temporary benefits from formal training. Other than that, mechanics may not perform any better than they would have after a few years of on-the-job training, and at that point in time, trained mechanics will not do any better than untrained mechanics. Mechanics may not adjust cars to a point far below the cutpoints, but will stop at a margin of safety point with which they are comfortable. ------- 29 Another possibility is that mechanics can be convinced to adjust the vehicles to far below the cutpoints, to about the same levels as Portland mechanics achieve (with the same FTP emissions). No extra time or equipment is needed, making this idea plausible. Evidence from Portland shows it is easily possible and can be done with low repair costs. If this possibility occurs, there will be a permanent benefit from formal training. EPA believes that the second possibility is likely to occur. However, only a training experiment in an I/M area with cutpoints much higher than in Portland would verify this. ------- 30 EMISSION REPfUR KNOWLEDGE VERSUS TIME (Conceptual Diagram) UJ o o UJ o z ae UNTRflJNED TRfllNED TIME (YEflRS) ------- 31 8.0 MECHANIC TRAINING EMISSION REDUCTION BENEFITS CONTAINED IN MOBILE2 Based on this study of mechanic training, I/M emission reduction benefits associated with mechanic training were derived and are available in MOBILE2.. As would be expected from the above analyses, the incremental benefits due to mechanic training are substantially smaller than those appearing in MOBILE1. Generally, the assumption used in the derivation of MOBILE2 was that the after-maintenance mean FTP levels are approximately equal to those seen in this mechanic training study if Portland short test cutpoints are assumed. If the cutpoints differ from Portland's, an adjustment is made. For a full explanation of the derivation, see the documentation for derivation of pre-1981 I/M credits[3]. Mechanic training only effects I/M benefits for pre-1981 model year cars in MOBILE2. In brief, it is felt that 1981 and later model year cars will tend to experience component failure instead of the maladjustments typical of pre-1981 model year cars. Thus, a car of advanced technology experiences quantun instead of continuous emission reductions. For more detail see the documentation for derivation of 1981 and later I/M credits[4]. Table 15 gives examples of the incremental fifth-year benefits due to mechanic training as seen in MOBILE2. These examples show typical results of 0-2 percent HC and 0-6 percent CO reductions associated with emissions from pre-1981 model year cars. After averaging these effects with the portion of the fleet experiencing no effect from mechanic training, incremental benefits due to mechanic training are typically 0-1 percent for HC and 0-4 percent for CO. Table 15 MOBILE2 Incremental Benefits Due to Mechanic Training Start I/M 1/1/83 Evaluate 1/1/88 Pre-1981 Pre-1981 Total Stringency Fleet* Fleet** HC CO HC CO 20 05 03 30 2614 40 26 14 * Incremental benefits in percent reduction to pre-1981 model year emission factors. ** Incremental benefits in percent reduction to total fleet emission factors assuming a 50% Identification Rate for 1981 and later model year cars. ------- 32 References 1. National Academy of Sciences, Report by the Committee on Motor Vehicle Emissions, February 12, 1973, pp 74-75. 2. Automotive News, December 19, 1978, pp 18-19. 3. "Derivation of I/M Benefits for Pre-1981 Light Duty Vehicles for Low Altitude, Non-California Areas", EPA-AA-IMS/81-4 (In preparation at the time of this report). 4. "Derivation of I/M Benefits for Post-1980 Light Duty Vehicles for Low Altitude, Non-California Areas", EPA-AA-IMS/80-8, January 1981. ------- 33 APPENDIX Vehicle Test Fleet ------- Vehicle List Pre-Training Vehicles Used For The Final Matched Analysis Vehicle # Make CID CYL CARS TRANS. AIR ODOMETER I/M FAIL 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 60506 40602 42001 61057 42002 61002 61006 62002 62102 60151 61000 41501 71015 71105 52057 51016 52018 52019 72111 72005 71120 Chev Olds Ford Ford Ford Chev Chrys Ford Chrys Chrys Chev Chrys Pont Ford Chrys Chev Chev Chrys Chrys Chrys Ford 250 350 302 250 302 305 318 140 318 318 305 225 301 140 225 350 140 225 318 225 250 6. 8. 8. 6. 8. 8. 8. 4. 8. 8. 8. 6. 8. 4. 6. 8. 4. 6. 8. 6. 6. 1-V 4-V 2-V 1-V 2-V 2-V 2-V 2-V 2-V 2-V 2-V 1-V 2-V 2-V 1-V 2-V 2-V 1-V 2-V 1-V 2-V AUTO AUTO AUTO AUTO AUTO AUTO AUTO MANUAL AUTO AUTO AUTO AUTO AUTO AUTO AUTO AUTO MANUAL AUTO AUTO AUTO AUTO NO NO NO YES NO NO NO YES NO NO NO NO NO NO NO NO NO NO YES YES NO 34615 42503 74020 25541 75391 47364 30329 75065 51893 25304 93687 63158 41184 38866 42182 51366 49444 75571 33306 24192 20676 HC, CO CO HC, CO CO HC, HC, HC, CO HC, CO HC, CO HC, HC, ... -- JIG, "HC, CO HC, HC, CO CO CO CO CO CO CO CO CO CO CO CO CO > X) •a 0> 3 a X ------- Vehicle List Post-Training Vehicles Used For The Final Matched Analysis Vehicle # Make CID CYL CARB TRANS. ATR? ODOMETER I/M FAIL 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 52001 40605 42003 60568 42005 60416 62201 70398 60302 60507 76012 41503 70262 61101 60584 60308 60385 60388 51055 70287 70508 Chev Olds Ford Ford Ford Chev Chrys Ford Chrys Chrys Chev Chrys Pont Ford Chrys Chev Chev Chrys Chrys Chrys Ford 250 350 302 250 302 305 318 140 318 318 305 225 301 140 225 350 140 225 318 225 250 6. 8. 8. 6. 8. 8. 8. 4. 8. 8. 8. 6. 8. 4. 6. 8. 4. 6. 8. 6. 6. 1-V 4-V 2-V 1-V 2-V 2-V 2-V 2-V 2-V 2-V 2-V 2-V 2-V 2-V 1-V 2-V 2-V 2-V 2-V 2-V 1-V AUTO AUTO AUTO AUTO AUTO AUTO AUTO AUTO AUTO AUTO AUTO AUTO AUTO MANUAL AUTO AUTO AUTO AUTO AUTO AUTO AUTO NO NO NO YES NO NO NO YES NO NO NO NO NO NO NO NO NO NO NO YES YES 58393 56942 32763 39789 95156 46938 26254 58393 37182 44303 36401 35147 42703 23668 32107 29508 61747 30424 27652 45990 16517 HC, CO CO HC, CO HC, HC, HC, HC, CO HC, CO HC, CO HC, HC, HC, HC, CO HC, HC, CO CO CO CO CO CO CO CO CO CO CO CO CO CO •§• *o m « a. H» X ------- |