United States Environmental Protection Agency Office of Air and Radiation EPA-AA-EPSD-I/M-94-8 Office of Mobile Sources August 1994 Ann Arbor, Michigan 48105 Air vvEPA Audit of the Enhanced I/M Program in Maine ------- Executive Summary On August 9, 10, and 11, 1994, a four person EPA audit team from the National Vehicle and Fuels Emission Laboratory in Ann Arbor, Michigan, conducted a brief audit of the Maine Car Test Program. The purpose of this audit was not to conduct a comprehensive or in-depth review of every aspect of the program but, rather, to evaluate the technical and organizational aspects of the program. The Maine inspection program is the first to employ IM240, evaporative system purge, and evaporative system pressure tests on a network-wide basis. The Maine Car Test program was found to be technically well designed and implemented. Minor technical problems were observed, however, none are likely to result in false test failures. The major area that needs to be addressed at this time is public interface, especially revising the test report and other information provided to motorists that fail the test. Minor refinements are needed in the testing process, and procedures for suspending evaporative system tests when pattern failure problems are encountered are needed Additional informational training for the repair industry would be useful in helping insure efficient and effective communications and repairs, although the data on retest pass rates to date do not indicate a major problem with repair effectiveness. Introduction From August 9-11, 1994, a four person EPA audit team from the National Vehicle and Fuels Emission Laboratory in Ann Arbor, Michigan, conducted a brief audit of the Maine Car Test Program. The purpose of this audit was not to conduct a comprehensive or in-depth review of every aspect of the program but, rather, to evaluate the technical and organizational aspects of the program. The Maine inspection program is the first to employ IM240, evaporative system purge, and evaporative system pressure tests on a network- wide basis. These tests are being implemented in enhanced I/M programs throughout the country and Maine's experience will help those that follow to refine the design and operation of the enhanced I/M tests. The audit team visited four stations in the seven station network: Westbrook, Kennebunk, Lewiston, and Topsham. The first three of these stations are the largest volume stations in the system. The audit team focused on observing the testing process, monitoring inspector performance, assessing equipment and quality control, and evaluating the overall system. General Description of the Test Process The testing process in the Maine program uses a three position system that begins when the motorist pulls into the lane. At the first position, an inspector greets the motorist and obtains basic information and the vehicle registration. This information is used to call up the pre-existing vehicle record, if available, or to create a new record for the vehicle. The inspector determines whether this vehicle is scheduled for an inspection and, if not, asks the motorist if he or she would like a voluntary test. If the information collection is occurring in the lane (as opposed to outside the lane in the queue, when other cars occupy the test positions in the lane) the vehicle is turned off and, on 1981 and newer vehicles, another inspector prepares the vehicle for pressure testing by removing the vent lines from the evaporative canister under the hood and attaching a pressure test hose and for the purge test by connecting two purge hoses between the canister and the engine. A swinging arm is then brought into position in front of the car and the pressure test system is attached to the pressure test hose. Once attached, the inspector presses a button and the pressure test ------- proceeds automatically. When the test is completed, a light automatically illuminates to notify the inspector. Once the pressure test is completed, the hood is lowered (but not closed) and the vehicle is moved to the second position in the lane where the IM240 and the evaporative purge tests are performed. With one inspector driving the vehicle and another inspector assisting, the vehicle is positioned on the dynamometer. While still running, the vehicle is prepped by one or two inspectors for the IM240 test: • chock blocks are placed in front of the non-drive wheel tires • for some front wheel drive cars, straps are attached to the front of the vehicle • the cooling fan is positioned in front of the vehicle • the purge meter (mounted on the cooling fan) is attached to the purge test hoses • the sample funnel is attached to the tail pipe, and • the test control console is hooked onto the steering wheel. Once prepared, the inspector sits in the driver's seat and runs the IM240 and the evaporative system purge test. Maine employs fast-pass software so the test may last as little as 30 seconds or as long as 240 seconds. Once the IM240 is complete, the vehicle is deprepped and moved to the final position at which point the motorist resumes control of the vehicle, is given the test report and a brochure, and is advised of the test results. At this point, the motorist may be referred to the customer service office for more assistance. Over the course of the three day visit the audit team observed this testing process at all four of the test stations visited. In most respects, inspectors followed the test procedures established by EPA and the State. There were several relatively minor deviations observed during the audit, that are unlikely to have any major impacts but do need to be addressed: • In some stations, vehicle hoods were not left fully open during the IM240. Opening the hood during the transient test is important to simulate normal air flow cooling during the test. The effect on emissions of failing to open the hood is likely negligible, however, it is important to prevent vehicles from overheating. • At one station, the sampling system was routinely removed from the tailpipe prior to the end of the test due to the mistaken belief that the final deceleration did not contribute to the test results. Pulling the sampling system off early is unlikely to result in false failures, however, there are some types of emission related malfunctions that might not be detected. • At one station, the pressure and purge tests were skipped if the inspector observed that the vent line or purge lines were "brittle." As far as we could tell, these vehicles were not failed and required to get new hoses. EPA believes it is best to test these vehicles, however, if hoses are too brittle for the vehicle to be tested then it should fail. • Vehicles were not always restarted right after the pressure test. EPA guidance requires a minimum of 30 seconds of engine operation after restart prior to the IM240. Minimizing the length of time the engine is shut off is important to Page 3 August 19, 1994 Maine Audit Report ------- keep the vehicle in a fully warmed up condition. This will become more important when the weather gets colder. The audit team also noted that the State's instructions to the contractor do not seem to include a comprehensive visual inspection of the evaporative canister. This check is part of the credit for the pressure test. A visual check of the evaporative canister needs to be performed and must include an assessment of the canister and the hose connections. Obviously tampered or damaged canisters or obviously missing or misrouted hoses must result in test failure. There are several other recommendations that the State should consider in terms of how the test is run and which tests are employed: • Suspend evaporative pressure or purge tests for problem vehicles such as Ford Broncos. EPA has approved alternative pressure and purge tests that should alleviate many of the problems that Maine has encountered. These tests will not be ready for use for approximately 6 months. • Use fast-pass only when there are vehicles waiting in line to be tested. The fast-pass results only predict the final test outcome; they do not provide an accurate, absolute measure of the vehicle's condition. When time allows, completing the full test will provide the motorist and the State with more accurate information about the vehicle's emission rates. • Check tire pressure when there are no lines. While the audit team did not notice any vehicles with low tire pressure being tested, we also did not observe any tire inflation occurring, which is required when low tire pressure is observed. It is often difficult to visually detect low tire pressure. Inflating tires to recommended tire sidewall pressure when there is no time pressure in the lane will yield important benefits: fuel economy will improve, emission rates may be reduced, and a more accurate test result will be insured. The audit team did not note any other problems with the performance of the test but some potential problems are difficult to spot during overt observations. Inspectors should be reminded to: • Verify vehicle operating temperature by checking the temperature gauge • Check drive wheel tires for inflation and safety conditions • Turn off all accessories during the IM240 Physical Structure and Equipment Maine's contractor, Systems Control, Inc. has done an outstanding job in designing and deploying the IM240 testing system. The equipment is well designed in most respects and it appears to meet the specifications established in EPA guidance. The pressure test arm is very convenient to use - it swings into position and back out of the way with ease. The monitors and controls for the emission test are intuitive and practical. The equipment provides excellent feedback to the driver on the degree to which the trace is being followed. A possible problem observed in one case is that the software did not seem to identify the fact that the emission sample was lost after the sample funnel fell out of the tailpipe. An inspector noticed the problem and alerted the driver and the test was aborted. In another case, the sample funnel kept collapsing when the inspector tried to attach it to the tailpipe. The technical staff indicated that they were aware of the problem and plan to increase the Page 4 August 19, 1994 Maine Audit Report ------- rigidity of the funnel. The purge equipment is mounted directly on the cooling fan, which is manually placed in front of the vehicle very easily and easily retracted after the test. The vehicle restraint system - which is also not automatic - is more cumbersome to deploy and, in two cases, inspectors did not remove chock blocks or straps prior to moving the vehicle to the third position. (The vehicles were not apparently damaged in these cases.) The layout of the lane is designed to minimize noise levels - blowers are placed near the ceiling in insulated boxes. Enclosed booths are provided along with fenced areas for observing tests to insure the safety and comfort of motorists. In general, the physical structure and the equipment meets EPA's expectations of how an effective system is designed and deployed. Public Interface The audit team observed the testing process from beginning to end in as many cases as possible. By necessity, inspectors must interact with motorists in order to conduct the test process. From the initial greeting to the final results, the audit team observed that inspectors acted courteously and helpfully. Even in cases where a customer was upset about failing the test or some other problem, the inspectors maintained their composure and performed their roles in a responsible manner. EPA does recommend one procedural change that is used in other centralized I7M programs. • Inspectors should provide motorists with only a very brief verbal report on the results of the test and then refer motorists that have additional questions or would like assistance to the customer service office. This will help increase throughput and insure that motorists are given consistent, accurate and comprehensive information. The audit team noticed that testing was frequently delayed because one member of the three-person team was occupied answering a motorist's questions in the third position. This does not seem to be an efficient use of the inspector's time. The training required to insure optimum responses to the wide variety of questions or potentially difficult situations is probably too much to ask of this type of position. In keeping with this, a system needs to be devised (e.g., a buzzer or light) such that when motorists do enter the customer service office, a customer service agent can respond promptly if they are not already present at the desk. The other major interface with the public is the test report. This is one area where major changes need to be implemented right away. The audit team found the report to be confusing and the information in conflict with the intent of EPA guidance. It should be noted that EPA's guidance was not absolutely clear in this respect. EPA plans to make changes to clarify its intent. The following changes should be made to the test report: • The composite emission standard should always be the only standard printed on the test report for each pollutant, regardless of the decision process used to determine pass/fail status. While the logic being used for determining the overall test result is correct, the report is confusing. Two standards are being printed now - the composite standard or the fast pass standard, and the phase 2 standard. This is very confusing - especially when a vehicle fails only the composite or only the phase 2 standard. When a vehicle fails only one, then the overall result is a pass. The fact that the printout says "Fail" for one of them implies a failure for that pollutant overall. • Current practice is to print the fast-pass standard instead of the composite standard if the vehicle is fast-passed. This is even more confusing since vehicles of the same class appear to be subject to different standards. The Page 5 August 19, 1994 Maine Audit Report ------- potential for this type of confusion was highlighted in a recent newspaper report in which it was inferred that one vehicle was tested at different standards at different stations. The fast pass standards should be viewed as subsets of the composite score since they are derived from the composite standard. As above, only the composite standard should be printed on the test report. Two emission scores are also being reported for hydrocarbons and carbon monoxide, and the score reported is a function of the decision process used to determine pass/fail status. This is also confusing to motorists. The emission scores reported should be as follows: • If the vehicle passes the composite standard, report only the composite emission score. • If the vehicle fails the composite standard, report only the composite emission score. • If the vehicle fails the composite standard but passes the phase 2 standard, report only the phase 2 results. • If the vehicle fast passes, report the gram per mile emission rate for the test. Another essential element of successful public interface is providing motorists that fail the test with information on what to do. The State and the contractor have produced an excellent brochure providing motorists with general information on how to respond to a test failure. At this time, however, that brochure, the test report, and verbal information is all that is provided to motorists. EPA's I/M rule requires that motorists also be provided with a list of repair facilities. This list must include all facilities that have performed repairs on one or more vehicles that failed the I/M test. The list may be segmented in various ways, including groupings of certified facilities and non-certified facilities. Naturally, at the start of the program, the list will be in a rapidly evolving state as more and more repair facilities become involved in the program. In addition to merely listing the facilities, EPA rules also require that the list include information on the success of stations that repair vehicles. Again, that information will take some time to accumulate and EPA does not expect such information to be reported in the first few months of a program. • Give every failing motorist a list of repair facilities that are certified or have conducted repairs on vehicles that failed a test. Conversations with the Maine Department of Environment's I/M staff indicate that such a list is now available and will be distributed. EPA's I/M rule also requires that repair technicians be provided with software generated diagnostic information when vehicles fail the test. The minimum that EPA intended was that motorists be given second-by-second emission results if they failed the test. This information could be passed on to repair technicians to use in the diagnostic process. The State is planning to provide this information to repair technicians via an electronic interface service. The printed information should also be provided so that the information is readily available to all technicians. • Print out second-by-second trace information and give it to failing motorists. Effective maintenance is the key to a successful I/M program. The State has taken commendable steps to establish training and certification programs for repair technicians. The Maine Central Technical College provides testing and training courses that relate Page 6 August 19, 1994 Maine Audit Report ------- directly to the inspection program. The State and the contractor have also produced a series of newsletters that have been sent to repair facilities throughout the seven county region. Nevertheless, the audit team got the impression that some in the repair industry did not have a full understanding of the nature of the I/M tests being performed, general information on why the were necessary and what they were intended to find, information on how to interpret the results and other program specific information. • EPA recommends that the State institute a series of brief repair technician training sessions to provide information about the program and how it works (i.e., not training on repairing vehicles, per se; Maine is already providing such a program). One of EPA's grantees has developed a training course that serves this purpose. Failure Rates The audit team reviewed data on the pass and fail rates for the first month of the program. The overall failure rate is 21%. The by-model-year failure rates track very closely with the failure rates predicted by EPA based on the standards being used. The audit team also looked at retest failure rates. Among vehicles that got repaired and returned to the test station for a retest, about 62% passed the retest. This rate is essentially the same as that experienced in other (basic) test-only I/M programs. This indicates that, in the majority of cases, repair technicians are able to repair the vehicle on the first try. EPA was concerned that in an area that has never had an I/M program and one that starts with enhanced I/M, unusual problems could be experienced with repair effectiveness. This does not seem to be the case; the repair community seems to be as capable as other I/M areas in fixing failed vehicles. Summary and Conclusions The Maine Car Test program is technically well designed and implemented. We conclude that Maine has successfully implemented the IM240, purge and pressure tests in mass production in a networked system. This system has been developed and tested by EPA over the last five years. Given that this is the first full-scale, enhanced IM240 program, the few technical problems noted in this audit are an indication of the outstanding job done by the Maine Department of Environment and its contractor, Systems Control. This is not to say, however, that there are no problems that need to be addressed. The technical problems, however, are minor. The major area that needs to be addressed at this time is public interface, especially revising the test report and other information provided to motorists that fail the test. Minor refinements are needed in the testing process, and procedures for suspending evaporative system tests when pattern failure problems are encountered are needed Additional informational training for the repair industry would be useful in helping insure efficient and effective communications and repairs, although the data on retest pass rates to date do not indicate a major problem with repair effectiveness. Page 7 August 19, 1994 Maine Audit Report ------- |