EPA420-P-99-022
May 1999
DRAFT
Evaluation of Two Prototype
Purge Flow Test Instruments
Martin Reineman
Regional and State Programs Division
Office of Mobile Sources
U.S. Environmental Protection Agency
NOTICE
This technical report does not necessarily represent final EPA decisions or positions.
It is intended to present technical analysis of issues using data which are currently available.
The purpose in the release of such reports is to facilitate the exchange of
technical information and to inform the public of technical developments which
may form the basis for a final EPA decision, position, or regulatory action.
-------
DRAFT
Abstract:
This report presents the results of an evaluation of two prototype purge test devices conducted
at Automotive Testing Laboratories, Inc. in January 1999. The evaluation was performed
under contract for EPA. One prototype device was submitted by Sensors Inc., and a second
device by Leo Breton and Dennis Johnson of EPA's Office of Mobile Sources Engine Programs
and Compliance Division. Each device was evaluated on approximately 90 test vehicles
recruited from the Mesa, Arizona emission inspection station. Each method was compared to a
reference method as a check on the prototype method's accuracy. Due principally to problems
with intrusiveness (the need for slave purge lines or the need to create a small hole in the purge
line by inserting a needle), lack of timeliness of the methods, and high false failure rates, EPA
determined neither method merits further development effort.
Copies of Appendices 2-5 are available only in hard copy form, by request.
Comments on this report may be submitted electronically to Martin Reineman at:
Reineman.Martin@epa.gov or by Fax at: 734-214-4052, or by mail at: U.S. EPA, attn Martin
Reineman, 2000 Traverwood Dr., Ann Arbor, Ml 48105.
Comments will be accepted for 30 days following posting on the EPA web site. Comments will
be considered and may be reflected in changes to the final report.
-------
DRAFT
Background:
EPA requirements for "high enhanced I/M testing" include conducting pressure and purge tests
for identifying vehicles with excess evaporative emissions. Since the start of IM240 exhaust
and evaporative emission testing, conducting the purge test has been problematic for several
reasons. The original problems included the time constraints to run the EPA flow meter method
in high volume I/M test lanes, and the intrusive nature of the purge test which sometimes
resulted in breaking evaporative emission components while installing the purge flow meter
described in EPA IM240 and Evap Technical Guidance. These problems have been shown to
be related to the design specific nature of location and design of vehicles' evaporative emission
control systems, and the fact that they were never designed to be easily compatible with an
IM240test.
The purpose of the I/M purge test is not to provide an accurate quantitative mass measurement
of total evaporative emission hydrocarbon, but rather provide an accurate qualitative indication
of whether a vehicle has a malfunctioning purge system. Although EPA was able to develop a
qualitative purge test using a flowmeter installed between the canister and the engine air intake,
this method did not translate well from the laboratory environment with its lack of time
constraints on vehicle throughput and higher skilled technicians, to the time constrained and
less experienced lane technicians in actual I/M test lanes.
EPA has previously evaluated alternative purge test equipment and procedures such as the
helium tracer method, and discussed alternative concepts with auto manufacturers and
research and testing organizations.
This evaluation compared the ability of two I/M prototype instruments to correctly determine
whether the purge systems on 1981-1995 model year vehicles are functional. The instruments
were provided by Sensors Inc. and by Leo Breton and Dennis Johnson of EPA's Engine
Programs and Compliance Division.
Objective:
The purpose of this evaluation was to examine the prototypes and determine if it is desirable to
commit further development effort in pursuit of an I/M purge test for pre-OBD II vehicles. This
report presents the results of the evaluation of the two prototypes at Automotive Testing
Laboratories (ATL) in January, 1999.
An acceptable purge test requires that the instrument and test procedures meet design specific
challenges which are both vehicle and I/M test specific.
These challenges include the following vehicle specific issues:
1) The method must be applicable to the variety of evaporative emission designs used
in light duty vehicles and trucks from the 1981-1995 model years. Purge flow is
controlled by solenoids and ECMs, thermal switches, engine vacuum, and sometimes
vehicle speed sensors. Purge flow may be constant when some of the criteria above
are satisfied, or modulated, cycling between zero and maximum flow when commanded
-------
DRAFT
to do so. Flow rates vary greatly, typically between 10-200 cc/sec of air and
hydrocarbon mixture.
2) The canister purge lines vary greatly in their accessibility and length of line. In
approximately 20% of the in-use fleet, the canister is not visible from the engine
compartment, is sometimes sealed in body components, or is visible only from
underneath the vehicle.
3) The purge lines vary from less than a 1/4 inch outside diameter (OD) to about 1/4 inch
OD, and vary in material composition from hard plastic and age/temperature hardened
polymer lines to soft lines. Age hardened lines are often fragile and will break when
they are removed.
4) The lines are frequently located close to hot engine surfaces. Electrical noise and
engine vibration are usually present in the engine compartment.
Purge test criteria specific to I/M purge testing include the following test specific issues:
1) The qualitative identification of purge system failures should be accurate, with a goal
of no more than a 5% false fail rate, and a 10% false pass rate. These 5 and 10% rates
are not based on established EPA guidelines, but rather are thought to provide a
reasonable engineering objective that would be acceptable to I/M stakeholders, and
provided a common design objective for the suppliers of the two prototype purge
devices.
2) For each model year, the method should be able to be run on at least 80% of the
1981-1995 model year vehicles. Again, the 80% criteria is based on EPA judgement as
to what might be acceptable to I/M stakeholders. 1996 and later vehicles will be able to
use OBD II scan tools to identify purge system failures.
3) Preferably, the instrument should be able to be installed without shutting off the
engine, as turning the engine off will result in delaying the start of purge flow in some
vehicles for 2-4 minutes.
4) It is desirable if the purge flow measurement is conducted simultaneously with
conducting the I/M exhaust test as this will not add to the total test time. If the method is
run simultaneously with the I/M exhaust test it must not influence the emission results. If
the method must be run as a stand-alone test, the test length should be no longer than
60 seconds.
5) The method must not be intrusive. Removing or cutting purge lines and installing a
flow meter is an intrusive method. The method should not leave evidence that a purge
test has been run.
6) The instrument should be operational under ambient conditions varying from 0-40 °C
and engine compartment temperatures of about 10-70 °C.
7) The instrument must be capable of extremely rugged use in a production
-------
DRAFT
environment by I/M lane inspectors.
8) The instrument must include a means of verifying its functionality on a daily or
weekly basis.
This set of criteria, excepting items 6-8, served as the basis for deciding if the purge test
prototypes warranted further development effort.
Instrument Descriptions
The description of the operating techniques behind the two prototypes is limited for two
reasons. 1) The design concepts for the two instruments are proprietary, and patents are being
explored for parts of one or both designs. 2) The designs evaluated were prototypes, and
therefore some of the limitations and problem areas observed in this test program could be
resolved by redesign and building second generation prototypes.
EPA - The EPA computer controlled prototype is based on a strain gage (load cell) design
where purge flow is sensed by comparing the purge line diameter in a no-flow state and in then
in a condition when purge flow is present. The EPA method uses two pieces of hardware
placed over the purge line to make its assessment, a pneumatically actuated clamping device,
which alternately produces a flow and no-flow condition, and the sensing element which is a
strain gage load cell which measures the difference in purge line wall diameter between a flow
and no-flow condition. The no-flow condition is achieved by momentarily clamping the purge
line using a pneumatically actuated piston and then returning the purge line to its undamped
state by the natural flexure of the purge line. When the line has returned to its original
configuration, the load cell again measures the wall diameter, and compares the analog
voltages measured by the load cell.
This measurement method is a qualitative technique, and depends on the ability to sense purge
flow a selected number of times (a minimum of eight in this evaluation) during a maximum test
period of four minutes. The EPA prototype is computer controlled and calibrated, and shows
the purge or no-purge condition by displaying the number of times purge flow is sensed. Eight
determinations of purge flow in a period of four minutes or less produce a "pass" status on the
computer monitor.
A more complete description of the EPA method is presented in Appendix 1. Appendix 2 is a
figure provided by ATL and shows the positioning of the load cell sensor and the clamping
device used for the 85 vehicles examined with the EPA tester.
Sensors - The Sensors prototype is based on an application of flow measurement used in
medical science. The basic principle is flow measurement using an approximate 0.040 in.
diameter needle probe and an application of hot wire anemometry to measure flow rate. A
computer initiates the test and displays instantaneous and cumulative purge flow over time.
When the cumulative flow equals one liter the computer displays a "pass" condition. If less than
one liter of flow is measured by the end of the four minute driving cycle, a fail determination is
be made.
-------
DRAFT
The method requires care to position the needle in the approximate center of the purge line
cross sectional area, which for this program, was done manually by the test technician.
Sensors has provided EPA with sketches of a "packaging concept" which would always ensure
the needle was properly located in the flow stream.
At the time of this evaluation, the only documentation available was a brief operating procedure
supplied by Sensors and engineering concept drawings of a proposed centering and packing
design. The operating procedure is included as Appendix 3.
Test Fleet
100 1981-1995 vehicles were recruited from the Mesa, AZ I/M emission test station during the
period January 7-28, 1999. The 100 vehicle sample included a variety of model years and an
approximate 50% split of light duty vehicles and light duty trucks. No attempt was made to
obtain a sample representative of the AZ in-use fleet because the program design is simply an
evaluation of the prototype devices and does not attempt to make inferences about the in-use
test fleet. A $25 cash incentive was paid to obtain each test vehicle for a period of about 30
minutes.
Test Protocol
All tests were conducted between January 7 and January 28 in the inspection bay of the ATL
Mesa, AZ emission test facility. In its most simplistic design, the protocol called for installing
each prototype in series with a reference method and observing the response. This was done
both for vehicles in an as-received condition, and after an induced failure of the purge system.
The test protocol consisted of the following steps:
1) Record vehicle and purge system descriptive data from each vehicle.
2) Connect a 0-10 SCFH roto-meter (with a shutoff valve to induce a "failed" mode)
in line with the canister purge system. Install a slave purge line if necessary.
3) Alternate the installation of either prototype method.
4) Exercise the vehicle at combinations of steady state operation and/or rapid accelerator
variation to induce and verify purge flow using the roto-meter. Take the appropriate time to
ensure that vehicles with operating purge systems satisfy any engine temperature, drive time
criteria, throttle actuation requirements, and, in some cases, wheel speed operation by placing
the vehicle on a hoist in the inspection bay and placing the transmission in drive.
5) Compare the roto-meter and prototype device qualitative determinations of purge flow.
6) Turn the shut-off valve on the roto-meter to induce a "purge failure."
7) Repeat steps 4 and 5 with the first prototype instrument.
-------
DRAFT
8) Install the second prototype and repeat steps 4 and 5.
9) Turn the shut-off valve on the roto-meter to induce a "purge failure."
10) Repeat steps 4 and 5 with the second prototype.
Years of ATL testing experience has established that observation of flow with a roto-meter will,
with near certainty, produce the same qualitative observation of purge flow as if an in-line flow
meter was used to verify the measurement of at least 1.0 liter of purge flow over the IM240
driving cycle. Success with either prototype device was defined as qualitative agreement with
the roto-meter method, i.e. can they tell when the purge system has some flow, or none.
Neither prototype was designed to produce accurate quantitative correlation with a known
reference method such as a totalizing flow meter.
If the prototype device could not sense flow when flow was observed with the roto-meter, the
prototype was not tested in the induced failure mode. The rationale for this was that the
prototype would again have indicated no-flow, but only because it was not performing correctly
for that particular vehicle.
Test time with each method was recorded for vehicles 33-100. Average test times for the
prototypes are calculated for the as-received tests, and do not include time to locate the purge
line or exercise the vehicle to obtain a purge condition as proven by the roto-meter. For the
induced failure mode, the test was run for four minutes, thus simulating what might be done in
an IM240 test if the purge device did not show a fast pass during the IM240 exhaust test.
Results
The evaluation program did not attempt to get a true blind test evaluation, and therefore care
was exercised in analyzing the data to attempt to simulate what would have been observed if
only the first test result with one of the prototypes was used when more than one attempt was
made to ascertain the purge system status. As this test program was part development testing
and part evaluation testing of the devices, there were instances where multiple attempts were
run with a prototype to attempt to match the purge condition shown by the roto-meter.
12 vehicles had inoperative purge systems in their as-received condition. 7 of the 12 vehicles
with true failed purge systems could be identified visually by ATL's experienced technicians. 5
of the 7 vehicles visually identified by ATL's technicians were independently identified as visual
failures by AZ I/M lane test technicians when the vehicles were returned to the I/M lane to
receive their required state inspection test.
47 of 100 vehicles had plastic lines, or polymer lines which had become hard due to aging.
The evaluation results for average test time, testability, and accuracy are summarized in Table
1. Average test time is simply the arithmetic average of time with each prototype on an as-
received test. Testability statistics show the number of instances a slave hose was required to
test either prototype. Accuracy statistics describe the number of times each prototype agreed
with the determinations of the roto-meter, the reference method.
-------
DRAFT
More detailed results of the as-received and induced failure tests are summarized in Appendix
4. Individual data sheets for each of the 100 vehicles which were analyzed to produce Table 1
and Table1-A of the Appendix, are presented in Appendix 5, and are available upon request.
EPA Test Results
Timeliness - Total time was recorded for installing, entering data, initializing, testing, and
removing the instrument. Average test time for 46 passing vehicles was about 2 minutes, 24
seconds.
Testability - 67 of 97 vehicles tested needed to add a slave line to install the two components of
the EPA test device. The slave line was 3/8 inch OD fuel line hose. In its current design, the
purge line length required for installation of the EPA device was approximately seven inches of
accessible, flexible purge line. The actual lengths of the slave hose used in this study were
typically much longer than seven inches to allow more convenient underhood access.
As-Received Tests - 88 vehicles had purge flow as observed using the roto-meter, and 12 of
the 100 vehicle sample had failed purge systems. 3 of those 12 vehicles had broken
components and could not be tested with any method, including the roto-meter.
The EPA method correctly identified 60 passing vehicles, with 43 of the passing vehicles
requiring slave hoses; 25 were false failures (29%) and 14 of the false failures were obtained
using a slave hose; 3 vehicles could not be tested due to problems with the instrument's usage
on a particular vehicle. These 3 vehicles are not included in the 25 false failures.
Of the 12 failures on the as-received tests, 9 could be tested with the roto-meter and EPA
methods, and all 9 were correctly identified using the EPA method.
Induced Failure Tests - The EPA method was tested on 67 vehicles with induced failures, 63 of
these correctly showed the no-flow condition, 4 were false passes (6%).
Overall Observations - The following areas of concern were noted based on the results above,
and observations during testing conducted on January 6-8, daily phone conversations with ATL
staff during the conduct of the evaluation, and in follow-up conversations with ATL during the
week of February 1.
1) The operating principle of the method was demonstrated.
2) A high false fail rate (29%) was observed.
3) False pass (6%) was not a concern.
4) At over 2 minutes per test, not including the time to locate the purge line, and not
including the vehicle operation time to induce purge, the prototype is not timely.
5) To avoid the use of a slave hose the ATL technicians recommended the EPA
method be no larger than the size of an amperage probe (the combined size of both
-------
DRAFT
components including separation of the clamping and sensing components), which is
similar in size to the clamp on a battery jumper cable, and is typically about 2 by 4
inches by 3/4 of an inch.
6) The method did not work on plastic or age/temperature hardened lines without a
slave hose.
7) Part of the setup time comes from the need to manually adjust a threshold voltage
for the sensing device due to differences in purge line diameter and hardness.
Automating this operation seems necessary to reduce the total test time.
8) The method is sensitive to the orientation of the position of the clamping device and
the load cell sensor, both the distance between them (getting the sensor too close to the
clamp sometimes produced a false reading) and whether the clamp or sensor should be
closest to the source of purge vacuum. For several tests, the orientation of the clamp
and sensor needed to be reversed to obtain purge flow.
9) 4 vehicles produced notable changes in engine speed due to clamping and
unclamping action, which changes engine airflow and therefore engine speed.
Changes in engine speed are qualitatively linked to exhaust emission effects and would
not be acceptable during exhaust measurement.
10) 2 vehicles produced false results due to movement of the sensor caused by engine
vibration related to misfire, or other vehicle specific reasons.
11) Purge lines less than or equal to 1/4 inch OD, found on Toyota Corollas for
example, are too small to test in the device's current configuration.
12) The EPA method appears to lack sensitivity at the lower purge flow rates, such as
less than 3-5 SCFH.
13) The durability of a strain gage method for high volume production testing was not
evaluated in this program.
Sensors Test Results
The evaluation of the Sensors device was also not a blind test because the status of the purge
system was known by the presence of the roto-meter. As with the evaluation of the EPA
device, sometimes more than one attempt was made to obtain a valid reading, whether the true
result was either a flow or no-flow condition.
The Sensors device produced an indication of "pass" when the equivalent of 1.0 liter or greater
of air and hydrocarbon mixture was measured on or before 240 seconds of engine operation.
The observations summarized in Table 1 are based on the presence or absence of the "pass"
indication on the computer monitor which displays cumulative volume and instantaneous flow
rate. The presence of voltage spikes during as-received tests can result in a false or premature
pass condition, and during the induced failure mode, a false pass determination.
-------
DRAFT
Timeliness - Using the same criteria for defining test time that was applied to the EPA method,
the average test time for installing, entering data, initializing the test, obtaining the test result,
and removing the device was about 2 minutes and 44 seconds.
Testability - 8 of 97 (8%) vehicles tested needed a slave line in which to insert the 0.040 in.
diameter probe. Although not documented why a slave line was used for the Sensors method,
the reasons may include: access to the purge line was still a problem on some vehicles, and
some of the lines were plastic or age hardened, and therefore a soft slave line was necessary.
All 8 vehicles which used a slave line to test the Sensors device also needed the slave line for
corresponding tests with the EPA device.
As-received Tests - The Sensors method correctly identified 68 vehicles with passing purge
systems; 13 were false failures (16%); 7 vehicles could not be tested with the method due to
instrument problems on specific vehicles. Consistent with the treatment of the EPA data, these
7 tests were not included in the 13 false failures.
9 true failures were evaluated with the Sensors method on as-received tests. One vehicle was
incorrectly identified as a passing vehicle, and 8 were correctly identified.
Table 1-A of the Appendix indicates when a voltage spike was observed on the plot of
instantaneous flow rate versus time. These results are coded with the letter V. In theory, the
presence of one or more voltage spikes could produce a false "pass" reading. An examination
of the Sensor's results shows that the presence of voltage spikes during as-received tests did
not change the instrument's ability to correctly identify passing vehicles.
Induced Failure Tests - The Sensors method was tested on 71 vehicles, 7 vehicles showed a
false pass (10%). The presence of voltage spikes on induced failure tests had a notable
impact. Had voltage spikes not been present on these tests, only 3 vehicles would have falsely
passed (4%).
Overall Observations -
1) The operating principle of the method was demonstrated.
2) The voltage spike problem resulted in false determinations of "pass" on the induced
failure tests.
3) A high false fail rate (16%) was observed.
4) The method is not timely in its current state.
5) A slave hose is generally not needed to test with the method.
6) The prototype did not use a centering method to assure the probe is located in the
center of the flow stream. Development of such a technique is required to make the
method timely and accurate.
-------
DRAFT
7) No attempt was made to determine the adequacy of the silicone post treatment
applied to a hole left by removing the probe after the test. Inserting a small probe such
as the 0.040 in. diameter needle in a purge line does not meet the definition of a non-
intrusive test. (Subsequent to
8) Needle durability was a problem with this prototype as at least 4 probes were
required to complete the 100 vehicle test program.
Conclusions
EPA Test Device -
The following problem areas were most notable with the EPA prototype.
1) The prototype required a slave line to make a determination of purge flow due to
plastic and age hardened lines on the vehicles' purge systems.
2) In its current design, the prototype is far too large to be used in the restricted
configurations of real world vehicles' purge systems.
3) The prototype method is not timely.
4) False failure rates were high.
Items 1 and 2 above are severe limitations of the EPA prototype and it is unlikely that a second
generation prototype would eliminate these problems. Without solving these problems, and
proving success with a final prototype design in a high volume IM240 pilot study, I/M
stakeholders are unlikely to be interested in the current EPA prototype.
Sensors Test Device -
The following problem areas were most notable with the Sensors prototype.
1) The intrusive nature of the method, and its unknown effects on purge system
integrity and its possible effects when accidently applied to electrical lines and fuel lines,
are significant concerns in a real world, high volume test environment, such as IM240.
2) An automated centering method is necessary for locating the probe.
3) The prototype method is not timely.
4) Probe durability is a problem with the prototype.
Item 1 is a significant problem that cannot be addressed by designing a second generation of
prototype - inserting a probe into a purge line is fundamental to the method. Subsequent to the
January evaluation, Sensors has met with EPA and stated that preliminary testing suggested
that inserting and removing a 0.040 in. diameter probe in a sample of new and used purge lines
-------
DRAFT
did not show a leak was created with their test method. Even if that is correct, or a post
treatment of the purge line were successfully demonstrated, the perception of the intrusive
nature of this test, and its safety concerns in a high volume IM240 lane application would likely
prevent I/M stakeholders from being interested in this concept.
Other -
7 of 12 true failures of the purge system were identified visually by the experienced ATL
technicians, and 5 of the 7 visual failures were found independently during the AZ I/M
underhood inspection in the test lanes.
Recommendations
EPA Prototype -
Due principally to testability problems (the need to miniaturize the concept and concerns about
hard purge lines which are common in older vehicles) and effects on engine speed during the
exhaust test, it is not recommended that further effort be placed on additional development of
this prototype.
Sensors Prototype -
Due to the intrusive nature of this flow measurement concept, it is not recommended that
further effort be placed on additional development of this prototype.
-------
DRAFT
Appendix 1
Results of Prototype Purge Flow Instrument Evaluation
Test results with the two prototypes are compared against readings obtained with the roto-
meter. For all methods, a "1" indicates purge flow was measured and a "2" indicates no purge
flow was measured. These values appear in the last six columns of Table 1. Two other
numeric values appear in the last six columns of Table 1, a "3", or a "4". 3 was recorded where
a test could not be run due to problems with a prototype that prevented it from being used for a
particular vehicle. A 4 appears when no result was obtained due to a tampered or missing
purge component on the vehicle. A 4 was also used to show a no-test result when a prototype
device could not measure purge flow when it was detected with the roto-meter, and it was
decided not to run a test with an induced failure because the prototype would likely have
produced a correct failure reading, but only because the prototype was not operating correctly.
The letters "V", "E", "S", and "SG" are used to provide further detail on the test results. The
letter V is unique to the Sensors prototype, and shows where a voltage spike was observed on
the plot of instantaneous flow versus time on the computer screen. The letter E appears on test
results for vehicles 25 and 37. Engine vibration during a test appeared to have falsely
influenced the purge status. The letter S was used to show which vehicles needed to have a
slave purge line installed in order to test with either the EPA or the Sensors prototypes. SG
appears on EPA results from vehicles 18, 23, 26, and 47 when the clamping and unclamping
action of the piston assembly caused a change in engine RPM at a steady state condition..
-------
DRAFT
Appendix 2
-------
DRAFT
Appendix 3
-------
DRAFT
Appendix 4
-------
DRAFT
Appendix 5
------- |