EPA-AA-TSS-I/M-87-3
EPA AUDITS OF STATE AND LOCAL
INSPECTION/MAINTENANCE PROGRAMS:
FEDERAL FISCAL YEAR 1986
By:
John M. Cabaniss, Jr.
Jeffrey A. Houk
This paper was presented at the Air Pollution Control
Association Specialty Meeting in Windsor, Ontario, on
March 24, 1987.
Technical Support Staff
Emission Control Technology Division
Office of Mobile Sources
Office of Air and Radiation
U. S. Environmental Protection Agency
-------
Abstract
EPA AUDITS OF STATE AND LOCAL INSPECTION/MAINTENANCE PROGRAMS:
FEDERAL FISCAL YEAR 1986
J.M. Cabaniss, J.A. Houk
U.S. Environmental Protection Agency, Ann Arbor, MI 48105
This paper reviews the results of EPA's I/M program audits
during Federal fiscal year 1986 (FY86). EPA performed eleven
initial program audits and eight follow-up audits during FY86.
The paper highlights design elements that have proven successful
in these programs, and discusses the applicability of those design
elements to other I/M programs. The paper also summarizes the
progress that has been made in resolving operating problems
identified in FY86 and earlier years.
-2-
-------
EPA AUDITS OF STATE AND LOCAL
INSPECTION/MAINTENANCE PROGRAMS:
FEDERAL FISCAL YEAR 1986
1.0 INTRODUCTION/BACKGROUND
As a result of the Clean Air Act Amendments of 1977, many
motor vehicle inspection and maintenance (I/M) programs have
been implemented across the country. I/M programs are
currently operating in fifty-nine urban areas in thirty-one
States and the District of Columbia (see Table 1).
Approximately fifty million vehicles undergo inspections
annually in these programs.
EPA's Office of Mobile Sources (QMS) decided early in 1984
that there was a definite need for a national I/M audit system
and national I/M audit guidelines. EPA and State and local
agencies could use such an audit program to: (1) ensure that
statutory and State Implementation Plan (SIP) requirements are
being met; (2) assist in developing an acceptable level of I/M
program quality; (3) document the achievements, shortcomings,
and needs of the various I/M programs; (4) identify programs
needing further technical support or other assistance; and (5)
identify technical issues common to all or many programs which
need further investigation.
After consultations among officials of the State and
Territorial Air Pollution Program Administrators and the
Association of Local Air Pollution Control Officers
(STAPPA/ALAPCO), EPA's Office of Air Quality Planning and
Standards (OAQPS), and EPA's Office of Mobile Sources (OMS), it
was decided that the best approach for developing the I/M audit
program would be to add an I/M element to the National Air
Audit System (NAAS) in federal FY1985. This necessitated the
development of the national I/M audit guidelines during
FY1984. In keeping with the NAAS process, a STAPPA/ALAPCO I/M
subcommittee was formed to work with OMS in developing the
national I/M audit guidelines. With the help of STAPPA/ALAPCO
and I/M officials around the country who served as reviewers,
the I/M audit guidelines were developed on schedule and appear
as Chapter 6 of the National Air Audit System Guidance Manual
for FY85 (EPA-450/2-84/008; December, 1984) and the National
Air Audit System Guidance Manual for FY86/87 (EPA-450/2-85/008;
December, 1985).
EPA started the I/M audit program in FY1984 when eight
pilot audits were conducted during the spring and summer of
1984. (See Table 2 for the FY1984-85 audit schedule.) The
eight pilot audits were used to test auditing concepts intended
for use in the formal I/M audit guidelines as well as to get a
start on the task of auditing the then twenty-five operating
I/M programs. As displayed in Table 2, the FY1984 I/M audit
schedule was designed to cover a cross-section of program types
in order to properly test the audit concepts. The majority of
-3-
-------
Table 1
Operating I/M Programs
as of March, 1987
Location
Region I
Connecticut
Massachusetts
Rhode Island
Region II
New Jersey
New York
Region III
District of
Columbia
Delaware
Maryland
Pennsylvania
Virginia
Region IV
Georgia
Kentucky
Louisville
Cincinnati
Suburbs
North Carolina
Charlotte
Raleigh
Tennessee
Memphis
Nashville
Start Date
1/83
4/83
1/79
2/74
1/82
1/83
1/83
2/84
6/84
12/81
Start Date
Region V
Illinois
Indiana
Michigan
Wisconsin
4/82
1/84
9/86
12/82
11/86
8/83
1/85
5/86
6/84
12/85
4/84
Location
Region VI
Louisiana 9/85
Oklahoma
Tulsa 1/86
Oklahoma City 1/87
Texas
Houston 7/84
Dallas/Ft. Worth 1/86
El Paso 1/86
Region VII
Missouri 1/84
Region VIII
Colorado 1/82
Utah
Salt Lake City 4/84
Davis County 4/84
Provo 7/86
Region IX
Arizona 1/77
California
South Coast 3/84
Fresno 10/84
Bakersfield 1/86
Nevada
Las Vegas 10/83
Reno 10/83
Region X
Alaska
Anchorage 7/85
Fairbanks 7/85
Idaho 8/84
Oregon
Portland 7/75
Medford 1/86
Washington
Seattle 1/82
Spokane 7/85
-4-
-------
the ten FY1985 I/M audits involved decentralized programs (six
of eight). All of the programs, except the Texas (Harris
County) program, had been in operation for at least one year at
the time of the audits.
In addition to the initial audits performed in FY1985,
follow-up I/M audits were conducted in several States which
were audited in FY1984. The follow-up audits focused on
specific aspects of the programs where problems were identified
or which could not be fully evaluated during the previous audit
visits.
The FY84-85
measures
programs.
I/M program
by which EPA would
These include:
audits
made apparent
judge the
several key
success of these
1) High vehicle owner compliance with the inspection
requirement. Any lack of participation in an I/M
program directly reduces the possible emission
reduction benefit of the program. In addition, as
many vehicle owners who avoid inspection do so
because they know their vehicles would require
repair, the program emission reduction benefit is
reduced further.
2)
3)
A failure rate which meets design projections
For
an I/M program to function as designed, vehicles
which exceed the appropriate emissions standards must
be failed and repaired properly. Allowing
high-emitting vehicles to escape repair also reduces
the emission reduction benefit of the program.
A low waiver rate. Waiver provisions exist in most
I/M program regulations to protect vehicle owners
from excessive repair costs. However, the emission
reductions of vehicles not repaired to pass program
cutpoints are lower than those for vehicles which can
be repaired to pass. In some cases, reviews of I/M
program data indicate that no reductions are achieved
on waived vehicles.
4) Adequate analyzer quality control. Inaccurate
equipment can cause vehicles to be passed or failed
improperly. Accurate analyzers are especially
important for inspection of newer vehicles, which are
subject to tighter emission standards and may be
5)
eligible for warranty coverage.
Effective
data
analysis.
Effective use of
high-quality inspection data is vital for I/M program
management. Summaries of important program
statistics (failure rate, waiver rate, etc.) are
extremely useful in identifying problem stations and
inspectors. In addition, overall summaries allow for
evaluation of the success of the program as a whole.
-5-
-------
Table 2
FY1984-85 I/M Audit Schedule
Location
Connecticut
Massachusetts
Colorado
Arizona
District of Columbia
Virginia
Memphis, TN
New Jersey
Nevada
New York
Georgia
Missouri
Delaware
North Carolina
Texas
Oregon
Wisconsin
Indiana
Dates
5/14 -
5/16 -
5/21 -
5/23 -
ia 6/04 -
6/06 -
6/27 -
7/10 -
10/15 -
12/10 -
1/22 -
3/04 -
3/07 -
3/18 -
3/26 -
4/02 -
4/15 -
7/08 -
8/06 -
5/16/84
5/18/84
5/23/84
5/25/84
6/06/84
6/08/84
6/29/84
7/13/84
10/19/84
12/14/84
1/25/85
3/07/85
3/08/85
3/22/85
3/27/85
4/04/85
4/19/85
7/10/85
8/08/85
Program
Type*
CC
DC
DM
CC
CG
DM
CG
CG/DM
DM'
DC
DM
DM
CG
DM
DT
CG
CC
CC
Follow-up audits:
District of Columbia
Texas
Colorado
5/11 - 5/12/85
8/26 - 8/30/85
9/24 - 9/25/85
CG
DT
DM
*CC
CG
DM
DC
DT
centralized, contractor
centralized, government-run
decentralized with manual analyzers
decentralized with computerized analyzers
decentralized tampering only
-6-
-------
The FY86 I/M program audits (Table 3) differed somewhat
from those in the previous two fiscal years. EPA teams
generally remained on-site longer, to visit more inspection
stations and evaluate the performance of additional auditors.
EPA personnel also spent more time during audits examining
inspection forms in stations and reviewing data with program
officials. Two additional areas of concern were noted:
1) Success of overall quality assurance efforts. Fiscal
problems in some States led to I/M program budget
cuts. This in turn led to staff cutbacks, longer
periods of time between audits, and less time (and
attention) spent in stations. Previously noted
quality assurance deficiencies continue to appear,
including lack of useful program data and a
reluctance to aggressively enforce program
regulations. Many programs continue to lack a
meaningful covert auditing program.
2) Performance of tampering inspections in decentralized
programs. Many decentralized programs include a
requirement for some type of tampering inspection,
and some rely totally on tampering inspections to
meet the minimum emission reduction requirement for
SIP approval. However, observation of tampering
inspections during EPA audits show that many are
performed in a cursory manner, and inspectors often
have difficulty performing a complete inspection or
locating specific components when requested to do so
by an auditor. In addition, roadside surveys show in
some areas that tampering rates remain high even with
an inspection program in place. Vehicles often go
uninspected during covert audits.
2.0 SUCCESSFUL DESIGN ELEMENTS OF I/M PROGRAMS
In the course of auditing 29 I/M programs during FY84-86,
EPA has identified many features of program design that have
resulted in more effective and more smoothly running programs.
The remainder of this paper gives a brief overview of some of
these design features. Examples are given where I/M programs
have reported success with these design elements, or where they
have made changes to correct operating problems.
3.0 ENFORCEMENT MECHANISMS
Registration enforcement has generally proven the most
effective means of maintaining high rates of owner compliance
with inspection requirements. In fact, several areas have
adopted registration enforcement mechanisms after experiencing
problems with sticker enforcement, including Colorado and
Georgia. Programs which used registration enforcement from the
onset of the I/M program have reported little or no decline in
registration volume due to owners seeking to avoid the
inspection requirement.
-7-
-------
Table 3
FY1986 I/M Audit Schedule
Location
Salt Lake County, UT
Davis County, UT
Nashville, TN
Maryland
California
Seattle, WA
Louisville, KY
Pennsylvania
Idaho
Fairbanks, AK
Anchorage, AK
Follow-up audits:
Georgia
Houston, TX
Salt Lake County,
Davis County, UT
New York
Memphis, TN
Massachusetts
Connecticut
UT
Dates
10/07
10/09
11/05
11/19
1/27
2/24
3/04
3/10
3/18
8/20
8/25
10/09/85
10/10/85
11/07/85
11/21/85
2/06/86
2/28/86
2/07/86
3/13/86
3/21/86
8/22/86
8/27/86
1/29 - 1/30/86
6/11 - 6/12/86
4/28 - 5/02/86
5/29/86
5/29/86
6/11/86
6/16 - 6/18/86
7/07 - 7/11/86
9/23 - 9/25/86
Program
Type
DM
DM
CC
CC
DC
CC
CC
DC
DM
DC
DC
DM
DM
DT
DM
DM
DC
CG
DC
CC
*CC = centralized, contractor-run
CG = centralized, government-run
DM = decentralized with manual analyzers
DC = decentralized with computer analyzers
DT = decentralized tampering only
-8-
-------
Sticker enforcement mechanisms can be successful,
however. For example, Texas, Louisiana, Massachusetts, and
Pennsylvania all report high compliance rates. One factor
contributing to the success of these programs is the fact that
these areas all have statewide safety inspection programs which
had been in operation for several years before emission (or
tampering) inspections began. Authorizing police agencies to
cite parked vehicles, and keeping collected fines in local
coffers, may contribute to aggressive enforcement and low
non-compliance rates in some areas.
A few areas have adopted effective "data-link" enforcement
mechanisms. Data-link enforcement systems consist of matching
lists of vehicles subject to inspection (most often
registration data) with inspection data. Penalties for
noncomplying vehicle owners which make this system effective
include: suspension of vehicle registration, court summonses,
and fines. Boise, Idaho, Louisville, Kentucky, and Maryland
have implemented effective data-link enforcement systems.
4.0 FAILURE RATES
The reported program failure rate is one of the most
important indicators of the success of program operation.
Centralized programs, and decentralized programs with
computerized analyzers, have traditionally reported failure
rates close to what would be expected for a given set of
cutpoints. This has not been the case in decentralized, manual
analyzer programs, where the manual nature of the equipment
does not allow for control over the inspection procedure (See
Table 4).
The State of Colorado, for example, has experienced
serious problems with inspector performance, leading to a very
low failure rate. An overall failure rate of 7.9% was reported
for 1986, although voluntary emissions inspections at the
Denver area Emissions Technical Centers consistently resulted
in failure rates over 30%. The Colorado Department of Health
has estimated that manual design elements, improper inspections
and waivers had led to a 30% loss of program benefit. In an
effort to eliminate improper inspection procedures, the State
adopted a requirement for computerized analyzers, to take
effect in July 1987. In response to the 1984 EPA audit, and
due to problems with aging equipment, New Jersey also switched
to a requirement for computerized analyzers in the
decentralized inspection stations. New York is currently in
the process of making this changeover.
5.0 WAIVER PROVISIONS
Waiver systems are managed with varying degrees of
success. The best systems are characterized by stringent
requirements for obtaining a waiver, and strict State
oversight. One example of a well-designed waiver system is the
system in effect in El Paso, Texas. To receive a waiver,
-Q-
-------
Table 4
REPORTED EMISSION TEST FAILURE RATES IN I/M PROGRAMS*
CENTRALIZED (CC)
Arizona 20.2
Connecticut 17.2
Delaware 13.7
Kentucky 15.7
Maryland 14.6
Memphis, TN 8.1
Nashville, TN 24.5
New Jersey 26.1
Oregon 24.0
Washington, D.C. 18.4
Washington 19.0
Wisconsin 15.3
DECENTRALIZED
Computerized Analyzers (DC)
Alaska
Anchorage 15.7 %
Fairbanks 19.4
California 27.7
Michigan 15.8
New York** 5.1
Pennsylvania 17.6
Manual Analyzers (DM)
Georgia 6.6 %
Idaho 9.8
Missouri 6.7
North Carolina 5.6
Nevada
Clark County 9 . 5
Washoe County 11.0
Utah
Davis County 8.7
Salt Lake County 10.0
Virginia 2.3
* For all model years, including light duty trucks.
** New York's analyzers are only partially computerized
-10-
-------
vehicles must go through a low-emission tune-up sequence,
receive all warranty repairs, and any necessary repairs not
covered by warranty up to a $200 cost limit. In subsequent
years, the cost limit rises to $400. Waivers are only issued
by the Department of Public Safety, which oversees the
inspection program.
There are ways of maintaining administrative control over
a waiver system without direct State issuance of waivers. One
option is to monitor waivers through data analysis. Mechanics
who issue a higher than average number of waivers can be
targeted for auditor attention and perhaps additional
training. The Salt Lake County, Utah data handling system
produces a monthly "improper waiver" report, which lists
vehicles that were issued a waiver without meeting the
appropriate requirements by station and mechanic. Another
method of limiting the number of waivers administratively is to
increase the amount of paperwork required of vehicle owners and
mechanics to better ensure adherence to waiver provisions; for
example, use of detailed application forms, and requiring
complete repair documentation. Pennsylvania reports success
with the use of such forms, and Salt Lake County, Utah has
adopted a similar system.
The effectiveness of repairs sought in the course of
obtaining a waiver may be supported by a requirement for a
minimum emission reduction. A reduction requirement of 10-20%
provides a tangible improvement in idle emissions of waived
vehicles, and ensures that vehicles in which emissions become
worse by improper repairs are not given waivers.
Repair effectiveness may also be improved by providing
emission-related training to mechanics in the program area.
This can be scheduled as part of an inspector recertification
course, or offered on a voluntary basis through a local
community college or vocational center. In some programs,
repair facilities employing mechanics who have received this
training can be advertised as "certified" repair facilities.
Alternatively, stations identified through data analysis as
reporting the lowest retest emission levels or the lowest
waiver rates can be publicized as having the best repair
records. The Louisville, Kentucky program collects data on
repairs performed, waiver rates, and repair cost, and regularly
publishes a repair facility report.
Many States have had problems with high waiver rates,
usually due to lenient requirements. High waiver rates usually
appear in areas requiring an engine parameter adjustment
procedure, or setting a low cost limit (for example, $25 for
pre-1981 vehicles and $50 for 1981 and newer vehicles) for
required repairs. Several programs have tightened their waiver
requirements recently, including Arizona, Colorado, and
Louisville, Kentucky.
-11-
-------
6.0 DATA ANALYSIS
The FY84-85 audits revealed serious data processing
shortfalls in most programs. The programs audited in FY86 were
generally found to have more successfully implemented data
analysis systems.
Data collection is easiest in centralized programs with
on-line emission analyzers. In decentralized programs,
computerized analyzers greatly simplify data handling for
inspectors and program managers by storing keyboard-entered
inspection data on cassettes for later transfer onto computer
tape. A few decentralized programs also have had some success
with inspection forms designed to be read by an optical
scanner. Data loss due to soiled or improperly filled-out
forms can be high in this type of system; however, some smaller
I/M programs return unreadable forms or forms with errors to
inspection stations for copying or correction. The Utah
County, Utah I/M program takes this approach and reports an
unusable form rate of less than 1%.
Many I/M programs have excellent capabilities for
analyzing the data they collect. Reports of failure rates by
model year are useful in determining whether the stringency of
the cutpoints is fairly consistent throughout the range of
years tested, or if the cutpoints for any given model year (or
model year group) are too tight or too lenient. In addition,
reports of failure rate by emission level (i.e., what
percentage of vehicles would fail at a given cutpoint) have
been extremely useful to EPA and State and Local agencies.
Reports of failure rate by pollutant have also proven useful to
program managers in determining whether cutpoints have been
selected properly (officials in areas experiencing CO
violations only, for example, often do not wish to fail a large
percentage of vehicles for HC alone).
Reports of waiver rates and calculation of average idle
emission reductions on waived vehicles are useful in evaluating
the effectiveness of the repair requirements for any given
model year or model year group. Analysis of waiver rate by
pollutant is helpful in determining the appropriateness of
specific repair sequences (the so-called "five-parameter
adjustments"); Colorado, among other States, has found that
engine adjustments alone are somewhat effective in reducing CO
emissions but have less impact on HC emissions. Colorado has
recently supplemented the adjustment requirements with a
minimum repair expenditure, and increased the repair
expenditure requirement for 1981 and newer vehicles as well.
Most useful to I/M program managers, especially in
decentralized programs, are reports of program statistics by
station and by inspector. Reports of failure rates by station
and inspector, in conjunction with information on the average
model year mix of vehicles inspected by each station, can be
-12-
-------
used to pinpoint inspectors who are reporting a lower than
average failure rate, and who may be performing inspections
improperly. This information is useful in both tampering and
emissions programs. Reports of waiver rate by
inspector/mechanic may be used to identify mechanics who are
waiving vehicles at a higher than average rate, indicating that
they may be waiving vehicles improperly or may need additional
repair training. Individual vehicles which have been passed or
waived improperly can be brought to the attention of the
responsible inspectors; Utah County, Utah I/M officials require
inspectors to recall these vehicles to the station and process
them properly, or face suspension. Reports of repair cost by
station can be useful in identifying stations which may be
unfairly overcharging for repairs or performing unneeded work,
as well as stations which are improperly passing vehicles
rather than performing necessary repairs. These types of
summaries of station performance can be useful to auditors
during station visits to point out possible problem areas, and
to enable station personnel to compare their performance to the
performance of the I/M program as a whole.
7.0 INSPECTION STATION AUDITING PROCEDURES
An important part of each EPA audit is an evaluation of a
program's station audit and surveillance efforts. A strong
quality assurance program is essential to ensure that the
program is operating as intended and that program objectives
are being met.
EPA audits have shown that most station auditors do their
jobs well. Most are conscientious, well-organized, and have a
good rapport with the station personnel with whom they work.
Most programs appear adequately staffed, although some States
have experienced cutbacks recently due to budget problems.
Nearly all programs have established standardized audit
procedures, and some programs periodically rotate auditors to
different sections of the I/M area. These measures help ensure
that all of the stations in the I/M program are treated
consistently.
Consistent follow-up on problems identified during
stations visits is vital. Several I/M programs have adopted
penalty schedules, standardizing the penalties for various
offenses. A quick process for closing stations or suspending
inspectors in violation of program regulations permits program
officials to retain control over station and inspector
performance.
Covert auditing is an extremely useful tool in
decentralized programs for learning how well inspection
stations are performing and for providing a basis for action
against bad operators. Although most programs do some
undercover work, few have organized programs capable of
-13-
-------
visiting each station at least once per year. The best systems
are those in which used cars are obtained periodically and sent
to stations in a maladjusted or tampered condition. Part-time
or temporary personnel often operate the vehicles, and funds
are provided to cover the cost of inspection fees. Colorado
uses a fleet of several vehicles, and performed over 1800
covert audits at 1350 stations in 1986, resulting in nearly 300
enforcement actions.
8.0 CONCLUSION
At this time, almost all operating I/M programs have been
audited by EPA. EPA is encouraged by the results of the FY86
audits and by the generally successful implementation of I/M
programs, although some areas continue to have serious
operating problems. Substantial progress has been made in some
of these areas toward correcting operating deficiencies. The
solutions to most problems are well known and have been
successfully implemented by many I/M programs, as outlined
above. EPA is confident that the quality of I/M programs
nationwide will improve as more areas identify problems and
adopt design enhancements to address them.
EPA's program of auditing I/M programs is an ongoing
process. The first cycle of audits was basically completed in
FY86, and future audits will be directed toward evaluating the
few remaining new programs and following up on States' efforts
to correct problems already identified. EPA also plans to work
with STAPPA/ALAPCO during FY87 to develop revised I/M audit
guidelines for FY88 and FY89.
I/M programs are a major part of most areas' strategies
for reaching attainment and assuring maintenance of the ambient
ozone and carbon monoxide standards. I/M will play an even
greater role as some areas look at additional strategies to
deal with intractable ozone and CO problems. The audit process
will continue to play an important role in achieving the
greatest possible environmental benefit from I/M.
-14-
------- |