GUIDANCE MANUAL
USING UNMARKED VEHICLES IN
DECENTRALIZED l/M PROGRAMS
MARCH 1987
SUBMITTED TO:
EMISSION CONTROL
TECHNOLOGY DIVISION
U.S. ENVIRONMENTAL PROTECTION AGENCY
2565 PLYMOUTH ROAD
ANN ARBOR, Ml. 48105
-------
GUIDANCE MANUAL
Using Unmarked Vehicles in
Decentralized I/M Programs
Submitted to
Emission Control Technology Division
U.S. Environmental Protection Agency
2565 Plymouth Road
Ann Arbor, MI 48105
March 1987
46924.00/51,51A
Submitted by
Engineering-Science
Two Flint Hill
10521 Rosehaven Street
Fairfax, Virginia 22030
-------
TABLE OF CONTENTS
List of Tables lii
List of Figures iv
Executive Summary v
Disclaimer vi
Acknowledgment vii
CHAPTER 1 THE ROLE OF UNMARKED VEHICLES IN DECENTRALIZED 1-1
I/M PROGRAMS
Introduction 1-1
Comprehensive QA in a Decentralized I/M Program 1-2
Covert Auditing - The Role of Unmarked Vehicles 1-4
Outline of a Covert Auditing Program 1-4
Structure of this Manual 1-7
CHAPTER 2 ENFORCEMENT OF PROGRAM REGULATIONS 2-1
Focus of Enforcement Activities 2-1
Prioritizing Stations: 2-6
An Aid towards Documentation and Deterrence
The Issue of Entrapment 2-14
Penalties, Publicity and the Durability of 2-14
Deterrence
CHAPTER 3 COLLECTION OF DATA ON PROGRAM OPERATIONS 3-1
Coordination with Enforcement-Oriented Auditing 3-1
Specific Types of Studies 3-3
CHAPTER 4 EXAMPLES OF RESULTS FROM CURRENT AUDITING PROGRAMS 4-1
Prevalence of Detrimental Practices 4-1
Deterrence Through Publicity 4-3
Development and Disposition of Cases 4-3
CHAPTER 5 PROGRAM ISSUES 5-1
Program Elements 5-1
Two Model Programs 5-7
REFERENCES R-1
APPENDIX A SHOP GUIDANCE A-1
APPENDIX B BACKGROUND MATERIAL FROM ONGOING I/M PROGRAMS B-1
ii
-------
LIST OF TABLES
1.1 Capabilities and Limitation of Formal Periodic 1-3
Field Auditing in Decentralized I/M Program
1.2 Interrelationships of Program Elements in Alterna- 1-6
tive Approaches to Covert Auditing
2.1 Suggested Checklist for Determining Overall Com- 2-2
pliance with Decentralized I/M Program Regula-
tions
2.2 Reason for Prioritizing Stations for Covert Audit 2-7
By Station Type in California (July 1984 -
June 1985)
2.3 Summary of Potential Evidence for Specific Repair 2-11
Practices
3.1 Potential Areas of Data-Gathering Investigation 3-2
For Covert Vehicles
3.2 Potential Comparisons To Be Used in Characterizing 3-4
I/M Operations
4.1 Summary of Covert Investigations in the Colorado 4-2
I/M Program (January 1 - November 22, 1985)
4.2 Administrative Summary-Covert Audits in California 4-4
5.1 Checklist of Questions for Ensuring That Cover is 5-3
Maintained During A Covert Investigation
5.2 Two Model Programs 5-8
in
-------
LIST OF FIGURES
1.4 Structure of the Manual 1-8
2.1 Development of Enforcement-Oriented Covert 2-3
Auditing
2.2 Conceptual QA Framework for Enforcement-Oriented 2-8
Covert Audits
2.3 California Station/Mechanic Summary 2-10
A.1 Sample Tools for an Audit Set-Up Shop A-3
iv
-------
EXECUTIVE SUMMARY
The problems of quality assurance in a decentralized I/M program
are well-known. As a result, every current decentralized I/M program
does some formal periodic audit of licensed stations. However, formal
auditing cannot detect certain quality assurance problems. Specifical-
ly, while formal auditing can certainly test what station personnel may
know, it does not necessarily test what is actually done during inspec-
tions and repair.
Covert auditing allows I/M program managers to determine what is
actually happening in a program. "Undercover" cars have historically
been used in safety programs to gather evidence on stations which may
have been flagrantly violating the rules of the program. To be sure,
covert auditing may serve this role for I/M as well, but it also serves
to encourage comprehensive inspections and proper repairs. In addition,
it may be used to gather data on program operations.
It is the purpose of this manual to present program approaches
and techniques for covert auditing. Issues in both enforcement-related
and data-related covert auditing are described. Results of existing
covert auditing programs are discussed. Finally, some of the practical
issues in program development are investigated, and two model programs
(an initial program and a mature program) are presented.
v
-------
DISCLAIMER
This summary report was furnished to the U.S. Environmental Protec-
tion Agency by Engineering-Science, 10521 Rosehaven Street, Fairfax, Vir-
ginia, in fulfillment of Work Assignment 24, Contract No. 68-02-3888.
The opinions, findings, and conclusions expressed are those of the authors
and not necessarily those of the U.S. Environmental Protection Agency.
Similarly, mention of company or product names should not be considered
as an endorsement either by the U.S. Environmental Protection Agency or by
Engineering-Science.
vi
-------
ACKNOWLEDGMENTS
Engineering-Science wishes to express its gratitude to Federal,
State, and local personnel who aided in the development of this manual.
In particular, Engineering-Science would like to thank Bob Grosso and Phil
Ryan of the New York Department of Motor Vehicles, John Duncan of the
Colorado Department of Revenue, and Gary Hunter of the California Bureau
of Automotive Repair for setting up and coordinating two-day program
visits in Westbury (Long Island), Denver, and Sacramento. Finally, Engi-
neering-Science would like to express its appreciation to Phil Lorang,
Chief of USEPA/ECTD's Technical Support Staff, Jane Armstrong, USEPA's
Project Officer, and John Cabaniss, USEPA's Project Coordinator, for their
assistance, guidance, and cooperation.
vii
-------
CHAPTER 1
THE ROLE OF UNMARKED VEHICLES IN DECENTRALIZED
I/M PROGRAMS
INTRODUCTION
Decentralized motor vehicle emissions inspection/maintenance I/M pro-
grams* have long been favored by many State and local governments because
of the convenience they offer to motor vehicle owners. In addition, in
places where decentralized safety programs were already in place, legisla-
tures and agencies establishing I/M programs generally implemented decen-
tralized I/M programs to avoid any inconvenience and administrative dupli-
cation which would possibly result from parallel but separate inspection
systems. Despite the obvious advantages of a decentralized system, at the
same time the quality assurance (QA) problems inherent in decentralized
programs have always posed a challenge to program managers.
QA in a decentralized program involves four distinct steps:
o specification of the data and other information
to be collected;
o design and development of a surveillance system to deter-
mine whether procedural regulations are met;
o collection, reporting, and analysis of information; and
o modifying the system and/or the program on the
basis of analysis of results.
The key to developing an effective I/M QA approach, therefore, is to
identify what to monitor and what to control. Early assessments of QA
issues in I/M tended to focus on the ability of individual mechanics to
perform proper repairs (USEPA, 1974). The need for oversight of licensed
stations became more clear as decentralized I/M was analyzed as a specific
program option (USEPA, 1978a and USEPA, 1978b). The essence of quality
control is to take actions which ensure that program implementation is as
consistent as possible with program design. In the case of I/M, it is more
difficult to match program design with a decentralized program than with
* For the purposes of this document, a decentralized I/M program is
any program which, at least in part, licenses privately owned
stations to inspect vehicles with respect to certain mandatory
emissions-related requirements, and to issue official certificates
or stickers as a proof of complying with those requirements.
1-1
-------
a centralized program. Simply stated, there are generally fewer stations
and inspectors who must be monitored in a centralized program; in addition,
inspectors in a centralized program, either by contract or employment, are
directly responsible to the implementing agency.
I/M QA can be directed both at auditing proper inspections and at
auditing proper maintenance. Generally, I/M QA has been oriented towards
periodic visits to licensed stations, checking records, reviewing compli-
ance with administrative requirements, and auditing the performance of
emission analyzers, with a calibration gas of known concentration (USEPA,
1982; and Pienta, et al, 1985). Virtually all States with inspection pro-
grams have some sort of periodic surveillance. Program elements usually
include not only periodic surveillance, but also referee stations for ve-
hicle owners who wish an independent check on test results (see for exam-
ple Wallauch, 1983).
There is a limit though to what structured, overt program elements can
accomplish (Carhart, 1985). Periodic overt field auditing can do much to
gather information which can indicate the direction of an I/M program; it
can also serve to bolster enforcement activities run by other parts of the
implementing agency. However, as indicated in Table 1-1, formal periodic
auditing cannot test inspectors' and mechanics' normal everyday operations
and practices directly. If, for instance, inspectors have the knowledge
and background to check for tampering of a certain emission control compo-
nent, they will almost certainly be able to demonstrate that capability to
an auditor if requested. However, in a routine workday, inspectors and
mechanics may not be as thorough as when they know they are being tested
by program auditors. They may even circumvent the requirements of the
program, such as passing cars that should be failing or passing vehicles
that are not inspected at all.
COMPREHENSIVE QA IN A DECENTRALIZED I/M PROGRAM
Discrepancies in decentralized I/M program operation have become more
evident in recent years (Cabamss, 1984; and [JSGAO, 1985). In particular,
failure rates in several programs are well under expectations, especially
when compared to centralized programs with similar cutpoints and require-
ments. This phenomenon has developed despite the existence of periodic
field auditing. As a result, questions have been raised regarding the
ability of formal periodic auditing to assure that vehicles fail when they
should, the ability of stations to record program data properly, and the
ability of decentralized I/M as a whole to be effective in reducing emis-
sions .
Failure rates may be affected by many factors. Program phenomena
which may affect overall failure rates include "false fails" (vehicle
which fail but should pass), false passes (vehicles which pass, but
should fail) and "pre-repair" (vehicles which fail and are repaired but
are reported as initially passing) . Program failure rates are only one
example of how a program parameter may need specific indepth investiga-
tion and analysis in order to reveal what is going on in an I/M program.
Auditing station operations enables program managers and others to obtain
information which would allow this and other kinds of program analysis.
1-2
-------
TABLE 1 .1
CAPABILITIES AND LIMITATIONS OF FORMAL PERIODIC
FIELD AUDITING IN DECENTRALIZED I/M PROGRAMS
Capabilities
o Can review an inspector's ability to work with the analyzer
on-site.
o Can determine knowledge of test procedures and record keeping
procedures.
o Can analyze basic approach to check for emission control
components.
o Can identify discrepancies in repairs if repair orders
are available.
o Can verify accountability of inspection certificates.
Limitations
o Cannot confirm the station's usual method of testing vehicles.
o Cannot verify or disprove by itself fraudulent repair practices.
o Cannot verify or disprove by itself inadequate knowledge or
implementation of program procedures by inspectors and
mechanics.
1-3
-------
However, as explained above, formal periodic field auditing can go only
so far in identifying discrepancies in program operation. There is,
therefore, a need to obtain information in a way which will not affect the
everyday operations that the auditor may wish to observe. This need can
be met by covert auditing.
COVERT AUDITING - THE ROLE OF UNMARKED VEHICLES
In any decentralized inspection system, there is always a need for at
least a "last resort" mechanism to remove "problem" stations from the sys-
tem. Because administrative requirements generally require a reason for
license suspension or revocation, explicit evidence of violation of pro-
gram requirements is usually needed. Frequently the only way to obtain
first hand evidence is for the implementing agency to obtain that informa-
tion through an undercover vehicle.
Covert auditing is a generic designation for information-gathering
activities in an I/M program when the auditors do not identify themselves
as anything other than ordinary citizens. Generally the term "covert au-
diting" has been used in the same way as the term "undercover vehicles,"
with the implication that such vehicles are engaged only in uncovering
consumer fraud and enforcing program regulations. To be sure, unmarked
vehicles can and have been used for such purposes, not only for decen-
tralized I/M programs, but also historically for safety programs and con-
sumer protection investigations. It is a mistake though to believe that
unmarked vehicles may only serve to support enforcement. As indicated
above in the previous section, the essential purpose of covert auditing
is to find out what is happening in an I/M program, while identifying
where program implementation is failing to track program design. Covert
auditing programs for decentralized I/M programs should, therefore, be
developed with a clear goal of what information is to be gathered, and
how the information will be used beyond just enforcement of program regu-
lations. There are many ways this information can be used to improve
program performance. Details on specific data to be gathered and how to
gather it are presented in Chapters 2-3.
OUTLINE OF A COVERT AUDITING PROGRAM: AN INTRODUCTION TO PROGRAM PLANNING
Once the specific data to be gathered is established, the basic struc-
ture of the audits themselves must be evaluated. Two basic issues which
need to be decided early are whether the covert vehicles themselves should
be set to pass or fail, and how to establish incontrovertibly the results
of the investigations.
Setting Up Vehicles to Fail or Pass
Historically there has been concern that decentralized inspection
systems, though they offer substantial convenience to the vehicle owner,
also open the way to consumer fraud. Specifically, there is an inherent
conflict-of-interest in a decentralized inspection program, where a sta-
tion stands to gain financially from failing a vehicle which is being
tested; as a result, a station mechanic may have an obvious motivation to
1-4
-------
fail cars that should be passing. False fails would therefore have to
be investigated using vehicles set to pass. To a certain extent, compu-
terized analyzers (generally those which meet BAR84 or similar specifica-
tions) make such fraud more difficult to perpetrate. The pass/fail
decision on the vehicle is made by the analyzer, and in many analyzer
systems, integrated automatic data collection systems allow the tracking
of issuance of compliance certificates.
It appears though, that as of 1986, a more significant problem in de-
centralized I/M systems is stations that incorrectly pass vehicles which
should fail ("false passes"). It may not be stated categorically that this
situation exists in all decentralized I/M programs, but rather this prac-
tice is well enough established that investigating the phenomenon should
be considered in the establishment or modification of a covert auditing
program. Some programs have noted that when an inspection fee is high
enough, a significant portion of stations and/or inspectors will purposely
pass cars in order to capture fees in as short a time as possible. To
test the false pass phenomenon discussed above, clearly vehicles must be
set to fail. Vehicles must also be set to fail to investigate the pre-
repair phenomenon.
Observation of Inspections; Documentation of Results
Actual observation of inspections may or may not be necessary to be
able to document the results of covert investigations. How auditors are to
conduct themselves and what data they are to collect will depend in large
part on program regulations. It also may depend in part on how much prac-
tically can be seen during an investigation. Frequently customers are not
allowed in service bays. In such cases, auditors may be forced to wait
outside, and first hand observation of certain parts of a test (such as an
EGR functional test) could be impossible.
Other Issues
Observation of inspections is directly related to other structural
elements of an auditing program. For example, direct observations of
inspections may not be necessary if the program has a well-equipped shop
which can thoroughly document a vehicle before and after an inspection.
Similarly, if the question of interest is whether a failing vehicle will
be passed, the documentation of the vehicle can probably be accomplished by
agency shop facility personnel. An auditor needs little training to
simply take a vehicle to a licensed station and bring back to the shop
results of the inspection. An approach of this sort may be an aid to an
implementing agency in terms of personnel; fewer requirements for qualifi-
cations makes hiring auditors easier. Details of these and other issues
are discussed in Chapter 5.
Table 1-2 presents two alternative approaches to developing a covert
auditing program and how the issues discussed above may be addressed in
different ways. There are of course advantages to each alternative, and
other alternative approaches could easily be generated. However, any co-
vert auditing program must address the issues presented here, as well as
many other considerations which are presented in detail in the following
chapters.
1-5
-------
TABLE 1.2
CONCEPTS OF COVERT AUDITING
Alternative
1
Alternative
2
Observations of
Inspections by
Auditors
Direct; must actually
observe conduct of
inspection
Only direct observations
necessary may be for
specific tampering
checks, e.g., fuel
filler inlet restrictors
Necessary Training
of Auditors
Thorough knowledge of
regulations; automo-
tive background help-
ful
Automotive training op-
tional; opens up auditor
jobs to general public
which may help to main-
tain cover of operations
Vehicles Set-Up to
Fail/Pass
Shop Facilities
Pass
Fail
Generally not necessary May range from minimal
except for service fac-
ility with analyzer
which is periodically
audited and calibrated
to extensive depending
on the range of investi-
gation areas and the
sophistication of the
set-up
Method of Documen-
tation of Results
Auditor makes a report
and will testify in
enforcement hearings
Documentation, though
supplemented by auditor,
performed primarily by
shop personnel
1-6
-------
STRUCTURE OF THIS MANUAL
The purpose of this chapter has been to describe QA in decentralized
I/M programs, and the conceptual role of unmarked vehicles in enhancing
I/M QA. Chapters 2 and 3 describes specific issues involving enforcement
oriented and data collection covert auditing programs. Chapter 4 presents
results available from existing programs. Chapter 5 then discusses pro-
grammatic issues for setting up or expanding covert auditing programs plus
suggestions for two specific model programs. Appendices are provided in
the areas of shop guidance and background materials from State programs.
Figure 1.1 presents the overall structure of the manual.
1-7
-------
FIGURE 1 .1
STRUCTURE OF THE MANUAL
INTRODUCTION
CHAPTER
Concepts
Goals
Methods
PROGRAMS RESULTS & ISSUES
Reported Data
Practical Considerations
Setting Up A Program
Model Programs
CHAPTERS 4 & 5
ENFORCEMENT-ORIENTED
COVERT AUDITING
CHAPTER 2
Issues
Structure
Approaches
Types of Studies
Relationship to Enforcement
FOR DATA-GATHERING
COVERT AUDITING
CHAPTER 3
1-8
-------
CHAPTER 2
ENFORCEMENT OF PROGRAM REGULATIONS
Every I/M program is different, and as a result the regulations which
must be enforced will be different as well. There are basic questions that
can be asked though, about almost any program (see Table 2-1). The initial
focus of covert auditing activities then should be to identify information
which can answer these questions.
FOCUS OF ENFORCEMENT ACTIVITIES
In order to structure an enforcement-oriented covert auditing pro-
gram, management must be clear on not only the overall program goals
which must be addressed (Table 2-1), but also how those goals could be
compromised. Figure 2-1 presents a way of analyzing this process.
Working from the goals of the program, it is possible to identify the
discrepancies which could compromise those goals; working from these
discrepancies, it is possible to identify program practices, or indica-
tors, which can cause these discrepancies. It is these indicators, or
program practices, which should be the focus of an enforcement-oriented
covert auditing program. Generally, these program practices will not be
detected through formal periodic auditing.
Omission of Test
Probably one of the more serious problems in a decentralized program
is the issuance of a compliance certificate or sticker without a test.
This program abuse will clearly lead to false passes; it also has the po-
tential for undermining the credibility of the program. Formal periodic
auditing cannot document this practice; it can only uncover suspicious
data totals (e.g., an extraordinarily low failure rate). Even covert au-
diting may not be able to uncover this abuse when it is performed private-
ly for customers that station personnel know well; however, well-planned
visits with a solid cover (see Chapter 5), can still uncover a lot. Be-
cause a station may omit a test on any vehicle, theoretically either pas-
sing or failing vehicles may be introduced into a station. However, if
passing vehicles are introduced into a station, auditors must be with
the vehicle so they can provide first hand information that certain parts
of the test were not performed. If failing vehicles are introduced, and
their condition before and after a test can be verified, auditors do not
have to observe the test. In many State and local programs, passing a
2-1
-------
TABLE 2.1
SUGGESTED CHECKLIST FOR DETERMINING OVERALL COMPLIANCE
WITH DECENTRALIZED I/M PROGRAM REGULATIONS
1. Are vehicles being tested according to established test procedures?
2. Are all tampering/fuel switching checks performed correctly?
3. If inspections are not being carried out completely or correctly,
what parts of the inspection pose the biggest problem?
4. Are results of the inspections recorded properly?
5. Are official tests being run only after adjustments are made?
6. Are the requirements for cost waivers being carried out correctly?
7. Are inspection certificates being issued properly?
2-2
-------
FIGURE 2.1
DEVELOPMENT OF ENFORCEMENT-ORIENTED COVERT AUDITING
GOALS OF THE I/M PROGRAM
o Identifying vehicles with high emissions
(i.e., proper implementation of test
procedures)
o Repairing vehicles to meet emission cut-
points or other specific requirements
(i.e., proper implementation of repair
requirements)
< r
PROGRAM DISCREPANCIES
o
emission of test
o
Incomplete test
o
False passes
o
False fails
o
Improper data recording which would impair
confirmation of program results
INDICATORS OF PROGRAM DISCREPANCIES **
o Sticker issuance without test
o Circumvention of rpm requirements
o Leaving the hood closed
o Badly maintained probes
o Omitting fuel filler inlet or catalyst check
o Suggestions from station personnel that
repair work is needed before a test
o High waiver rates/incompletely documented
repair orders
o Arbitrary pass/fail decisions by station
personnel (manual analyzers)
o Arbitrary data recording on manual forms
o Inserting probe into exhaust pipe of vehicle
other than test vehicle
o Lack of certificate control
** NOTE - Depends on type of program.
2-3
-------
vehicle which does not meet emission-related requirements constitutes a
violation of program regulations.
Incomplete Test
Incomplete tests may also lead to false passes. When there are un-
derhood tampering checks, if station personnel leave the hood of a vehicle
closed, this part of the inspection could not have been done. Similarly,
if the gas cap is not removed, or if the inspector does not look under the
car, fuel filler inlet and catalyst checks could not have been performed.
Rpm requirements may be circumvented by omitting use of a tachometer con-
nection. Such an omission is, of course, easier with a manual analyzer
or an automated analyzer which does not lock out improper rpm levels out
of an official I/M test. Incomplete tests can be determined with either
passing or failing vehicles. However, as with investigating test omis-
sions, there are advantages to using failing vehicles, i.e., direct obser-
vation of the inspection is probably not necessary.
Other False Passes
False passes may also be caused by inadequate understanding of cost
waiver procedures. (Obviously if no cost waivers are possible under a
particular program's regulations, this situation will not be a problem.)
Cost waivers usually require a minimum amount of work to be done on the
vehicle, and the program management would therefore have to devise ways
to demonstrate that required work was not performed. Failing vehicles
would have to be introduced in order to simulate a cost waiver situation.
"Pre-repair" of failing vehicles may also generate a false pass.
Vehicles which fail may be adjusted by station personnel and then be re-
corded as passing. This situation may even occur in programs with auto-
mated analyzers. In these programs, the analyzer normally has a computer-
ized "master menu," which allows a station inspector to choose either a
formal I/M test test mode or a manual emission test mode. If station
inspectors wish to check a vehicle's emissions prior to an official test,
they may choose the manual mode to see if the vehicle passes or fails.
If they find that the vehicle fails, they may simply perform the adjust-
ments the vehicle needs to pass, and then initiate the official test.
Badly maintained probes may leak, thereby diluting the sample and
causing a false pass. Automatic leak checks and flow checks, especially
on automated instruments, may minimize this phenomenon. However, to the
extent that leaky probes contribute to a dilution of a sample, additional
false passes will decrease the real failure rate.*
Manual analyzers exacerbate QA problems in a decentralized program,
because the pass/fail decision is left up to the individual inspector/
mechanic. This situation could lead to either a false pass or a false
fail decision (see below). There are no comprehensive studies or reports
* Badly maintained probes may of course also be identified through
formal periodic QA audits as well.
2-4
-------
available on the actual motivations of mechanics who purposely pass or
fail vehicles falsely. However, anecdotal information from current State
and local programs suggest that licensed stations do believe that their
return from an I/M program is maximized by test volume, and that test
volume is maximized by minimizing time associated with each test.
False Fails
False fails, either purposeful or accidental, are the subject of sub-
stantial concern.* When a station purposely fails a car which it should
pass, it is generally considered consumer fraud. A false fail can be
generated either by incorrect reporting of the result on the data form in
a program with manual analyzers, or by probing the exhaust pipe of a car
known to fail emission standards in a program with automated analyzers.
Deliberate disablements such as secretly disconnecting spark plug wires,
or misrepresenting the status of emission control components (for pro-
grams with tampering checks) are other ways to create false fails. It
should be pointed out that there is no data available to date which sug-
gests that false fails are a concern on a program-wide basis; rather it
only appears to be a problem on a station-by-station basis.
Based on historical experience with safety inspection programs,
some covert auditing programs have started by focusing on false fails.
Colorado, for example, initiated its program by investigating for false
fails, and found very few. The Colorado program has since changed its
emphasis and is now investigating false passes. As mentioned above,
reports from other programs also support the concept that false passes
are more of a problem in I/M programs than false fails. If it is deter-
mined that false fails are a real concern, then cars must be set up to pass
in order to investigate stations of concern.
Improper Data Recording
Improper data recording, while in and of itself is not a program
discrepancy, may prevent an accurate representation of program operations
from the data collected and analyzed. In programs with manual analyzers,
this problem is exacerbated by the fact that the inspector must accur-
ately read and record the data manually. Illegible and ambiguous markings
of inspection data forms may be discovered either through central office
review of records or through formal periodic field auditing. However,
some of the practices described above in this section overlap the area of
improper data recording and may become confused with them. For instance,
it may be important to know how much of a low failure rate is due to pre-
repairs versus false passes due to emission tests being omitted. The
* A false fail should not be confused with an error of commission, which
in a testing context means a vehicle which would pass an FTP but fails
a short test.
2-5
-------
first is a data recording problem and does not necessarily indicate that
program effectiveness is being compromised. The second is a program abuse.
PRIORITIZING STATIONS: AN AID TOWARDS DOCUMENTATION AND DETERRENCE
There are many ways of prioritizing stations, but they can generally
be classified into periodic random auditing, probable cause auditing, and
"problem station" auditing. For the purposes of this document, these
classifications can be defined as follows:
Periodic/Random Auditing - Auditing with a formal or informal
schedule to visit all stations in program within a specified
time period (i.e., strictly speaking, no prioritization).
Probable Cause Auditing - Auditing which targets particular
stations on the basis of the frequency of complaints, discre-
pancies on data records, etc.
"Problem Station" Auditing - Auditing which uses covert investi-
gations only to obtain evidence against a station for the pur-
poses of suspending or revoking that station's license.
Depending on the program, these systems could be combined into a hybrid
approach.
Currently, there are programs that utilize each of these approaches.
Colorado utilizes a combination of periodic auditing and probable cause
auditing. The Department of Revenue, the administering agency in Colorado,
visits each licensed station once a year. They also keep an informal list
of "target" stations, which are visited as convenient. New York is a good
example of a State which utilizes probable cause to target stations.
Though the Regional DMV offices do receive computer printouts of station
records, complaints and tips from periodic inspectors are the main fac-
tors used to prioritize stations for covert visits. As a result, the
State visits from 10-25% of the stations per year. In some ways, "prob-
lem station" audits really are a subset of "probable cause" audits. In a
program with a "problem station" approach, fewer audits are conducted,
because the primary reason for performing an audit is to gather evidence
against a station which is almost certainly committing serious violations
of the program regulations. As an example of how one State decides to
perform covert visits, a summary of reasons for covert audits in Califor-
nia is presented m Table 2-2.
Ultimately, the choice of an auditing prioritization scheme depends
on the goals of a covert auditing program, i.e., documenting and deterring
the practices which lead to program discrepancies (Figure 2-1). To be
sure, if stations know that covert auditing is not conducted except in
rare cases, stations' standard operating practices for I/M will probably
not change much. While periodic covert visits to every station may not
be possible due to limited agency funding, having covert audits on a
regular basis clearly increases the chance for documenting and deterring
detrimental program practices. Therefore, the need for efficient priori-
tization of stations is clear, and prioritization, even with relatively
few visits, can still lead to deterrence (see Figure 2-2).
2-6
-------
TABLE 2.2
REASON FOR PRIORITIZING STATIONS FOR
COVERT AUDIT BY STATION TYPE IN CALIFORNIA
(JULY 1984 - JUNE 1985)
New Car Used Car Independent Service Total
Dealers Dealers Shops Stations
Random Check 20 16 363 230 629
Complaints from
the Public 5 4 92 84 185
Suspiciously
high certi-
ficate issuance 0 0 7 4 11
Informant tips 2 5 34 28 69
Tips from QA
contractors
(formal auditors) 0 0 1 2 3
Referee actions 0 15 17
Follow-up visits 2 10 110 115 237
Other _0 _J_ _1£ _M_ 22
Total 29 37 622 475 1163
2-7
-------
FIGURE 2.2
CONCEPTUAL QA FRAMEWORK FOR ENFORCEMENT-ORIENTED COVERT AUDITS
EVALUATION OF
SERIOUSNESS OF
POTENTIAL VIOLATIONS, ETC
AVAILABLE
PERSONNEL
ENFORCEMENT ACTIONS
DETERRENCE
AND EDUCATION
PRIORITIZED STATIONS FOR
COVERT INVESTIGATIONS
COMPLAINTS, TIPS, AND
SPECIFIC STATIONS SCREENED
FROM COMPUTERIZED DATA BASES
IDENTIFIED
PROGRAM
PRACTICES
2-8
-------
Because of the increasing computerization of I/M data, with and
without computerized analyzers, station performance is much more subject
to scrutiny than it was in the late 1970's. However, few, if any, State
and local agencies actively use computerized summaries currently to
prioritize their undercover activities. One state, which is just start-
ing an effort in this direction, is the State of California. A copy of
the California station performance summary report is provided as Figure
2-3. Note that this form is only a sample and that many other comparisons
may also be generated from similar data bases. However, several categ-
ories will depend on data generated and recorded by an automated analyzer.
For example, "No. and % with fast repairs," depends on an analyzer with a
clock or timer, so that "fast repairs" may be defined for a screening of
a computerized data base. (See Appendix B for a full explanation of the
California summary report).
As discussed above, other ways of prioritizing stations include
methods that have traditionally been used by safety programs with probable
cause covert auditing, including:
o complaints by vehicle owners;
o tips from QA auditors;
o reputation of a shop's activities from
other sources (police, etc.).
When combined with computerized data results, a list of target stations
can rapidly be developed. Each State or local program should have its
own criteria to determine what stations should receive priority for
covert visits. Among the questions that could be asked are:
How serious are the violations to be investigated?
How prevalent do these violations appear to be?
How do the violations to be investigated obstruct the
proper implementation of the program?
How will covert visits to these stations help to deter
detrimental practices both at this station and the
program as a whole?
The last question is addressed below under "Penalties, Publicity, and the
Durability of Deterrence."
The question of how to prioritize stations then revolves around what
program practices need investigation. As shown in Table 2-3, observation
of the inspection is frequently not necessary if all that must be estab-
lished is that a station is passing cars that should fail. Direct obser-
vation or the equivalent is usually necessary to document what specific
practice is occurring (see Table 2-3, Note 2); however, in many instances
introducing a failing car can still allow the identification of a false
pass, though the exact reason may not be identified.
Therefore, depending on what program practices are of interest, cer-
tain data can be screened in computerized data bases. This information
then can be used iji addition to information from the more traditional,
informal channels, (such as tips from the QA auditors), to prioritize
stations.
2-9
-------
figure 2-3
California Station/Mechanic Summary
CtF^'TfFM C f CCNS LCC» AFFAIRS
6LCEAL OF AUTOMOTIVE REPAIR
STATION / CFCMNIC EVALUATION R f P C f T
CJTA CCLLECTEC 2ND OLARTEF 1985
FHCNE: (415)
PAGE 1
flELC.CffICE S£
( > (x) ( Xx xxx >
(x>( xxx x > < )
( ) ( XXX ) ( 3 ( )
STA »
nc d
I.I££I-S3 £ 12S11££.
TCT AL
iesis
1 4 1
141
R R CM
££H
2!
( X C f V F R OK CISTPIC1
£«U_E«IE .tc.
7 S E L 0 ui
7 £ E L C w
• 2 6
¦ 2 f
AVERAGE
-LQ.
"2C
-2C
TCTAl
2 £S2 £
-A-IESI-SI£2ISI2 CS.
F RCNT
fill
60
eO
( X DEV FROK C I S TRI CT AVERACE
fail.BfilE t£:BCL£ £ QzE cut
30 AECVE
3C AecVE
¦23
¦23
-1 7
-17
STA «
~ E C t
N)
I
STA «
C e C *
1 u3
K CERTS BY
INACCUFATE
PRE V EMS
I5I.3EUIS
2
2
PERCENTAGE
UNCERHCCP
Xb£t£UE4£I
1 s
1«
79
80
* DEV FROC
iTATEUICE
CILL!ION
mem...
c. 2
C.2
ERRCNECLS
ENTRIES Crv
fUN£l,.£S.
IDLE SPEED
DEVIATION
I TC A TEST
S-E££.2_C£V
u
7 4
35.0
35 .C
NC. ANC X
WITH FAST
BEESIES..
3 7 4.9
3 74 .9
NLPBER OF
RETESTS
»0J CC TC
IbBE5tl£lE
0
C
X A TEST
WITH NO
BEEfilBS..
0.0
O.C
F E RCENT
TESTS WIN
113 RFC OF
ItfiESbCLB.
23.5
23.5
* fULTIFLE
TESTS
-I.. -4-.
4.9
4.9
C .0
C .0
NC. ANC X A
TtSTS UTH
NC CATCHING
I-IESI
19.9
19.9
NC. ANC X CF
FAILED TESTS
UUU.CEEIS..
3 9.3
3 9.3
27
NOTE- See Appendix B-4 for descriptions of headings in this table.
-------
TABLE 2.3
SUMMARY OF POTENTIAL EVIDENCE FOR SPECIFIC REPAIR PRACTICES
Program Practices
Potential Evidence
Passing/Failing
Vehicles
for Covert Audits
Direct
Observation
Needed
Sticker/certificate
without a test,1
or without regard to
the results of a test.
High passing rates, fast
tests.
Passing
Failing
Yes
No
Circumvention of rpm
requirements
1
Badly maintained probes.1
Leaving the hood closed.1
Omitting fuel filler inlet
or catalyst check.1
Suggestions from station
personnel that repair work
is needed before a test.
Discrepancies in rpm and Passing
idle speed records.1 Failing
Discrepancies in dilution records.^ Both
High passing rates (tampering Passing
inspections). Failing
High passing rates (fuel filler Passing
inlet/catalyst checks). Failing
High failing rates, high repair Both
costs (possibly).
Yes
Sometimes^
Yes-*
Yes
Sometimes2
Yes
Sometimes^
No
Fraudulent fails (general).
High failures, especially for
specific emission control components
and HC.
Passing
No
-------
TABLE 2.3 (continued)
SUMMARY OF POTENTIAL EVIDENCE FOR SPECIFIC REPAIR PRACTICES
Program Practices
Inaccurate waiver
processing.
Potential Evidence
High waiver rates; short time
periods betweeen initial and
second tests.4
Passing/Failing
Vehicles for
Covert Audits
Falling
Direct
Observation
Needed
No
Arbitrary pass/fail
decisions.
High passing or high failing
rates.
Both
No-
Arbitrary data recording
on manual forms.
High passing or high failing
rates.5
Both
No5
Inserting exhaust pipe
into vehicle other than
a test vehicle.1
Small standard deviation of
test scores.
Failing
Yes
Waivers to 1981 and later
vehicles which are serviced
by untrained personnel.
High waiver rates for 1981 and
later vehicles; short time periods
between initial and second tests;
high incidence of specific, un-
anticipated repairs for late model
vehicles, carburetor repairs on
1981 and later vehicles and fuel-
mjected vehicles.
Failing No
-------
TABLE 2-3 (Continued)
Notes to Table 2-3
1. High passing rates may indicate a combination of phenomema occurring
simultaneously; those phenomena which may contribute are noted with
a "1".
2. Direct observation is needed to demonstrate a specific practice. A
vehicle may still be a failing vehicle and the station could be cited
for passing a failing car. If it is desired to determine whether a
specific station is, for example, adjusting engine parameters in a pre-
repair phase, then a program must develop a way to document the status
of those parameters before and after the test. See Chapter 5 under
"Shop Issues."
3. Badly maintained probes may be checked through formal periodic audit-
ing as well.
4. Data probably available only from certain automated analyzers.
5. Generally, only with manual analyzers. Direct observation is not
necessary if passing or failing status of the car can be demonstrated
before and after the test.
2-13
-------
THE ISSUE OF ENTRAPMENT
Based on surveys of operating programs in 1985, covert auditors
specifically do not ask inspectors or mechanics to perform work contrary
to program regulations. To do so would constitute entrapment,* i.e.,
inducing station personnel to commit a program violation. Instead,
auditors simply ask for an inspection.
Among the specific concerns voiced by State and local programs are
that:
o introducing a car that has altered or missing emission control de-
vices, or a car known to fail emission standards, may constitute
entrapment;
o altering emission control devices on vehicles, even in the inter-
est of air quality improvement, may be a violation of Federal or
State law;
o driving a vehicle on public roads with altered emission control de-
vices may be a violation of State law.
A review of other states may be helpful to understand these and other en-
trapment issues. To be sure, entrapment is a matter of law, and if chal-
lenged, a State or local program might have to defend itself in court.
However, as a practical matter for the States that do use failing and al-
tered cars, entrapment has not been a major issue. Generally, presenting
a falling vehicle (even with altered emission control devices) has not
been considered entrapment, because stations are not being induced into
doing anything that is contrary to program regulations; all the station
has to do is to fail the vehicle. For state air pollution control laws,
obviously the State air pollution control agency should be consulted. It
should be noted that with respect to Federal laws and regulations, the U.S.
Environmental Protection Agency has allowed emission control modifications
for this purpose on a case-by-case basis.
PENALTIES, PUBLICITY, AND THE DURABILITY OF DETERRENCE
Figure 2-2 indicates that the "bottom line" of an enforcement-based
covert auditing program is deterrence and education. The reason for this
dual goal is that deterrence from detrimental program practices and educa-
tion towards implementation of the program as it is designed are both im-
* Black's Law Dictionary defines entrapment as "the act of officers or
agents of the government in inducing a person to commit a crime not
contemplated by him, for the purpose of instituting a criminal prose-
cution against him" (emphasis added).
2-14
-------
o misunderstanding or ignorance of the program;
o partial or total disregard of the program;
o fraudulent implementation of the program requirements.
For station personnel who do not really know I/M program requirements,
education and retraining is essential. However, some state's experience
(e.g., Colorado) indicates that continuing conferences for some stations
seems to perform no useful function; at this point, stations who continue
to perform inspections or repairs incorrectly must be considered as risks
for correct implementation of the program. Nevertheless, some stations
do manage to revise their procedures.
Several State and local programs interviewed mentioned that the
swiftness, sureness, and severity of penalties were the key to successful
deterrence from a covert auditing program. The quality of deterrence may
be debatable (see below), but qualitatively there seems to be some merits
to this argument. Swiftness of penalties may be aided by fining authority
("ticketing") by the regulatory agency. As of January 1986, California
was granted this authority, and determining the impact of this approach
may be possible.
The deterrence effect of publicity must also be considered. First,
however, who publicity will reach must be assessed. Among the method I/M
programs have tried are:
o trade association newsletters;
o press releases;
o courses through vocational educational institutions
and service station organizations.
All of these options are to be encouraged, but realistically programs must
ask first who will be reached by specific publicity. A trade association
newsletter, for instance, is always a meaningful approach of outreach to
the service industry, but in and of itself, it may not be sufficient.
The Regional Office of New York DMV, for instance, reports that only 25%
of licensed stations belong to a professional trade organization (e.g.,
the Long Island Service Station Association). In other words, outreach
and general publicity to maximize the impact of limited covert auditing
must be looked at as a total package.
2-15
-------
CHAPTER 3
COLLECTION OF DATA ON PROGRAM OPERATIONS
Covert auditing presents opportunities to I/M program management to
collect information on what is happening in the program without affecting
inspectors' and mechanics' behavior. This advantage, though, is not and
should not be focused on enforcement activities alone. information
gathering about the program in general can be enhanced by having people
in the field watching what happens.
Similar to enforcement-oriented auditing, auditing for the purpose
of data collection must be well-planned from the beginning. It is impor-
tant to know what information to look for, plan a system to gather that
information, and analyze the information in a systematic way.
Among these areas which could be the subject of investigations are:
o Failure rates
o I/M effectiveness
o Repair cost
o Station's approach to I/M
o Types of repairs
o Warranties
Table 3-1 presents sample topics which could be explored within some of
these areas.
COORDINATION WITH ENFORCEMENT-ORIENTED COVERT AUDITING
Not every program can do a special study, and setting up methods for
collecting special data on a routine basis often takes time. Collecting
data to characterize program operations may be a valuable first step in
setting up an enforcement-oriented covert auditing program.
As discussed in Chapter 2, the goals of an I/M program ultimately
can be compromised by detrimental program practices (e.g., fraudulent
fails, false passes, etc). With limited resources for covert auditing,
it is more cost effective to focus the potential deterrent effect of a
covert auditing program on the practices which hurt the effectiveness
and/or credibility of the program the most. To be able to determine what
practices to highlight, program management must know how prevalent dif-
ferent practices are.
3-1
-------
TABLE 3.1
POTENTIAL AREAS OF DATA-GATHERING INVESTIGATION
FOR COVERT VEHICLES*
Subject
Sample Topics for Investigaton
I. Failure Rates
Are stations repairing or adjusting
vehicles and not reporting failures?
How are these actions affecting the
reported failure rate in the pro-
gram?
II. *Repair Cost
Are some stations charging a maxi-
mum amount (i.e., equivalent to
a cost waiver expense) for simple
adjustments?
Are simple adjustments being per-
formed for free?
Is there a difference in repair
cost in terms of the type of
service outlet?
III. Warranties
How responsive are dealerships
to warranty claims?
IV. Stations' Approach to I/M
Why do station personnel make
programmatic mistakes? Fraud?
Laziness? Ignorance?
I/M effectiveness and repair type investigations may be somewhat
more difficult to develop, based on the number of vehicles that
would be required to develop a good data base.
3-2
-------
What may be needed is a baseline, i.e., a quantitative description
of ongoing successes and flaws at a point in time. In other words, gather-
ing data in many cases provides an opportunity to determine how a program
is improving as a result of a covert auditing program. The issue of base-
line development is discussed in more detail in Chapter 5.
Managers of many I/M programs feel fortunate to have enough resour-
ces for the basic activities of program operation; extra data collection
and analysis may be considered to be a beneficial but expendable part of
a program. While at times special studies can be resource-intensive, in
many cases they can be integrated into existing operations (see Chapter
5). Regardless, probing into the operations of an I/M program can give
insights into not only what is happening, but why. Without reliable, ac-
curate information on what is really happening in I/M programs, managers
cannot really manage. If, for example, real failure rates are low and
emission reduction effectiveness is less than the effectiveness planned
in the program design, finding out why is the first step toward correct-
ing the problem.
SPECIFIC TYPES OF STUDIES
Gathering information about the operations of an I/M program can ei-
ther be through a special study or on a continuous basis. In any event,
studies must be organized. The following discussion presents a brief des-
cription of how to approach the four subjects presented in Table 3-1.
Table 3-2 presents some potential comparisons that can be generated as a
result of focused data collection analysis.
Failure Rates
Investigations of failure rates have been discussed in some detail in
Chapters 1 and 2. If false fails and false passes are being investigated,
passing vehicles and failing vehicles are usually needed. Special docu-
mentation/verification may be needed if the adjustment (or non-adjustment)
of certain engine parameters is to be studied. Failure rates are of
interest for all of the subgroups listed in Table 3-2 (different types of
analyzers, service outputs, etc.) particularly if a program wants to
identify where (if anyplace) low failure rates are concentrated.
Repair Cost
As with I/M effectiveness studies, repair cost studies need data on
the types of maladjustments and poor maintenance found in an area. Model
year groupings and types of service outlets are key groups to investigate.
For 1981 and later vehicles, vehicle preparation is critical.
Warranties
Warranty studies will of course be focused on dealerships, and will
involve only late model vehicles. Again, information on the sorts of fail-
ures that dealerships actually face is useful.
3-3
-------
TABLE 3.2
POTENTIAL COMPARISONS TO BE USED IN CHARACTERIZING
I/M OPERATIONS*
Analyzers by
Manufacturer
Type of Service
Outlet
Vehicle Model-
Year Groupings
Failure Repair Stations'
Rates Costs Warranties Approach
Physical Location
of Stations
* NOTE: Warranty investigations will be oriented primarily towards
individual stations and possibly individual manufacturers.
3-4
-------
Station's Approach to I/M
Investigating in some detail what stations do in an I/M program
(how much diagnostics are performed, what a "typical" inspection is like)
may be one of the more difficult areas to investigate. However, to the
extent that the reasons for inadequacies in inspections or repairs are
determined, remedies (e.g., retraining of inspectors) are much easier to
develop.
Other Potential Areas of Investigation
I/M effectiveness studies may be difficult because FTP testing is
generally needed to demonstrate mass emission rate reductions. As an al-
ternative, idle emission reductions are frequently used as substitutes
for summary reports to legislatures, the public, etc. Special vehicles,
set-ups, and documentation are generally not needed. However, it is
helpful to have a breakdown of typical maladjustments so that the repair
industry may be tested with a set of vehicles which reflect the vehicle
population as a whole. Types of service outlets, vehicle model year
groupings, and vehicle makes are of particular interest in I/M effective-
ness studies. For 1981 and later vehicles, vehicle preparation is criti-
cal (see Chapter 5 and Appendix B).
Activities in Current Programs
Data collection and analysis studies are either in progress or
planned in several areas. For example, the Salt Lake City/County
Health Department is considering a permanent group of stations for
data-gathering purposes. The Colorado Department of Revenue, in con-
junction with the Colorado Department of Health, is investigating
responsiveness to warranty claims by dealerships in the Denver metro-
politan area. An I/M effectiveness study is being performed by the
California Air Resources Board (CARB) in Southern California; utiliz-
ing the CARB laboratory in El Monte, California, full transient driving
cycle emission tests are being conducted.
3-5
-------
CHAPTER 4
EXAMPLES OF RESULTS FROM CURRENT AUDITING PROGRAMS
As discussed above, the goal of an enforcement-oriented QA covert
auditing program is to deter and document program practices detrimental
to the I/M program. Education is an essential part of that deterrence.
It is difficult though to define what constitutes effectiveness of a
covert auditing program quantitatively.
Ultimately a covert auditing program is effective if it reduces the
incidence of detrimental program practices. Data on the prevalence of
different types of program problems in decentralized programs is general-
ly not available. It is, therefore, impossible to present an analysis of
the effect of different types of covert auditing programs. One measure
of effectiveness that some programs use in percentage of hits:
H = h 100
I
Where:
H is the percentage of hits;
h is the number of investigations where investigators
found what they were expecting to find; and
I is the total number of investigations.
In some programs, H approaches 100%. Percentage of hits as a measure of
effectiveness though can be rather misleading. First, it may be based on
both safety and I/M requirements in combined programs. If a safety
program requires very detailed test procedures and the covert auditors are
very strict, then it is relatively easy to find that the station is not
carrying out a full test procedure. Second, to the degree that auditors
perform audits on a "problem station" basis, they maximize the chance that
H will be 100%. Third, as programs start to use measures which are only
indicators of possible problems, H may be expected to decrease. There
are, therefore, many potential factors which affect the percentage of
hits, and it is useful to investigate other ways to determine the effec-
tiveness of I/M programs.
PREVALENCE OF DETRIMENTAL PRACTICES
The first step to defining how effective a covert auditing program
is, is to define the prevalence of the practices the auditing is designed
to control. Table 4-1 provides summary data from the Colorado program.
4-1
-------
TABLE 4.1
SUMMARY OF COVERT INVESTIGATIONS IN THE COLORADO I/M PROGRAM
(January 1 - November 22, 1985)*
Improper
Issuance
of a Sticker
Improper
Visual
Inspection
Other
Visual Failures
(total attempts = 535)
7.5%
4.9%
6.3%
Emission Failures
(total attempts = 381 )
3.7%
0.8%**
5.9%
Total
(total attempts = 891)
6.8%
3.2%
7.2%
NOTE - Percentages do not total, as multiple violations may occur from
one audit. These totals also do not reflect violations which
were addressed in an office conference only.
* Includes only those investigations which have gone to formal
hearings by 11/22/85.
** Excludes one vehicle which had both emission and visual failures;
n = 356.
4-2
-------
In Colorado at least, it appears that inspectors are less likely to ig-
nore an exceedance of the I/M outpoints than they are a missing or modi-
fied fuel filler inlet restrictor or catalyst.
As discussed above, there is very little data available which indi-
cates over time what covert auditing approaches have worked in terms of
deterring detrimental program practices. Therefore, it is helpful to
characterize program operation so that a covert auditing approach can
either be developed or modified to focus on specific areas of a program.
Development of a data baseline may be a useful first step towards this
goal.
DETERRENCE THROUGH PUBLICITY
An example of what can be done in terms of publicity for covert
auditing activities occurred in California in the summer of 1985. The
California Bureau of Automotive Repair (BAR) had been receiving complaints
from buyers of used cars that cars were being sold with altered or missing
emission control devices. Because California runs a periodic I/M program,
which includes includes numerous tampering checks, purchasers of vehicles
falsely certified for sale would then be faced with paying for modifica-
tions of the vehicles to be able to pass I/M requirements. To substan-
tiate the complaints, BAR bought several sample vehicles which had missing
or altered components and removed the vehicles to the BAR shop to verify
and document their condition. Then, in a coordinated media approach, with
the aid of the California Highway Patrol, the car lots which sold the cars
were cited for violation of specific state laws and regulations. There
was substantial press coverage, not only in Sacramento, but in other
parts of the State as well. Informal feedback indicated that prospective
vehicle purchasers had heard of the state's operation on the car lots,
and were now asking questions of the car lots before buying a vehicle. The
owners of the car lots now appear to be aware of the fact that the state
could still occasionally buy a car off a lot.
Of course, this example presents one incident that could affect one
portion of an I/M program. There is no indication of how long the deter-
rence generated by this operation will last, no matter how good it was
initially. However, this incident does underscore the concept that publi-
city about ongoing operations is a useful part of an overall strategy to
encourage both deterrence and education. It also raises the possibility
of using consumer awareness to help in that process.
DEVELOPMENT AND DISPOSITION OF CASES
As described above, California does prioritize its audits and as a
result does try to target stations which are most likely to be performing
inadequate inspections or repairs. Table 4.2 presents a summary of over
1400 audits performed in California. As presented in the table, a sub-
stantial portion of the audits (61.3%) resulted in a documented violation
4-3
-------
TABLE 4.2
ADMINISTRATIVE SUMMARY
COVERT AUDITS IN CALIFORNIA*
Results of the Investigation (Total = 1404)
#_
%_
No violations
41 2
29.3
Aborted
87
6.2
Violation
861
61 .3
Other (Please specify)
44
3.1
Disposition of Case (Total = 1317)
Office conference
390
29.6
Submitted for disciplinary action
119
9.0
Submitted for administrative action
58
4.4
NOV issued
283
33.8
Closed no action
445
33.8
Pending
22
1 .7
Type of Station (Total - 1173)
Dealer (new)
29
2.5
Dealer (used)
37
3.2
Independent
628
53.5
Service station
479
40.8
* Approximately 50% random and 50% targeted audits.
4-4
-------
(but not close to the nearly 100% "hit" percentage reported by some pro-
grams). However, California (like New York) does attempt to minimize the
time taken per case, and reserves administrative and disciplinary actions.
Out of over 1400 audits, actions of this sort accounted for just over 13%
of the total. Office conferences, a much less time consuming action, were
held for more than twice as many cases.
4-5
-------
CHAPTER 5
PROGRAM ISSUES
Chapters 1, 2, and 3 have discussed why covert auditing is important
and how it can be focused towards both enforcement and data collection.
Chapter 4 discussed results to date of several current auditing programs.
The purpose of this chapter is to describe specific program elements, and
to discuss how different model programs can be assembled.
PROGRAM ELEMENTS
Among the decisions programs will have to make will be:
o how to maintain the cover of auditors
o whether to orient the covert program to passing
or failing cars
o what kind of people to hire as auditors and how
to organize a team
o what kinds of vehicles to procure and where to
procure them
o how to set up vehicles physically for covert
work and how to document audit results
o what kind of shop facilities are necessary
o how to approach administrative, civil, and criminal
enforcement actions
o how to publicize auditors' activities
o how to use the results to improve or manage the program.
Each is discussed below.
Maintaining Cover
Maintaining cover is critical for the effective operation of the
covert audit. If cover is compromised, at the very least station per-
sonnel may play along with the situation and attempt to perform in
accordance with what they believe the requirements of the program are.
In this situation, the covert audit may in fact be collecting informa-
tion which does not reflect ongoing practices in the program.
Many programs, including several with long experience with covert
investigations for safety programs, emphasize how stations frequently
ignore obvious tip-offs to an ongoing covert audit. Even when cars and
auditors are generally known, auditors are frequently able to perform an
5-1
-------
effective covert audit. (One state even reported effective audits using
a State vehicle with a State seal on the vehicle's door.) However main-
taining cover must not be taken lightly. Even if some audits can proceed,
two problems can surface if a station detects an audit and "plays along":
o the deterrent effect of being caught in a covert audit
will only be felt by those who fail to pick up the clues.
o information gathered by auditors will not be representative
of program operation, and neither the auditors nor the
program management will know it.
Accordingly, good planning is essential to maintaining cover, and
program managers must therefore consider how an auditor's cover could be
lost. Among the clues to stations which should be considered are:
o the auditors themselves
o auditors' dress
o the vehicles themselves
o the combinations of auditors and vehicles
o the auditor's story (is it plausible?)
To the extent the vehicles and auditors are known, an auditor's cover is
suspect. A combination of people, vehicles, etc., is desirable. In addi-
tion, auditors must develop a routine for themselves and managers of audi-
tors must review every part of an investigation package (the combination
of auditor/dress/vehicle/story) to make sure that suspicions are not arou-
sed. Periodic review of results in the field is helpful to adjust the
overall program to eliminate potential problems.
A checklist of questions is provided here as Table 5-1 which can be
used to assess the reasonableness of the audit approach.
Passing or Failing Cars
This issue has been discussed at some length in previous chapters,
and what makes sense may vary from program to program. Ideally, a program
should initiate a data gathering study to see what practices are occurring
in the field. A selection of vehicles aid in characterizing program
operations (see Chapter 3). In the absence of any data about the practices
occurring in a specific problem, using failing cars at least in part is
a good starting point. Given the false pass phenomenon and the ability in
many cases to avoid the need for direct observation of the inspection,
failing cars may provide an information baseline to build on. In addition,
even if there are doubts in a particular jurisdiction about the legality
of using failing cars in a specific case, using cars in a data collection
effort can never be considered entrapment.
Personnel
Position descriptions for auditors, and in some cases, supervisory
auditors are presented in Appendix B-3. It should be noted that the Cali-
fornia position description provided is for a Bureau representative who
works with licensed stations and auditors.
5-2
-------
TABLE 5.1
CHECKLIST OF QUESTIONS FOR ENSURING THAT COVER IS
MAINTAINED DURING A COVERT INVESTIGATION
o If the auditor has to leave a phone number, is the number local?
o If the vehicle must be left, is there someone at that phone number
to answer and maintain the cover of the investigation?
o Is the combination of the auditor (including the way the auditor is
dressed) and the vehicle reasonable for the type of clientele and
business normally performed by the station under investigation?
o Is the service the auditor asks for consistent with service requests a
station expects?
5-3
-------
There are of course many ways that State and local governments can
organize their teams, and there are many constraints to work with. One
interesting approach to this issue has been in California where the au-
ditors themselves have minimal responsibilities, and basically only in-
troduce the vehicle into the target station with a plausible story. The
pre- and post-audit documentation by the shape of the vehicle allows the
verification of what happened during the audit. As a result, the audit-
ors themselves may be anyone who is able to maintain a reasonable cover.
California uses former State employees, other retirees, local law stu-
dents, and a variety of other people on a pay-as-you-go basis. This ap-
proach avoids having to hire new employees.
Other programs have initiated similar approaches; employees of the
implementing agency or sister agencies, have been used with similar suc-
cess. Working with a limited number of employee-auditors can work also,
but then program management needs to be more aware of potential factors
which could compromise the auditors' cover. A group of diverse auditors,
plus a group of diverse vehicles (see below) eliminates this problem.
Organization of Personnel
The two basic approaches used by programs to date (other than the
California approach described above) are to:
o use dedicated covert auditors; or
o split the time of auditors between covert and formal
auditing.
There is no consensus on the better approach. Dedicated covert auditors
may have a very we 11-developed sense of their program responsibilities,
which may aid them in developing a commitment to their task. Splitting
the time of auditors on the other hand, may give auditors a better sense
of the program as a whole, and gives more variety to an auditor's day.
Colorado uses a team approach where one one auditor does covert
audits in another auditor's formal auditing territory and vice versa;
this approach may allow for trading of information about particular
stations.
Vehicles
The variety of vehicles, like the variety of auditors, is a plus for
the program because it helps to maintain a program's cover. Programs have
used many different types of vehicles. Vehicles have been borrowed from
employees, and purchased outright. New York has had success in procuring
vehicles from police impound lots.
The type of vehicles chosen will depend on the sort of audits planned.
For example, it is difficult to try to introduce an inexpensive compact au-
dit car into a luxury car dealershipl At the very least, unexpected sta-
tion/audit car combinations can make station personnel wonder what is hap-
pening. Borrowing cars on an as-needed basis, and having a designated
fund to pay for their use, is one method around this problem.
5-4
-------
Similarly, if a program plans to investigate stations' work on 1981
and later vehicles, then late model vehicles must obviously be utilized.
(This approach is important for testing late model maintenance practices
and tampering/fuel switching checks.) Setting up 1981 and later vehicles
to fail is difficult, and may involve modifications which would prevent
borrowing such vehicles.
Setting up Vehicles for Audits
How a vehicle should be set up again depends on what the audit will
be testing. There are only two requirements: that the vehicle clearly
be what it is supposed to be (i.e., a failing test car does in fact
fail), and that the program can prove it. The amount of proof necessary
will vary from state to state and area to area.
Setting up vehicles to pass still requires access to an emission ana-
lyzer which is regularly calibrated. However, scrutiny of the analyzer
and of the emission levels of audit vehicles themselves can be expected to
increase as the use of failing vehicles increases and stations are cited
for passing failing cars. More than anything else, emission levels and
particularly the pass/fail condition of vehicles must be repeatable. Work
undertaken by the Colorado Department of Health has shown that this is
particularly difficult for 1981 and later vehicles.* (It is worth noting
that if a hollowed out catalyst is used on an audit car, the engine sound
from the car may seem slightly abnormal.)
Setting up pre-1981 vehicles either to pass or fail is easier because
they are more adjustable. However, if a program wishes to check what was
done to a vehicle, then there must be very carefully documented verifica-
tions of the vehicle's condition. California performs such verifications,
not only for I/M, but also as a part of its covert audits to enforce con-
sumer protection laws on vehicle repairs. In California, a vehicle's con-
dition is thoroughly documented before it is sent to a station. For exam-
ple, to check whether spark plugs are actually charged, new plugs will be
gapped, photographed, aged, and installed prior to a covert investigation.
After the covert visit, the documentation process is repeated. It should
be noted this approach is not necessarily the model for all covert pro-
grams; the degree of documentation necessary is dependent on what is
being investigated and what is needed for successful disposition of cases
and actions resulting from audits.
Shop Facilities and Tools
Shops for vehicle preparation can be quite simple. The basic requir-
ement is an enclosed space where work may be performed on cars, preferably
with room for a few tools and an emission analyzer. Some of the more spe-
cialized tools for working on 1981 and later vehicles are shown in Figure
A-1 of Appendix A.
* See Schroeer, John, "Covert Vehicle Modification," Colorado Department
of Health, Denver, Colorado, 1985.
5-5
-------
Enforcement
Enforcement may take many forms. The typical hierarchy in current
inspection programs include:
o warning letters
o office conferences
o administrative hearings
o license suspension and/or revocation
Criminal actions sometimes are not pursued by programs because they be-
lieve that enforcement purposes are frequently served adequately by the
results of administrative actions. Sometimes actions against stations may
not even reach the administrative hearing stage. New York, for instance,
utilizes a plea bargaining approach which substantially reduces their
hearing load. Some programs are able to suspend stations without hear-
ings, but many programs are bound by administrative procedure statutes to
follow a specified legal protocol before a station can be removed from
the system.
With limited resources, covert audit programs must avoid resource
intensive, time consuming actions whenever possible. Hearings and even
criminal action will occasionally be required, but they should be kept
to a minimum. To the extent that enforcement and, for that matter, re-
training can be accomplished by letters and office conferences, resources
will be saved for other activities.
However, any plan to minimize case or hearing loads need to be re-
viewed carefully prior to implementation. On the basis of observations
and experience of existing programs, swift and meaningful penalties are
necessary. Fines may be useful if they are large enough to deter program
abuses but small enough to ensure that they are administered and paid
routinely.
Publicity
Part of deterrence for station personnel is the knowledge that they
could be inspecting a covert vehicle at any time. Stations have contacts
with three main groups and organizations as a result of an I/M program.
o implementing agency personnel
o other stations and their trade associations
o the public
A review of experience with current auditing programs indicates that
having each of these groups involved is helpful. The implementing agency
can stress during training, certification, and formal audits that covert
auditors are actively checking stations. Trade associations can be enlis-
ted to work for the credibility and reputation of the program and their
members. The public may be educated by the mass media on specific abuses
that they can look for during inspections and repairs.
5-6
-------
TWO MODEL PROGRAMS
Table 5-2 presents an outline of two model programs. The first,
Alternative 1, is for a program which is just starting or modifying its
covert auditing program. It assumes very limited resources, and is
oriented towards integrating covert auditing into an existing auditing
program. The second, Alternative 2, is for a more mature auditing pro-
gram, and may be used as a model either by an Alternative 1 program, or
by other auditing programs with a degree of existing covert experience.
These two alternative programs are directed towards attaining the basic
goals of a covert auditing program in a cost effective manner.
5-7
-------
TABLE 5.2
TWO MODEL PROGRAMS
Program Element ALTERNATIVE 1
ALTERNATIVE 2
Prioritization
of Stations
Work off of complaints and
the highest passing rates.
Develop a computerized
data system which allows
screening of unusual
data.
Maintaining
Auditors'
Cover
Start with formal auditors
working adjacent territo-
ries; experiment with dri-
vers outside the program.
Analyze situations which
may expose an audit; mi-
nimize visibility of par-
ticular vehicles and
auditors.
Passing/Failing
Cars
See comments below; use
both passing and failing
pre-1981 vehicles to start;
if only one vehicle may be
procured, use a failing
pre-1981 vehicle.
Develop audit vehicles
which are directed to-
wards detrimental pro-
gram practices found to
be prevalent in the pro-
gram; experiment with
1981 and later failures.
Personnel
Team
Organization
Use existing formal audi-
tors and start recruiting
outside people as possible.
Start with splitting the
time of formal auditors
between covert audits and
formal audits.
Use a mix of employees and
non-employees; determine an
optimum balance based on
costs, maintaining auditors'
cover, and flexibilty in
planning audits.
Maximize the number of peo-
ple which can be called on
to be auditors (office per-
sonnel, formal auditors,
etc.)
Kinds of
Vehicles
Emphasize pre-1981 vehicles
of a common make.
On the basis of data collec-
tion about program operations,
emphasize vehicles which are
involved in program discre-
pancies; use some 1981 and
later vehicles.
Vehicle
Procurement
Obtain at least one program
vehicle but plan for tran-
sition to a different vehi-
cle.
Prepare a list of all possi-
ble vehicle sources (inclu-
ding private vehicle owners);
try a variety of approaches.
5-8
-------
TABLE 5.2—Continued
Program Element
Alternative 1
Alternative 2
Vehicle Set-up
Minimize vehicle set up,
except to make sure that
failing vehicles really
fail.
Set up at least one 1981 and
later vehicle to fail; other
vehicles should be set up to
test for particular program
practices.
Audit Documenta-
tion/Verifica-
tion
Document only that informa-
tion necessary to support
the enforcement actions
con temp1a ted.
Document all data relevant
both to potential enforcement
actions and to the program
practices investigated.
Shop Facilites
Use any enclosed space with
basic tools and an emission
analyzer.
Develop a shop which supports
the set-ups chosen; equip the
shop so that 1981 and later
vehicles may be adjusted or
modified.
Enforcement
System
Utilize office conferences
as much as possible, rather
than extensive hearings and
prosecutions.
Depending on available legal
authority, experiment with
different approaches to de-
termine what methods provide
the most deterrence.
Publicity
Use any means possible (es-
pecially training and for-
mal audits) to remind sta-
tions that covert auditors
are in the field. Do not
divulge under any circum-
stances the size of the
operation.
Along with the suggestions
of Alternative 1, identify
and publicize areas in
which vehicle owners can
play a part.
Data Collection
and Program
Assessment
At the beginning of a pro-
gram, start the development
of baseline data. Send out
at least some failing vehi-
cles in audits which will
not lead to enforcement
actions to determine if
false passes are a problem.
Develop baseline data and
then track the progress of
the program.
5-9
-------
REFERENCES
Cabaniss, John M., Jr., "Overview of I/M Activities and Issues,"
Proceedings - Mobile Source Issues in the 1980s, SP-5 2,
Midwest Section, Air Pollution Control Association,
Pittsburgh, PA 1984.
Carhart, Bruce S., "Real World Approaches to inspection/Maintenance
Quality Assurance: Field Auditing Issues," Paper No. 85-40.5,
presented to the 78th Annual Meeting of the Air Pollution
Control Association, Detroit, Michigan, 1985.
Pienta, Walter J., et al, "Quality Assurance in Decentralized I/M
Networks: Analyzer Measurement Error and Field Performance,"
Paper No. 852110, Society of Automotive Engineers, Warrendale,
PA, 1985.
Schroeer, John, "Covert Vehicle Modification" (revised), Colorado
Department of Health, Denver, CO, November 1985.
U.S. General Accounting Office, "Vehicle Bnissions Inspection and
Maintenance Program is Behind Schedule," GAO/RCED-85-22,
Washington, DC, 1985.
U.S. Environmental Protection Agency, "A Review of Control Strategies
for In-Use Vehicles," EPA-460/3-74-021, Office of Mobile Source
Air Pollution Control, Ann Arbor, Michigan, 1974.
U.S. Environmental Protection Agency, "Information Document on Auto-
mobile Emissions Inspection and Maintenance Programs," EPA-400/2-
78-001, Office of Air and Waste Management, Washington, DC, 1978a.
U.S. Environmental Protection Agency, "Motor Vehicle Emissions Inspection/
Maintenance Information Kit," EPA-460/3-78-013, Office of Air, Noise,
and Radiation, Washington, DC, 1978b.
U.S. Environmental Protection Agency, "Guidance on Quality Control for
Inspection/Maintenance Programs," EPA 460/3-82-006, Office of Mobile
Source Air Pollution Control, Ann Arbor, MI, 1982.
Wallauch, John R., "The California Approach to Vehicle Inspection/Main-
tenance," Paper No. 831213, Society of Automotive Engineers, Warren-
dale, PA, 1983.
R-1
-------
APPENDIX A
SHOP GUIDANCE
Facilities
A beginning shop should be large enough to house at least one vehicle
so that all work may be done inside. In addition, the shop should be e-
quipped with mechanics' tools (see below), an emission analyzer, supplies,
a roll-around tool box, a workbench and reference material. Space for an
office and a storage area is also desirable.
The shop should ideally have two bays, but a one-bay shop can still
be sufficient. A car hoist is useful, but if there is no hoist, the shop
should be equipped with a floor jack and car stands. An air compressor
can be located either outside or inside the shop area.
Tools
See Table A-1 and Figure A-1 for suggested tools for a basic shop.
A-1
-------
TABLE A—1
Basic Tools
SUGGESTED TOOLS FOR A BASIC SHOP
Standard
Metric:
Standard
Metric:
open end; box end; hand wrenches 3/8" - 1"
open end; box end; hand wrenches 9 mm - 23 mm
closed sockets 1/4" - 3/8" - 1/2" drive
closed sockets 1/4" - 3/8" - 1/2" drive
socket drivers 1/4" - 3/8" - 1/2" drive
Standard tip screw drivers
philips tip screw drivers
Hammers
Files
Hack saw
Air operated socket drivers
3/8" hand held
1/2" impact driver
Special Tools
For Pre-1981 Vehicles:
Timing light with advance control
Tach and Dwell Meter
Vacuum-Pressure Tester
Vaccum Pump
Volt Meter
OHM Meter
Distributor Wrenches
Exhaust Gas Analyzer
Engine Analyzer (scope) (all of the above can be
incorporated in enginer analyzer)
For 1981 and later vehicles:
All of the above plus -
Computer Scanner
Digital OHM Meter
Needle type OHM Meter
A-2
-------
ES ENGINEERING-SCIENCE
Figure a.l Sample Tools for an Audit Set-Up Shop
gm computer scanner and volt/ohm tester
Hand-operated vacuum pump, vacuum-pressure tester, various GM computerized
carburetor adjusting tools, and a distributor hold-down wrench
A-3
-------
APPENDIX B
BACKGROUND MATERIAL FROM
ONGOING I/M PROGRAMS
1. Covert vehicle investigation report forms
2. Covert vehicle investigation procedures
3. Auditor position descriptions
4. Background information of station performance
report (Figure 2-3, California Bureau of Auto-
mative Repair)
-------
APPENDIX B-1
COVERT VEHICLE INVESTIGATION REPORT FORMS
(COLORADO AND NEW YORK)
Note - New York and Colorado also require a narrative
along with these forms where appropriate.
-------
INVE ST IGATOR/A3£
DR 1362 (3/84)
42-16-410
COVERT VEHICLE REPORT
INSPECTION OBSERVED
YES ~ NO ~
DATE
TIME
AIR PROGRAM REPORT NO.
STICKER NO.
MECHANIC NO.
STAT 1 ON
SIGN POSTED
YES
n
NO
~
EM 1SS1ONS ST AWARDS V1S1BLE
YES
~
NO
n
IN CUSTOMER AREA O
SHOP
~
HOW WAS
VEHICLE YEAR VERIFIED REGISTRATION
~
VIN
n
EMISSION DECAL
CD
OTHER
STAT 1ON
STARTED INSPECTION FORM
YES
~
NO
n
CHECKED
EMISSIONS LABEL FOR DEVICES
YES
~
NO
~
N/A
n
1NSPECT
CATALYTIC CONVERTER
YES
n
NO
~
N/A
~
1NSPECT
AIR SYSTEM
YES
~
NO
n
N/A
~
INSPECT
FUEL RESTRICTOR
YES
n
NO
~
N/A
n
WHAT VISUAL ITEM MUST BE REPA1 RED/REPLACED
CATALYST
~
AIR ~ HESTR ICTOR
n
MAXIMUM
TIME ALLOCATED BY STATION FOR
REPAI
IR
60
DAYS
~
OTHER ~
HOW WAS
ENGINE TEMPERATURE VERIFIED
GUAGE
n
RADIATOR
~ CUSTOMER
~
ZERO & ELECTRICALLY SPAN ANALYZER
YES
~
NO
~
PROBE INSERT ION/ADAPTER USAGE
YES
~
NO
~
ACCELERATE ENGINE TO DEACTIVATE OVERRIDE DEVICES (1980 & OLDER) YES ~ NO (ZD N/A I I
¦1ETER READINGS 1ST HC CO
2 5 00 HC CO
2ND HC CO
WAS TWO SPEED TEST PERFORMED Y./TACH. YES ~ NO ~ N/A ~
WAS DRIVER INFORMED OF FIRST EM ISSIONS TEST FAILURE YES ~ NO ~ N/A CD
VERBALLY INFORMED OF 2 07(B) WARRANTY STATUS YES ~ NO ~ N/A ~
HOW WERE ENGINE SPECIFICATIONS DETERMINED BOCK ~ DECAL ~ OTHER I I
TACHOMETER HOOKED UP YES ~ NO ~ N/A ~
PROPANE ENRICHMENT USED YES [~ NO ~ N/A ~
CARBURETOR ADJUSTMENT METHOO LO CO ~ LEAN BEST ~ PROPANE ENRICHMENT ~ OTHER d
VFFIXED EMISSIONS CERTIFICATE ADJUST ~ PASS ~ STICKER CERTIFICATE NO
YES ~ NO ~ N/A ~
AFFIXED TEMPORARY CERTIFICATE
CORRECT STANDARDS USED
YES ~ NO ~
COMMENTS/RECCMMENDATIONS
-------
MOTOR VEHICLE DIVISION
"AIR" PROGRAM
COVERT VEHICLE EMISSIONS CERTIFICATION
Date: Time Investigator
Vehicle Make: Year License VIN Number
VISUAL INSPECTION
Air Injection System Catalytic Converter Fuel Restrictor
EMISSIONS STANDARDS
Vehicle CO Standards Vehicle HC Standards
% ppm
EMISSIONS READINGS
CO % Idle HC PPM
2500
I CERTIFY THAT I HAVE INSPECTED THE ABOVE-DE SCR I BED VEHICLE IN ACCORDANCE
WITH THE RULES AND REGULATIONS GOVERNING COLORADO AUTOMOBILE INSPECTION AND
READJUSTMENT PROGRAM. THE ABOVE-LISTED EMISSIONS READINGS ARE CORRECT FOR
THE VEHICLE, OR THE VEHICLE FAILS THE VISUAL INSPECTION OF EMISSIONS
EQUIPMENT AS NOTED, AT THE TIME AND DATE LISTED.
S i gnature Title Date
Subscribed and sworn to before me this day of ,19
My Commission Expires:
Notary Public
-------
State of New York - Department of Motor Vehicles case No
INVESTIGATION REPORT
Facility No
Q CONCEALED IDENTITY Q COMPLAINT csr No (r o us«oni")
.jse Print or Type All Entries and Use Black Ink
Name of Complainant
1
Name of Facility
] it ess - Number and Street
D/b/a (if applicable)
State
Zip Code
Address - Number and Street
i uiephone No (including area code)
City State Zip Code
icle Year, Make, Model
Cyl
Telephone No (including area code)
Facility Expiration Date
V 1 N
Plate No
Name and Ident No of Certified Inspector
e of Transaction
Odometer Reading at Tlr
Transaction
ne of
Oate of C 1 /Complaint
Investigation
Odometer Reading at Time of
Investigation
rsuant to N. Y Vehicle and Traffic Law, Section: (Central Office Use Only)
Recommendation
Q Wo Action
On 0 I
~ Susp Pend Hrg
~ Hearing
~ Inspection Waiver
~ Other
Insp
Cert
Rep
-------
APPENDIX B-2
COVERT VEHICLE INVESTIGATION PROCEDURES
(CALIFORNIA AND NEW YORK)
-------
California
Docunented Vehicle Procedures - Outline
1. P.eccive job order from Tistrict Office.
?. Review job order and assign to shop teclmician.
3. Perform documentation to vehicle as per job order.
4. Run I/M test on T.A.S. and obtain 2 printouts.
5. Hiter documentation performed and 1 TAS printout into
documentation log book.
G. Complete a "BEFORE" VEHICLE DATA sheet on the vehicle.
7. Staple remainirig T.A.S. printout to back of job order.
8. Release vehicle to Field Representative.
a. See Attachment, Program Representative's Duties
9. Enter release information into mileage log (Date, Time, Mileage)
10. Receive (returned) vehicle, test report, invoice, and CofC (if
issued).
11. Enter the information into mileage log. (Date, Time, Mileage
and ro's signature)
12. Reinspect and retest vehicle on T.A.S. to document its as returned
condition and obtain 2 printouts.
13. Complete a "AFTER" VEHICLE DATA sheet on the vehicle.
14. Finish completing the as returned information on the job order.
15. Write Declaration if requested by Field Representative.
16. Complete a Cost Sheet if requested by Field Representative.
17. Complete Documented Run statistics.
-------
Documented Vehicle File
Assemble a file for Field Representative;
0 = Original Itx-ument C -- Copy of Document
Contents in file:
Repair Facility Invoice 0
'¦'epair Facility Test Report 0
Certificate of Compliance £if issue*!) 0
Job Order C
AFTER VTJ1ICLE DATA Sheet 0
E.A.R. As Returned Printout 0
REFORE VEHICLE DATA Sheet 0
Assembly a file for shqp records
Contents in file:
Job Order 0
B.A.R. "BEFORE" Printout 0
BEFORE VEHICLE DATA Sheet C
Repair Facility Invoice C
B.A.R. As Returned Printout 0
AFTER VEHICLE DATA Sheet C
-------
BUR1AU OF AUTOMOTIVI UPAIR
DOCUMENTED VEHICLE JOB ORDER
VdHltLg
YEAR
DATE IN
19
MILEAGE IN
MILEAGE OUT
nATP miT
10
SOURCE
~ Randon ~ Comph.nl
~ Follow-up ~ Cert Sale
~ Referee ~ q a
D T,P ~ Other
results
~ Negative
~ Positive
JOB ORDER
I I I I
— vpiiyiwn vrav nvogmg wirn inm \jp9fOVlOfl (.odl MfiflftlAg
in tfi« third column.
J. I* nor* ipoca » iiM
-------
OPERATION CODE
FRONT
END
BRAKES
I STEERING
BOX
2 tie.roos
AND ENOS
3 KINO PINS OR
BALL JOINTS
5 SERVICE
BRAKE
6 WHEEL AND
MASTER CU,
7 drums/discs
49
PUMP OR
CYLINDER
50
WHEELS OR
ALIGNMENT
51
52
ORAO UN*
PITMAN ARM
53
EMERGENCY
BRAKE SYSTEM
54
AIR
COMPRESSOR
53 Calipers
07
96 WHEEL BRGS-
SHAFTS-SEALS
99 SPRINGS AND
SHACKLES
100
101 HOSES VALVES
AND LINES
102 PARKING
BRAKE
103 "(TURN
SPRIMOS
REMARKS
8 WATER
PUMP
56
AIR
CONDITIONER
104
HOSES
COOLING
9 RADIATOR
OR SHROUD
57
HEATER
10S
THERMOSTAT
10
RAOIATOR Cft*
58
HEATER HOSt
106
1 1 VALVES. ROCK-
ER8 OR HEADS
59
SHORT BLOCK
107
CONTROLS
ENGINE
12 PISTONS
OR RINGS
60
CASKET
LEAK
108
|3 CRANKCA3C
CONTROL
61
CAMSHAFT
OR LIFTERS
109 CRANKSHAFT
OR BEARING
14 HCATCR HOSt
62
MAIN / ROO
BEARING
110
15 CARBURETOR
OR MANIFOLD
63
TANK OR
LINES
111
BLOWER
FUEL
16 ruci.
PUMPS
64
ruCL
FILTERS
112 ruci
INJECTORS
17
KVAP. 9T9.
65
113 IOLC 6
TRANS. CON.
18 PLUGS OR
WIRES
66
DISTRIBUTOR
OR MAGNETO
114 INSTRUMENTS
OR SWITCHES
ELECTRICAL
IB
67
REGULATOR OR
ALTERNATOR
115 LIGHTS
OR HORN
20
68
116 WIRING OR
CONNECTORS
21 oia
69
117 POINTS ft
CONO.
CLUTCH
TRANS.
22
70
TRANSFER
CASE
118 TOROUE
CONVERTOR
23 AUTOMATIC
TRANSMISSION
71
DRIVE LINE
119
24 MANUAL
TRANSMISSION
72
OOvCRNOR
120 COOLINO
LINCS
EXHAUST
25 SPARKARREST-
OR OR MUFFLER
73
MANIFOLD
GASKETS
121
26 EMISSION
CONTROL
74
hcaocr/tail
PIPES
122.IXMAUST STST
BRACKETS
27-
AILE SHAFTS
75
BEARING
OR SEALS
123 WHEEL,
CYLINOER
REAR
AXLE
28
76
DIFFERENTIAL
124 ALIGN OR
WHEELS
29 SPRINGS
OR SHACKLES
77
TIC-RODS
AND ENOS
125 PINS OR
JOINTS
SO tires
78
TORSION BARS
126
31
CAB
BODY
FRAME
DOOR
79
GLASS OR
REGULATORS
127
32
MIRRORS
80
W S WIPERS
128
UPHOLSTERY
33 HITCH
OR JACK
81
129
FRAME
34 RESERVOIR
SPOT OR HOPPER
82
LADDER
OR STEP
130
35
83
131
36
HYDRAULIC
PUMPS
84
132
37
85
HOSES
OR LINES
38
86
CYLINDERS
133 FILTERS OR
RESERVOIR
134
Cylinder
Wall Taper
l)
TT
2)
6)-
})
7)
4)
8)
Crankshaft
Rod Throws
2)
J)
4)
6)
7)
8)
Crankshaft
Mains
i)
2)
3)
4)
J)
6)
7)
8)
Drum/Disc.
Measurement
FRONT
REAR
DETAIL TASK CODE
1 MARK &
ADJUST
MEASURE
2 MARK &
REPAIR
INSTALL
3 MARK &
REPLACE
TORQUE
Form 771-24 (11-74)
-------
NEW YORK STATE
DEPARTMFNT OF MOTOR VEHICLES
DIVISION OF VEHICLE SAFETY SERVICES
REGIONAL OFFICE PROCEDURE FOH CONCEALED IDENTITY INSPECTION OF
INSPECTION STATIONS
Material Received;
1. Written or oral instruction from Sr. Automotive Facilities
Inspector to perform a concealed identity inspection of a specified
inspection station,
OR
2. For downstate regional offices, printout of inspection stations
from the Technical Services Bureau. Printout will indicate
information such as:
a. excessive inspections conducted daily and monthly;
h. excessive passes;
c. excessive failures*
d. excessive adjustments or repairs on certain items, etc.
Operations:
1. Preparation for inspection.
a. Obtain and sign for cash from CI fund to cover the cost of the
inspection and any minor repairs that nay be necessary.
b. Obtain a vehicle, registration plates, integrated windshield
stickers, and inspection sticker from Regional Office to use
during the inspection.
c. Check vehicle to be used.
1. Registration must correspond with plates.
2. Inspection sticker should indicate expiration or
near-expiration.
3. Re sure that the vehicle is in good repair".
-------
d. Wear appropriate attire (work, hunting, or fishing clothes) to
assume the identity of an ordinary motorist.
e. If possible, carry lunch box, local newspaper, packages,
fishing rods, tools, etc, in vehicle to strengthen impression
that you are an ordinary motorist.
f. Tune radio to local station.
g. When necessary, telephone inspection station in an attempt to
have the vehicle inspected. This will ensure that you do not
waste too much time waiting around,
h. visit the inspection station in an attempt to have the vehicle
inspected. If the inspection station says they are busy, ask
for an appointment.
2. Inspection
NOTE: When presenting the vehicle for inspection, be careful not
^to say anyting that might be considered entrapment. A
'single statement such as "I need a state inspection", or
"Can I please get my car inspected" is all that is
necessary.
a. Keep the vehicle under observation at all times.
Some suggestions:
1. Strike up conversation with the inspector, express curiosity
about the inspection, etc.
2. If the inspector asks, participate in the inspection by
applying the brakes, operating lights and directional signals
3. Ask to check exhaust system for leaks, etc. when the vehicle
is jacked up, or tell the inspector you're getting a rattle fror
the tail pipe.
b. Do not, under any circumstances, attempt to hurry the in9pectior.
c. If the inspector advises that adjustments or repairs are
necessary, use the following guide in making a decision on
whether to have them made at the station.
1. If the concealed identity inspection was conducted because
a routine inspection revealed many adjustments on a certain
item, or as a result of complaints concerning unnecessary
repairs, adjustments can be expected. In these cases the
repairs or adjustments should be made.
2. If the inspection was completed in a satisfactory manner,
adjustments should not be made (except in cases of
unnecessary repairs, as indicated in item 1 above).
3. If repairs are recommended by'the station,-, do not authorize
them, except in situations where the Regional Director has
-------
given pre-approval, In all other case9, request a written
estimate on the station's invoice.
d. After paying for services, obtain a receipt for the inspection
, and any repairs made. If no invoice is given, attach a signed
statement to the report indicating the amount paid.
NOTE; . If the AFI is told to wait in an area where he
will be unable to properly observe the inspection,
he should wait until the vehicle is brought in and
then identify himself and request that the inspection
be performed in his presence. This eliminates the
waste of time and money associated with an unobserved
inspection.
Post-Inspection
a. Evaluate inspection and complete Inspection Checklist
(Form VS-83.1) as soon as possible.
b. Rempve inspection sticker from vehicle aod cover adhesive
side with a transparent material.
c. If it is the policy of the Regional Office to return
to the station shortly after the CI has been completed;
• Show identification and review inspection with inspector
and/or owner.
• For safety inspections only, obtain VS-1G74 as proof of
CI and give station a receipt for it.
• For a safety/emissions inspection only, if no
analyzer receipt was obtained after the inspection,
inform your Senior AFI who will assign an AFI to
perform a program audit and obtain the receipt.
• Perform program audit on records and equipment.
d. Always complete the Investigation Report (Form VS-35.1).
Attach your written report, removed inspection sticker,
invoice, and any other pertinent information to the VS-35.1.
e. If an improper inspection hs been completed, complete CI
Inspection Notification Letter (Form VS-89) in triplicate.
Mail one copy to the station, one copy to the certified
inspector, and attache one copy to your completed report.
Disposition of Materialst
Forward completed report and attachments to Sr. AFI.
Forward copy of VS-35.1 and original receipt or statement of
payment to person responsible for replacement of QJ. funds.
-------
APPENDIX B-3
AUDITOR POSITION DESCRIPTION
-------
COLORADO DEPARTMENT OF PERSONNEL
A7180A
A7182A
MOTOR VEHICLE EMISSIONS COMPLIANCE OFFICER
NATURE OF WORK
This 1s a multiple-range class describing the responsibility for conducting
field inspections, field compliance and quality review of licensed automobile
emissions stations and mechanics.
An employee in this class uses the required techniques to license emission
inspection stations, instruct emission mechanics and emission station
owners/managers 1n the proper emission inspection procedures for compliance
with the Automobile Inspection and Readjustment air statutes, rules and-
regulations.
D1stinquishinq Factors: Positions in the next higher level class are
distinguished from positions in this class by having supervisory responsi-
bilities over employees 1n this class.
Range A
This range is characterized by the training/developmental nature of assign-
ments which utilize a wide range of technical skills and abilities and also
requires the application of specific knowledges of the statutes, rules and
regulations for the Automobile Emissions Program. Work includes considerable
public contact wherein judgment is exercised in executing departmental
policies. Initially, work is performed under close supervision, but as
training progresses, work is performed under general supervision.
Range 8
This is the full operating Emissions Compliance Officer level. At this level,
employees are expected to perform the full range of compliance, instructing
and enforcement duties for the Automobile Emissions Program. Employees''
work is subject to only occasional review for technical accuracy, adequacy
of technical judgment and results. Employees receive instructions in very
general terms; detailed instructions in work process or procedure are
rarely needed, except to deal with the most unusual or sensitive problems.
SOME EXAMPLES OF WORK
Performs compliance inspections of licensed motor vehicle emissions stations
and mechanics on a monthly basis. This requires the demonstrated technical
skills in the evaluation and calibration of sophisticated electronic testinn
equipment. An example is performing a gas span test on the Emission
Analyzer every 30 days.
Conducts field technical assistance and training as necessary to update
licensed emission mechanics and station owners/managers on changes in rules,
regulations and procedures.
-------
MOTOR VEHICLE EMISSIONS COMPLIANCE OFFICER (Cont'd)
Page 2
Examines station records to insure report forms are accounted for and properlj
filed; examines New Car Logs at dealerships for proper issuance on Emissions
Certificates and their accountability.
Does necessary follow-up due to supervision of emission mechanic/station
licenses or reinstatements.
Resolves consumer complaints concerning emissions related issues.
Prepares monthly contact forms for stations contacted; prepares investigative
reports on violation of the "AIR"~program.
Periodically conducts covert investigations on licensed emissions stations
and mechanics.
Attends and testifies 1n the Informal hearings conducted by a Motor Vehicle
Emissions Compliance Supervisor and/or the formal hearings held in the Motor
Vehicle Hearings Section.
Performs related work as assigned or required.
KNOWLEDGES, SKILLS AND ABILITIES
Thorough knowledge of basic tune up procedures, and ability to use tune up
equipment.
Considerable knowledge of "AIR" program statues, rules and regulations.
Considerable knowledge of relevant state and department rules, regulations,
policies and procedures.
Considerable knowledge of the functions involved in the control of harmful
pollutants from internal combustion engines.
Considerable knowledge of ignition systems, fuel systems., emission systems
and internal combustion engines.
Knowledge of the principles underlying the operation, use, calibration and
maintenance of scientific electronic instruments used In the testing of
motor vehicle emissions.
Knowledge of the purpose, function and effect of emission related parts of
an automobile.
Ability to perform all parts of an emission inspection and test.
Ability to use a gas span, use and calibrate an infrared analyzer.
Ability to read, comprehend, interpret and translate highly technical
emission data.
-------
MOTOR VEHICLE EMISSIONS COMPLIANCE OFFICER (Cont'd)
Page 3
Ability to interpret and apply laws and regulations uniformly.
Ability to maintain effective working relationships with other employees,
emissions station/dealer personnel and mechanics.
Ability to communicate effectively, both orally and in writing.
MINIMUM PREPARATION FOR WORK
Education and Experience
Three years of progressively responsible experience 1n automotive mechanics
wnich Included motor vehicle tune ups and repair and diagnosis of emissions
systems.
Substitution
Technical work in a motor vehicle emissions program will substitute for the
experience requirement on a year-for-year basis.
Full time teaching courses in tune up, emissions system diagnosis and repair
will substitute for the experience requirement on a year-for-year basis.
An Associate Degree from an accredited college with major coursework in auto
mechanics with the emphasis in tune up," emissions system diagnosis and repair
will substitute for two years of the required experience.
College, university or non-correspondence technical school course work 1n
auto mechanics Including tune ups, emissions system diagnosis and repair
will substitute on a year-for-year basis for two years of the required
experience.
Necessary Special Requirement
Possession of a valid State of Colorado driver's license. A valid Air Program
Mechanic Certificate Issued by the Health Department, State of Colorado must
be obtained prior to and/or during the probationary period.
NOTE
Positions are to be assigned to one of the ranges .in this class based on
agency evaluation of the employee functioning level relative to the range"
descriptions identified in the class specification.
Proposed Published Date; 3/4/85
(LDS)
-------
APPENDIX B-4
BACKGROUND INFORMATION OF STATION
PERFORMANCE REPORT
(FIGURE 2.3, CALIFORNIA BUREAU OF AUTOMOTIVE REPAIR)
-------
INSTRUCTIONS TOR THE STATION - MECHANIC EVALUATION RD'ORT
The Station / Medianic Evaluation rtrport has developed to assist
the field offices m iTDruLonng and enforcjriq the ro'-juirojnents of the
C-ilifornia 5'Xkj Check Program. By collet inq and analyzing tho TAS datn t-hn
retxx't can t>e a valuable tool to provide a clearer picture of how a particular
station or rrechanic is performing in the fi.ild.
The various areas of information oa rhe report will be explained as
to their purpose and the rrethods used to arrwe at the results.
The report is detailed by station number and mechanic number.
Group 1:
Column:
1. Total Tests
Lists the number of I (initial) or B (before repair) tests perforned.
2. Prcnt Fail
Indicates the percentage of these I or B tests that failed.
3. % Dev from District Average Fail Rate
The percentage above or belcw the District average fail rate for I tests.
4. % Dev from District Average HC
The purpose of this field is to reveal chronic problems due to analyzers
with excessive drift or air bleeds during testing.
The district average HC for each standards category are determined and a
baseline is established. The average HC emission for each station and
mechanic is determined for each standards category. These averages are
divided by the district averages for each category that the sta/mec
performed tests in. This factor is sunned and then divided by the number
of categories in which tests were performed. The result is multiplied by
100 to get the percentage deviation.
5. % Dev from District Average CO
Same as number 4 for CO.
6. Total Test3
Lists the number of A (after repair) tests performed.
7. Prcnt Fail
Indicates the percentage of these A tests that failed.
8. % Dev from District Average Fail Rate (A Tests)
The percentage above or below the District average fail rate for A tests.
9. % Dev frnm District Average HC Reduction
Similar to number 4 above except this calculates the average emission
reduction for HC. 'Hie first I test and the last A test for all vehicles
with rcultiple tests are used to determine the HC reduction. The figure
listed on the report is listed as a positive or negative (-) .-percent
deviation from the district average HC reduction.
10. % Dev fr"m District Average CO Reduction
Sam; as number 9 for CO.
-I-
-------
Group 2:
Column:
L. # Certs by Innacurate Pre-/ Emis Tst Resits
Indicates the number of certificates issued where tho A test emission
results do not indicate a reduction of emissions when canpared with the
I or B test. This is calculated by comparing die actual omission readings
of the first test for a vehicle, to the last test errussL^n readings Cor
the same vehicle. Tf they do not shew a reduction and a certificate was
Lssued the count is increirented by 0:10.
2. % Dev from Statewide Dilution Average
This field is used to determine if a irecliamc is diluting the exhaust sample
by partially removing the probe from the exhaust pipe or inducing leaks
elsewhere. A baseline dilution average for the district is determined by
averaging the sum of the 00 and 002 readings. The station and mechanic
dilution average is also calculated in the same manner.
The sta / mec dilution average is divided by the district average and a
percent deviation is calculated.
3. Idle Speed Deviation I to A Test (% pos) (% dev)
The purpose is to see if mechanics are increasing idle speeds close to
thresholds instead of factory specifications to gain emissions reductions.
The idle speed of the first test for a vehicle is ccnpared to the idle speed
of the next sequential test for the same vehicle. The percent they deviate
frcm each other is calculated. The first figure in this coluirn lists the
percent of tests that had an increase in idle speed. The second figure shows
the percent those tests deviated frcm the first test.
4. Number of Retests Ad j 00 to Threshold
The purpose is to indicate if mechanics are adjusting carburetor mixture
close to the CO threshold instead of factory specifications.
The first and next sequential test for a vehicle are corpared to each other.
If the vehicle failed for 00 emissions, the second test is checked to see
how close the idle 00 adjustment brought the CO emission result to the 00
threshold. If the 00 emission was brought to within .5 of the CD threshold
a counter is increased by one. The number on the report is the total of
that counter.
5. Percent Tests W/ln 113 PPM of Threshold
The idle speed is checked for each test to see if it acnes within 113 RPM
of the threshold for the specific emission standards category that the
vehicle falls into. The total number of tests that fit this pattern are
divided by the total number of tests and a percent is calculated and listed
in this column.
6. Number A Tests With No Matching I Tests
This is to see if mechanics are performing A tests without an I or B test
This ccxild indicate the previous emission readings may be ficticous.
Each A test is checked to see if it lvis a correspondinq I or B test. If m
matching test can found a count ;r is lnrr^nonted. The figmv in tin .
column represents the nu,iilx;r of tn*'S this wii the ca-ie.
-2-
-------
Group 3:
Column:
L. Percent Underhood Accuracy
The purpose of this field is to conpare the underhood inspection information
specified in the (^mission control systems application manual to the actual
underhood data entered ty the mechanic. The vehicle make, year, number of
cylinders and engine size indicated on the test are ccrnpared to a table for
the corresponding vehicle make, year, number of cylinders and engine size.
If the table and the actual underhood daca differ the mechanic will be
given sn incorrect response. The number of incorrect responses are divided
by the total number of vehicles tested and a percentage of accuracy is
listed.
2. Erroneous Entries on Functional Check
This is used to see if the mechanics are performing the functional chocks
in districts where the functional check is rot required or are not performing
the functional checks in districts where they are required to do so.
When this is the case a counter is added to and the total of that counter
is listed on the report.
3. No. and % With Past Repairs
This field is to indicate the percentage of time the A test begins in less
than 15 minutes following the oorpletion of the corresponding I or B test.
The first number in this colurm represents the number of times this occured.
The second number represents the percent of times this occured.
4. % A Test With No Repairs
This is used to see if mechanics are performing A tests without indicating
the repair action taken.
The A tests are checked in the repair action field. If no action //as
indicated a ccunter is incremented. The total of this counter is divided
ty the total number of A tests and that percentage is listed.
5. % Multiple Tests
This is used to 6ee if nultiple tests of the same type for the same vehicle
are being done. Multiple tests for a vehicle are checked to see if the
test type ( I or A) is the same as the previous type. Where nultiple test of
the same type for the same vehicle are done the percent is calculated and
listed in the appropriate colurm.
6. Number of Failed Tests With Certs
The purpose of this field is to determine if the mechanics are abusing
the emission reducton and exceeding cost limit to issue a certificate.
The number of times a certificate is issued due to a vehicle achieving
a reduction of emission and an exceed the cost amount is added to a counter.
This number can be converted to a percent and caipared with the district
average.
-3-
------- |