United States Solid Waste And EPA 510-S-92-801
Environmental Protection Emergency Response May 1988
• Agency 5403W
vxEPA Development of Procedures
To Assess The Performance
Of External Leak Detection
Devices
Executive Summary - Draft
) Printed on Recycled Paper
-------
-------
FINAL REPORT DUE MAY, 1988
DCN 88-231-121-05
DEVELOPMENT OF PROCEDURES
TO ASSESS THE PERFORMANCE OF
EXTERNAL LEAK DETECTION DEVICES
EXECUTIVE SUMMARY
DRAFT
EPA Contract No. 68-03-3409
Work Assignment No. 2
EPA EMSL Technical Project Monitor: J. R. Worlund
Prepared for:
CDM Federal Programs Corporation
13135 Lee Jackson Memorial Highway, Suite 200
Fairfax, Virgina 22030
Deputy Project Officer: Dr. M. S. Rosenberg
Prepared by:
Radian Corporation
P. 0. Box 201088
Austin, Texas 78720-1088
Program Manager: T. R. Blair
Project Director: A. G. Eklund
Principal Investigators: R. D. Achord
M. R. Kienbusch
D. B. Burrows
G. P. Behrens
W. L. Crow
B. M. Eklund
-------
-------
CONTENTS
1. Introduction. 1
Ob j ect ives • 2
Report Organization 3
2. Preliminary Summary Conclusions 4
3. Summary of Technical Approach 6
Vendor Survey & Performance Criteria 6
First Draft Performance Evaluation Test Procedures 6
Device Acquisition 12
Second Draft Performance Evaluation Test Procedures 13
Preliminary Leak Detection Performance Data 13
Validation of Performance Evaluation Test Procedures..... 13
Completion of Proposed Performance Evaluation Test
Procedures 14
4. Results 15
Results of Specificity Testing 16
Results of Accuracy and Response Time Testing. 21
References « 35
Appendix
A. Mathematical Formulas. 36
Glossary.
47
-------
-------
SECTION 1
INTRODUCTION
Underground storage systems comprised of tank, piping, and ancillary
components that contain hydrocarbons or other hazardous materials represent a
potential source of environmental contamination. Proper system design, in-
stallation, operation, and maintenance, along with a leak detection program,
can minimize the detrimental effects of leaking underground storage tanks and
piping. Devices capable of detecting petroleum hydrocarbons lost from under-
ground storage tank (UST) systems can be used inside an UST system, i.e.,
in-tank, or external to an UST system, i.e., out-of-tank. In-tank (internal)
UST leak detectors generally detect losses with liquid-level sensors. Out-
of-tank (external) UST detection systems measure the presence of liquid- or
vapor-phase hydrocarbons. Early detection of liquid- or vapor-phase petroleum
hydrocarbons allows leaking UST systems or components to be removed from
service and repaired or replaced, thereby
impairment and economic loss of product.
service and repaired or replaced, thereby minimizing both environmental
Historically, external (out-of-tank) petroleum hydrocarbon leak and
release detection devices have not been extensively used and, therefore, are '
used primarily in conjunction with new underground storage tank installations.
However, most external leak and release detection systems can be retrofitted
at existing facilities. Since existing UST facilities are potentially at
greater risk of failure (due to age), it is extremely important that leak and
release detection devices be used to detect releases of petroleum hydrocarbons
from UST installations.
There are numerous commercially available external leak and release
detecting devices (for petroleum hydrocarbons) designed exclusively for use
with UST systems; however, there are no established performance specifications
-------
or certification procedures for assessing the capabilities of these devices.
The U.S. Environmental Protection Agency (EPA) is currently promulgating
regulations and guidelines that address underground storage tanks. One aspect
of this program is to develop simple, benchmark test procedures that can be
used to assess the performance of external petroleum hydrocarbon leak and
release detection devices. Performance test results for external leak and
release detectors in actual field use may vary from performance measured by
these benchmark tests.
Radian Corporation, as a subcontractor to Camp Dresser McKee under con-
tract to EPA/EMSL, is providing technical support in developing test proce-
dures for performance evaluation of out-of-tank (external) leak or release
detection equipment (components) for petroleum hydrocarbons. Several tasks
critical to this program are complete or currently underway. Results of a
survey of vendors of external petroleum leak monitoring devices for use with
underground storage tanks have already been published (1). Standard test
procedures are under development to allow objective comparisons of external
leak and release detector performance. Draft performance methods designed to
be simple benchmarks for comparison of external leak and release detectors
under prescribed conditions have been submitted to EPA (2).
OBJECTIVES
The overall objective of Radian's UST work is to support EPA's regulatory
development. Radian's key contributions are to identify parameters that
affect leak detector performance and to develop standard test procedures for
assessing external petroleum hydrocarbon detector performance within the
meaningful ranges for each performance parameter. The objective of the cur-
rent task (Work Assignment 02) was to develop and execute draft standard test
procedures for performance evaluation of external detection systems for leak
and release monitoring. Procedures that address specificity, accuracy, and
response time for both vapor-phase and liquid-phase external detectors have
been submitted to EPA in a draft report (2).
-------
The performance evaluation test procedures submitted to EPA for review
were written as a laboratory test plan for Radian personnel involved in devel-
oping formal procedures. Each procedure will require validation before it is
used by the public. These performance evaluation procedures will also require
revision to provide further clarification, example calculations, and an
increased level of detail for performance by an operator. A final collection
of performance evaluation test procedures for external leak and release detec-
tion components will be submitted in a single document at a future date for
EPA review and validation.
REPORT ORGANIZATION
Preliminary conclusions and a summary of results from initial execution
of the performance -evaluation test procedures for a limited number of external
leak and release detection devices are given in Section 2. Section 3 provides
a description of the technical approach for this work assignment in the con-
text of prior work. Section A gives a preliminary discussion of the results
from initial testing of external leak and release detection equipment using
the test procedures for evaluating performance. References, mathematical
formulas used for vapor-phase and liquid-phase detector performance calcu-
lations (Appendix A), and a glossary are given at the end of this summary
report.
-------
SECTION 2
PRELIMINARY SUMMARY CONCLUSIONS
Test procedures used to date to assess the performance of external
petroleum hydrocarbon leak and release detection systems were developed to
provide a uniform, inexpensive, benchmark protocol for comparing parameters
such as specificity, accuracy, bias, precision, and detection time. Although
only 13 detectors have been tested by Radian, the procedures appear to be
applicable to a wide variety of external petroleum hydrocarbon detectors.
Categorization of external detectors provided adequate separation of operating
principles for ease of performance testing. Intermittent and continuous
detectors, separated into liquid-phase and vapor-phase categories, were easily
tested with the same procedures.
No major problems or obstacles were reported by Radian personnel per-
forming these initial test runs, and it is likely that these procedures could
be performed easily by manufacturers or technicians knowledgeable in UST
technology.
Preliminary test results of the performance evaluation procedures indi-
cate that external vapor-phase and liquid-phase detectors respond to most
petroleum hydrocarbon components; however, specific detection systems (espe-
cially vapor-phase sensors) may have better response factors for certain com-
pound classes (aromatics or aliphatics). These results also indicate that the
250 ppmv concentration used for specificity testing of vapor-phase detectors
may be too low for evaluation of some types of qualitative-output detectors.
All the liquid-phase detectors responded within 24 hours or less to a 0.5-inch
thickness of petroleum product layer floating on water.
-------
Liquid-phase accuracy, bias, precision, and response time results appear
to be independent of the composition of the hydrocarbon test liquid; there-
fore, the simulated standard gasoline mixture probably provides a good repre-
sentation of a commercial product while offering the advantages of a uniform,
laboratory-generated test liquid. In general, the liquid-phase detectors that
were tested responded in the presence of a hydrocarbon. A larger range of
accuracy was observed for the vapor-phase detectors that were tested.
Detection times for vapor-phase detectors ranged from less than 15
seconds to 24 minutes, with most of the responses occurring in less than 1
minute. Vapor-phase fall time values were all under 15 minutes, and most
values were under 1 minute. Detection times for liquid-phase detectors were
generally less than 2 hours and were less than 15 minutes for most detectors
when the hydrocarbon layer thickness was above the vendor's specifications for
lower detection limit.
Future testing of a wider array of external leak and release detection
devices may require modification to test containers and chambers to accommo-
date all detector probe configurations. New performance evaluation procedures
may be required to address the scientific principles of operation of future
detection equipment as technology presents new modes of external leak and
release monitoring.
For test procedures to progress past the draft stage, validation tests
will be required to evaluate each procedure's acceptability, and possibly, the
relationship of leak detector performance in the laboratory to leak detector
performance in the field.
-------
SECTION 3
SUMMARY OF TECHNICAL APPROACH
VENDOR SURVEY AND PERFORMANCE CRITERIA (WORK ASSIGNMENTS 44 AND 70)
The external petroleum hydrocarbon leak and release detection devices
that were identified in a prior study (1) were categorized by function into
four groups (i.e., intermittent and continuous devices for both vapor-phase
and liquid-phase petroleum hydrocarbon detection). Separate performance eval-
uation test procedures are being developed for vapor-phase and liquid-phase
petroleum hydrocarbon detection systems, but intermittent and continuous
detection systems of each type are being tested using identical procedures.
Parameters that describe petroleum hydrocarbon leak and release detection
device performance have been identified (3) and are being used to develop
performance evaluation test methods. Parameters related to long-term perfor-
mance (reliability), safety, and cost were not considered in the development
of the present test procedures.
FIRST DRAFT PERFORMANCE EVALUATION TEST PROCEDURES (WORK ASSIGNMENT 92)
In March 1987. a first draft of example performance evaluation procedures
was submitted to EPA for review (4). These procedures addressed: 1) speci-
ficity of vapor-phase leak and release detection equipment and 2) accuracy and
response time for liquid-phase detectors. These test procedures were based
upon information from the literature, vendor and manufacturer testing, and
engineering and scientific knowledge of the equipment to be tested.
Prior to writing the first draft, performance parameters were grouped
together, where possible, to save time and resources and to decrease the
-------
variability of reproduced test procedure performances. The performance param-
eters were combined in some cases, as shown in Table 1. so that initially a
total of 10 performance evaluation test procedures were to be written. Proce-
dures have been written in EPA's standard format. The generic performance
procedure outline is given in Table 2. The following paragraphs describe each
of the outlined sections' objectives.
The Scope and Application section of each procedure names and describes
the performance parameter(s) that the procedure was designed to measure, such
as accuracy, signal-to-noise ratio, or specificity. The applicability and
limitations of the procedure are discussed. Procedures are specific to either
vapor-phase or liquid-phase detectors but are usually applicable to both
intermittent and continuous external leak and release detectors. Technical
limitations of the method are given.
The Stunmary section provides a brief overview of operational concepts and
principles that are necessary for performing the procedure.
The Interferences section is a detailed guide on how to detect and mini-
mize potential interferences. Specific recommendations on how to measure the
effects of contamination are made in this section.
The Safety section provides information and guidelines that are needed to
perform a procedure safely. This section discusses minimum safety practices
that must be implemented to reduce the possibilities of an accident or
exposure.
The Apparatus and Materials section describes equipment, materials, and
specifications for each. Descriptions of test containers or chambers, gas
rotameters, manometers, and timers are included in this section. Materials
such as tubing and fittings are covered also.
-------
TABLE 1 PROPOSED PERFORMANCE EVALUATION TEST PROCEDURE MATRIX FOR
EXTERNAL LEAK AND RELEASE DETECTION EQUIPMENT
1) Test Procedure for Measuring Accuracy and Response Time for Liquid-Phase
Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors
2) Test Procedure for Measuring Accuracy and Response Time for Vapor-Phase
Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors
3) Test Procedure for Determining the Specificity of Liquid-Phase Out-of-
Tank Petroleum Hydrocarbon Leak and Release Detectors
4) Test Procedure for Determining the Specificity of Vapor-Phase Out-of-Tank
Petroleum Hydrocarbon Leak and Release Detectors
5) Test Procedure for Determining the Detection Limit and the Signal-to-
Noise Ratio of Liquid-Phase Out-of-Tank Petroleum Hydrocarbon Leak and
Release Detectors
6) Test Procedure for Determining the Detection Limit and the Signal-to-
Noise Ratio of Vapor-Phase Out-of-Tank Petroleum Hydrocarbon Leak and
Release Detectors
7) Test Procedure for Measuring Drift for Liquid-Phase Out-of-Tank Petroleum
Hydrocarbon Leak and Release Detectors
8) Test Procedure for Measuring Drift for Vapor-Phase Out-of-Tank Petroleum
Hydrocarbon Leak and Release Detectors
9) Test Procedure for Determining the Effect of Potential Interferences of
Liquid-Phase Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors
10) Test Procedure for Determining the Effect of Potential Interferences of
Vapor-Phase Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors
^Submitted to EPA in draft form for review in March 1988.
-------
TABLE 2. OUTLINE FOR PERFORMANCE EVALUATION TEST PROCEDURES FOR--EXTERNAL
LEAK AND RELEASE DETECTION EQUIPMENT
1. Scope and Application
Method Covers (e.g.. detection limit)
Application (e.g., continuous gas-phase detectors)
Limitations
Experienced Personnel to Employ Method
2. Summary of Method
Brief Discussion of Method Principles
Optional Method(s)
:
3. Interferences
Sources of Contamination
Practices/Materials to Avoid
QC Checks for Contamination
4. Safety
Compliance to OSHA Regulations
Special Chemical Concerns
Safety Procedures
5. Apparatus and Materials
Major Systems
Testing equipment
Functioning ranges
Specifications
Alternative systems
V-
6. Test Gases or Liquids
Chemicals/Gases
Define if necessary
Purchasing information
Procedures to make solutions
Storage requirements
Expiration
QC reagents discussed in QC section
7. Calibration
Per Instrument Used
Stepwise procedures
Alternative Procedures (include equations/define variables)
(Continued)
-------
TABLE 2. (Continued)
8. Quality Control
Established QC Program
Minimum requirements
Record keeping
Special procedures
Accuracy and precision
Blanks
Spikes
Ongoing system control
9. Procedure •• .
Stepwise Guide Through Method, Including:
Operating conditions
Estimated responses
Calibration
QC
Atypical procedures
10. Calculations
Discussion of All Calculations
Definition All Variables Used
11. Method Performance (i.e., how reliable is this method)
Reliability of Method
Specific ranges (if applicable)
Operator
Overall
Method Accuracy
12. Definitions
For terms that have meanings that are particular to the method or
are ambiguous
13. References
10
-------
The Reagent section describes requirements for all chemicals that are
used in a procedure. This section is entitled "Test Gases" or "Test Liquids"
for vapor-phase or liquid-phase methods, respectively. Requirements include
concentrations, tolerances, and quality or purity. Storage recommendations
and necessary mixing instructions are also given.
The Calibration section provides a stepwise guide for calibrating equip-
ment. This equipment may include rotameters, timers, thermocouples, etc.
Frequency and acceptance criteria for calibrations are provided. Calibration
of actual petroleum hydrocarbon detectors is only briefly discussed in this
section. Complete calibration instructions for detectors should be provided
by the manufacturer or vendor.
The Quality Control section describes minimum steps and acceptance cri-
teria that are necessary to establish that procedure results are reliable.
Quality control steps may include replicate analyses, record keeping, and
blank testing. Frequency, acceptance criteria, and corrective actions for
quality control methods are listed in this section.
The Procedure section includes detailed step-by-step instructions of how
to perform the procedure. This section guides the technician through equip-
ment assembly, quality control procedures, calibration, testing, and expected
results.
The Calculations section provides all the equations necessary to perform
the procedure and reduce resulting data. Equations for quality control, data
manipulation, and data reporting are provided.
The Method Performance section gives the technician an impression of the
procedure's reliability. This section addresses procedure accuracy, repeata-
bility, reproducibility, range, applicability, ruggedness, testing duration,
and limitations. (This section can be completed only after validation testing
11
-------
and is incomplete in the methods that have thus far been submitted for
review.)
The Definitions section is a list of terms and their meanings that apply
specifically to these methods.
References provide a list of references that were used in writing the
procedure.
DEVICE ACQUISITION (WORK ASSIGNMENT 02)
After sets of representative vapor-phase and liquid-phase detection
devices were identified and categorized, suppliers were contacted. In August
1987. Radian Corporation began acquiring a limited number of external leak and
release detection devices from vendors to undergo initial execution of newly
drafted performance evaluation test procedures. Efforts were made to acquire
several devices from each category of sensor type established in the prior
vendor survey (1).
Several selection criteria were established to guide selection of the
external leak and release detection devices to undergo initial performance
testing. The selection criteria were: detector type, number of devices in
service, specificity for UST leak detection market, and commercial availabil-
ity. Vendors were contacted; agreements with Radian to borrow the sensor
systems for testing were executed; and the devices were sent to the Radian
laboratories for performance evaluation testing. A more detailed description
of the device acquisition plan was presented in a draft technical note to EPA
(5) and subsequently updated in monthly progress reports.
Devices that were acquired represented several categories of scientific
operating principles within the larger categories of liquid-phase and vapor-
phase leak and release detection devices. The following types of external
petroleum hydrocarbon liquid-phase leak and release detection devices were
12
-------
acquired: electrical conductivity sensors, interface probes, product-
permeable probes, and product-soluble sensors. External petroleum hydrocarbon
vapor-phase leak and release detection devices were representative of the
following sensor classes: metal oxide semiconductors, diffusion sensors,
catalytic sensors, photoionization sensors, and infrared sensors.
SECOND DRAFT PERFORMANCE EVALUATION TEST PROCEDURES (WORK ASSIGNMENT 02)
After the draft performance evaluation test procedures were executed on a
limited basis in the Radiap laboratories, these procedures were modified from
the procedures given in the Quality Assurance Project Plan for Work Assignment
02 (6). The second draft performance evaluation test procedures were sub-
mitted to EPA in March 1988 (2). Modifications included minor changes to the
test apparatuses to accommodate different sensor shapes and sizes.
PRELIMINARY LEAK DETECTION PERFORMANCE DATA
Only 13 external petroleum hydrocarbon leak and release detectors were
tested using the draft procedures. No more than two detectors representing
any one operating principle were tested. Therefore, the discussion of results
(Section 4) of this initial phase of testing using draft procedures is
limited. No specific conclusions have been made about the adequacy of opera-
ting principles from performance of the draft test procedures for performance
evaluation. Since these methods have not been validated, these data should be
treated as preliminary.
VALIDATION OF PERFORMANCE EVALUATION TEST PROCEDURES
•Validation tests will be developed and performed in the next phase of
this program. Individual tests for validating each of the draft performance
evaluation test procedures will be developed. The major objective of these
validation test procedures will be to evaluate the ability of each of the
draft procedures to produce valid test data within acceptable error limits.
13
-------
The results from performing the validation tests (in a future work assignment)
will be used to identify any limitations in the performance evaluation test
procedures.
The exact details of each validation test will depend on the associated
performance test procedure. The overall approach to be used in developing the
validation process will be the same for all cases and is patterned after the
successful approach developed by the EPA for the new source performance test
methods. The overall approach consists of the following steps:
• Technical examination;
• Laboratory evaluation;
• Ruggedness testing; and
• Validation testing.
Subsequent to the validation testing, final versions of performance eval-
uation test procedures will be submitted for EPA review. Final validation may
require establishing the relationship of laboratory performance to field per-
formance for network leak detection design.
COMPLETION OF PROPOSED PERFORMANCE EVALUATION TEST PROCEDURES
The purpose of developing performance evaluation test procedures is to
provide a means of establishing a uniform, consistent database of performance
characteristics for external leak and release detection and monitoring
devices. Over the long-term course of this project, it is likely that other
procedures for new technologies and additional performance parameters to those
given in Table 1 will be identified. As performance evaluation procedures are
refined-and validated, the procedures document will be updated.
14
-------
SECTION 4
RESULTS
This section is a summary of results and observations made during the
preliminary execution of performance evaluation test procedures to determine
specificity, accuracy, bias, precision, and response time of external
petroleum hydrocarbon leak and release detection systems. The test matrix
given in the project plan (6) and procedures given in the draft methods manual
(2) were followed.
None of the results presented in this section have been validated or
approved for disclosure by EPA or any of its contractors. The performance
evaluation results obtained for specific devices were secondary to the task of
developing test procedures. Although all due diligence was taken in the exe-
cution of these procedures, the results summarized below must be considered
preliminary for several reasons. First, the draft procedures performed have
not yet been scientifically validated. Changes in the procedures may alter
the responses observed for some of the devices. Secondly, the number of
devices tested was extremely small compared to the universe of devices on the
market. Therefore, no correlation trends assigned to specific operating prin-
ciples can be made at this time. Although a wide variety of operating princi-
ples, were represented, no more than two detectors from a single category were
tested. Again, the major emphasis of this work was to develop test proce-
dures, not to test the devices.
The following discussion is organized by the performance categories:
specificity, accuracy, and response time. Within each performance category,
the test matrix, test procedure, and results are organized under subcategories
for vapor-phase and liquid-phase detectors. Special efforts, have been made in
15
-------
the data presentation to maintain the anonymity of devices and associated
principles of operations, as agreed upon by the participating vendors. Radian
Corporation, and EPA Environmental Monitoring Services Laboratory in Las
Vegas, Nevada.
RESULTS OF SPECIFICITY TESTING
The significance of specificity (degree of response) of an external leak
detector is twofold. If a detector's response is not specific, it may respond
to compounds in the environment that are not associated with UST petroleum
hydrocarbon releases, causing a false alarm (false positive) situation. On
the other hand, if the detector is highly specific (selective), then a low
probability of detection may result (false negative). Therefore, one of the
most important performance evaluation parameters for external petroleum hydro-
carbon leak detection systems is specificity. Specificity answers the ques-
tion, "What will the detector detect?" Specificity is especially important
when the stored product is a single-component hydrocarbon.
Proper network design, site-specific factors, and calibration for exter-
nal detectors will affect field specificity. These issues were not addressed
in these laboratory benchmark test procedures.
Vapor-Phase Detectors - Specificity Response
The specificity of a vapor-phase detector was defined for the procedure
as a system's ability to indicate either the presence or concentration of a
pure chemical or a mixture of petroleum hydrocarbons at 250 ppmv (certified
gas concentration in ultrahigh-purity air). The test gases were monocomponent
n-hexane, benzene, toluene, n-butane, isobutane, 2-methylpentane. and 3-
methylpentane gas standards. Each of the seven vapor-phase leak detection
systems was randomly tested with each gas near 250 ppmv with 20% replicate
measurements. The replicate quality control test gas was chosen at random for
16
-------
each leak detector. Detector responses were initially measured in ultrahigh-
purity air, monitored for response to a test gas, then purged with ultrahigh-
purity air according to the test matrix given in the project test plan (6)
using the second draft test procedure (2). Results from these tests were used
to calculate response factors for quantitative-output detectors for each test
gas, and activated versus nonactivated responses for qualitative-output detec-
tors for each test gas. These results are summarized in Table 3.
Three qualitative-output vapor-phase detectors (A, B, and C) failed to
respond to any of the test .gases at a concentration of 250 ppmv. These detec-
tors were subsequently tested with n-hexane at 500 ppmv, and only Detector C
responded at the higher concentration. These detectors had similar modes of
operation and do not represent the wide variety of qualitative-output vapor-
phase detectors on the market.
The four quantitative-output vapor-phase detectors (D, E, F, and G)
showed specificity for the test gases, with differences in response factors.
Some detectors were specific to the chemical class of compound (aliphatic
versus aromatic), while others showed specificity response factors that
appeared to be a combination of chemical class and other factors such as
molecular weight. For example, Detector D was most specific (had high
response factors) for 2-methylpentane, isobutane, n-hexane, and isopentane,
respectively. It was less specific (had lower response factors) for n-butane
and toluene. While Detector E's specificity was consistent, response factors
exceeded 100% for four of the test gases. The gas concentrations tested were
at the lower end of Detector E's operational range. Detector F was most spe-
cific for benzene and toluene (aromatic compounds) but demonstrated very low
specificity response factors for the other test gases consisting of aliphatic
compounds. Detector G had accurate specificity for n-hexane and 2-methyl-
pentane but lower response factors for the other aliphatic compounds (n-
butane, isobutane, and isopentane) and very low response factors for benzene
and toluene (common aromatic components in fuels).
17
-------
oo
TABLE 3. SUMMARY OF VAPOR-PHASE LEAK DETECTORS SPECIFICITY TEST RESULTS8
(Percent Response to Test Gases)
Bact
Detector
Identification Ul
:ground
n-
1P Air Butane Isobutane
Test Gases
n- 2-Methyl-
Isopentane Hexane pentane
Benzene Toluene
Qualitative-Output Detectors
A
B
C
Quantitative-Output
D
E
F
G
NA NA NA
I
NA NA NA
NA NA NA .
Detectors
NRd 11 63
NRd 100 200
NRd 1 1
NRd 68 63
NA — b NA .
•»
NA ~b NA
NA ~° NA
57 61 65
200 200 200
6 10 12
77 96 92
..
NA NA
NA NA
NA NA
42 17
100 150
100 102
2 14
Specificity, the ability of a detector to indicate the presence or concentration of a test gas at
250 ppmv. is reported in terms of response factors calculated as the percent of the detection
system's response compared to the certified concentration of the test gas for quantitative-output
detectors and as "activated" (A) or "non-activated" (NA) for qualitative-output detectors.
bNo response to either 250 ppmv or 500 ppmv of n-hexane.
CActivated response to 500 ppmv of n-hexane but no response to 250 ppmv of n-hexane.
dNR = no response for quantitative-output detectors.
-------
Liquid-Phase Detectors - Specificity Response
For liquid-phase petroleum hydrocarbon leak and release detection sys-
tems, specificity was defined in the performance evaluation test procedure as
the response to a 1.27-cm (1/2-in.) layer on water of different chemical
substances representing components or types of petroleum hydrocarbons commonly
stored in UST systems. The test liquids used in the performance evaluation
procedure were: n-hexane, xylene(s), toluene, commercial gasoline, a
laboratory-mixed "standard" gasoline, commercial diesel fuel, and commercial
jet fuel. Each of six external liquid-phase petroleum hydrocarbon leak and
release detection devices was randomly tested using each test liquid with 20%
replicate measurements. The replicate quality control test liquids were
chosen at random for each liquid-phase detector. Detectors were measured for
background responses in tap water, then for responses in each test liquid with
a complete cleaning of the test apparatus between tests. The matrix from the
project test plan (6) and the second draft test procedure (2) were followed.
Results from these tests were used to determine if there was a response for
specific compounds within 24 hours.
Test results indicated five of the six liquid-phase detectors responded
to a 1.27-cm (1/2-in.) thick layer for all seven test liquids on water within
24 hours. The sixth liquid-phase detector responded to all test liquids
within 24 hours, except commercial diesel fuel, to which it responded in 26
hours.
The ranges of observed response times during specificity testing are
summarized by test liquid in Table 4. The longest time until response for all
test liquids was observed for the same detector (Detector D). Likewise, the
shortest time required until response to all test liquids was observed for a
single detector (Detector B).
Detector D required over 16 hours to respond to n-hexane, xylene(s),
diesel fuel, and jet fuel; it responded to toluene, commercial gasoline, and
synthetic gasoline mixture in under 2 hours. Detector B showed no specificity
19
-------
TABLE 4. SUMMARY OF RESPONSE TIMES TO TEST LIQUIDS DURING LIQUID-PHASE
SPECIFICITY TESTING
Test Liquid
, Ranges of Observed^
Time Until Response'
n-Hexane
Xylene(s)
Toluene
Simulated Gasoline-
Commercial Gasoline
Diesel Fuel
Jet Fuel
<1 sec - 17.5 hr
<1 sec - 21.5 hr
<1 sec - 1.5 hr
<1 sec - 1 hr
<1 sec - 1 hr
<1 sec - 26 hr
<1 sec - 16.5 hr
aThe range is based on the response times for six detectors.
20
-------
toward any of the test liquids, responding to all of them in less than 1
second. Detector A required over 9 hours to respond to diesel fuel but
responded to all other test liquids in less than 20 minutes. Detector E
responded to xylene(s), toluene, synthetic gasoline mixture, and commercial
gasoline in 2 minutes or less; it required 6 to 7 hours to respond to n-
hexane, diesel fuel, and jet fuel. Detectors C and F responded to all test
liquids in under 10 minutes.
i
RESULTS OF ACCURACY AND RESPONSE TIME TESTING
Accuracy, in general, is the degree of agreement between measured values
and the true values. In these methods, the description of accuracy is quite
different for qualitative-output versus quantitative-output detectors. Quali-
tative accuracy in these methods is a measure of positive responses, where
100% accurate means the detector responded positively 100% of the time. Quan-
titative accuracy is defined as "relative accuracy," following the convention
used for continuous emission monitoring systems where 100% relative accuracy
means the maximum expected bias (at the 95% confidence level) is +100% of the
true value. Thus, qualitative and quantitative accuracy in these methods can-
not be directly compared. Relative accuracy has no meaning for qualitative-
output detectors; however, relative accuracy may serve as the prime comparator
for quantitative-output detectors and may be determined in the laboratory
against known test conditions or in the field versus a reference method.
Bias is the term used for the systematic error inherent in a method, and
it can be positive or negative. Precision is the degree of agreement between
repeated measurements of the same parameter, and it reflects random errors
that are unaffected by bias. Neither bias nor precision are applicable to
qualitative devices, which only produce a binary ("on/off") signal.
Detection time is the sum of lag time (the elapsed time from a detector's
first contact with the test gas or liquid to the first detectable signal) and
rise time (the elapsed time from a detector's first detectable signal in
response to hydrocarbon liquid or gas to an output that is within 95% of full
21
-------
scale or "activated"). Fall time is the elapsed time from a hydrocarbon-free
condition after exposure to hydrocarbon until a detector's output returns to
within 95% of its original baseline level or there is no detectable signal
output.
Several parameters measured in this suite of tests describe accuracy;
others describe time-related statistics that are important to the owner of
underground storage tanks. Accuracy answers the question, "How close is the
hydrocarbon value measured by a leak detector to the actual hydrocarbon level
present?" Bias answers the question, "Is the detection device higher or lower
than the real value?" Precision answers the question. "Is the test result
repeatable and is the uncertainty in the test result large enough to adversely
influence any decision process?" Detection time answers the question, "How
fast will the detector signal an alarm after it contacts hydrocarbons?" Fall
time answers the question. "How long will it take after a positive response
for a sensor placed in a clean environment to cease its positive alarm sig-
nal?" Answers to these questions are important because they indicate whether
an alarm will occur at an acceptable performance level. Again, the following
test results pertain to laboratory performance and not field performance.
Vapor-Phase Detectors - Accuracy Testing
Accuracy for qualitative-output vapor-phase detectors was defined in the
test procedure as the number of positive responses expressed as a percentage
of the total number of tests (five in this case). For quantitative-output
detectors, the relative accuracy was calculated as the absolute mean diffe-
rence between the measured value and the true value, plus the 2.5% error con-
fidence coefficient for each series of five replicate tests, divided by the
true value. It is expressed as a percentage of the reference (certified gas)
standard concentration. Accuracy for qualitative-output detectors is neither
conceptually nor numerically comparable to relative accuracy for quantitative-
output detectors.
22
-------
Bias, for this method, was expressed as the signed (positive or negative)
percent difference between the average measured value for a series of five
replicate tests and the true value. Precision was expressed in terms of the
percent coefficient of variation (CV), which is equal to the standard devia-
tion for each set of five values, divided by the mean of the set, and multi-
plied by 100.
Each of six external vapor-phase petroleum hydrocarbon leak and release
detection systems was tested with certified gas standards of benzene and
isopentane. Four concentration ranges (approximately 50, 250, 500, and 1000
ppmv) were tested for both compounds. Benzene and isopentane were used as
test gases because they are common components in petroleum products and many
types of vapor-phase detectors are sensitive to them. Concentration levels
were chosen based on preliminary information of the working concentration
ranges for total hydrocarbons in which vapor release detection might be most
appropriate.
The test procedure (2) required an initial background test with
ultrahigh-purity air to monitor the response of a test gas; record detection
time, response value, and fall timej then repeat these steps until all of the
tests were replicated five times for each test gas concentration, according to
the matrix of 240 total tests in the project test plan (6). Each of the rep-
licate tests was performed under equivalent test conditions. Duplicate analy-
ses consisted of immediate repetition of test conditions for 10% of the tests
chosen at random.
Table 5 summarizes vapor-phase accuracy, bias, precision, and response
time test results. In general, some of the vapor-phase detectors were very
accurate, while other detectors were less accurate. Some of the vapor-phase
detection systems were more accurate for a particular test gas, while others
showed similar accuracy for both test gases. Some detectors were more accu-
rate at higher test gas concentrations, while others were just as accurate at
the lower test gas concentrations as they were at the higher concentrations.
23
-------
TABLE 5. VAPOR-PHASE ACCURACY, BIAS, PRECISION, AND RESPONSE TIME RESULT SUMMARY
to
Accuracy "
Detector
Test Gas Concentration
(ppmv)
Qualitative
Accuracy
(%r
Relative
Accuracy Bias
(%)c (%)
Precision
(%)
Detection
Time
(minutes)
Fall
Time
(minutes)
Qualitative Detectors
B
B
B
B
B
B
B
B
C
C
C
C
C
C
C
C
quantitative
D
D
D
D
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
Detectors
Benzene
Benzene
Benzene
Benzene
48
240
500
990
49
253
499
991
48
240
500
990
49
253
499
991
48
240
500
990
NTd
NT
NT
0
NT
NT
0
0
NT
0
100
100
0
100
100
100
—
NLe
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
44 -38
58 -55
61 -56
64 -55
NL
NL
.. NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
7
6
8
16
NDf
ND
ND
ND
ND
ND
ND
ND
ND
ND
42.40
9.60
ND
24.00
42.00
42.40
8.00
8.00
8.00
16.60
ND
ND
ND
ND
ND
ND
ND
ND
ND
ND
3.40
4.40
ND
3.20
7.80
3.20
8.00
8.00
8.00
8.00
-------
TABLE 5. (Continued)
to
Cn
Detector3
Test Gas
Qualitative
Concentration Accuracy
(ppmv) (%)
Relative
Accuracy
Bias
Precision
Detection
Time
(minutes)
Fall
Time
. (minutes)
Quantitative Detectors (Cont.)
G
G
G
G
G
G
G
G
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
48
240
500
990
49 — — —
253
499
991
99
98
98
98
16
17
20
25
-99
-98
-98
-98
-13
-16
-18
-24
39
8
- 5
5
3
1
2
1
1.15
0.25
0.60
0.35
0.30
0.35
0.30
0.50
0.30
0.25
0.30
0.35
0.45
1.05
2.30
0.70
Tletector A was not tested because it did not respond during specificity tests.
Qualitative Accuracy - For qualitative-output detectors, the number of positive responses to the test gas. expressed as a
percentage of the number of tests (five) performed. A value of 100% means the detector responded appropriately for all five
tests in the series; 0% means the detector failed to respond during any of the tests.
Relative Accuracy - For quantitative-output detectors, the absolute mean difference between the measured value and the true
value for the test gas. plus the 2.5% error confidence coefficient, expressed as a percentage of the true value. A value of
100% for relative accuracy means that, within the 95% confidence interval, the measurement bias can be expected not to
exceed +100% of the true value.
NT = not tested at this concentration.
g
NL = not applicable because the detector's output was qualitative.
ND = not applicable because the test gas was not detected.
""Detection time and fall time values are only for test gas responses that were above baseline responses with ultrahigh-purity
air.
-------
TABLE 5. (Continued)
to
Detector
Test Gas Concentration
(ppmv)
Quantitative Detectors (Cont.!
D
D
D
D '
E
E
E
.E
E
E
E
E
E
E
E
E
F
F
F
F
F
F
F
F
Isopentane
Isopentane
Isopentane
Isopentane
Benzene
Benzene
Benzene
Benzene
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
>
49
253
499
991
48
240
500
990
48
240
500
990
49
253
499
991
48
240
500
990
49
253
499
991
Qualitative Relative
Accuracy Accuracy
(%)B tt)c
46
54
29
42
687
370
150
173
687
370
150
173
1206
295
543
267
13
g
20
35
95
97
98
97
Bias
(%)
-33
-42
24
61
108
254
150
132
108
254
150
132
512
295
321
233
10
-6
-17
-34
-93
-96
-96
-97
Precision
(%)
16
17
3
- 7
224
26
0
14
224
26
0
14
91
0
43
8
3
3
4
1
28
20
25
0
Detection
Time
(minutes)
6.40
8.00
4.00
6.40
0.25B
0.45
0.50
0.90
0.25s
0.45
0.50
0.90
0.55
0.30
0.65
2.40
0.25
0.30
0.25
0.25
0.25
0.30
0.30
0.25
Fall
Tine
(minutes)
7.20
8.00
7.20
8.00
0.258
0.80
1.00
2.15
0.25s
0.80
1.00
2.15
0.30
0.70
1.70
1.85
0.40
0.55
0.30
0.95
0.30
0.25
0.25
0.20
(Continued)
-------
Vapor-phase Detectors A, B, and C were qualitative-output detectors.
Detector A was not tested because it did not respond to any of the specificity
test gases at 250 ppmv or to hexane at 500 ppmv. Its mode of operation was
almost identical to Detector B. Detector B did not respond during accuracy
testing to either test gas at any concentration tested. Detector C responded
in all of the accuracy tests for 500 ppmv and 990 ppmv of benzene and for 253
ppmv, 499 ppmv. and 991 ppmv of isopentane. (The fact that Detector C re-
sponded at 253 ppmv in these tests and did not respond at the same concen-
tration during the specificity tests indicates that this concentration may be
very close to the detector's detection limits.)
Vapor-phase Detectors D, E, F, and G were quantitative-output detectors.
Detector D's relative accuracy ranged from 29% to 64% for benzene and isopen-
tane. - Relative accuracy (RA) values were similar for benzene and isopentane
at all concentration levels. Detector E demonstrated poorer relative accuracy
(RA = 150% to 1206%). Detector F was more accurate for benzene (RA = 9% to
35%) than for isopentane (RA = 95% to 98%). Detector G's relative accuracy
values were more accurate for isopentane (RA = 16% to 25%) than for benzene
(RA = 98% to 99%).
Average bias values for quantitative-output vapor-phase detectors ranged
from -99% to a high of 512%. In general, isopentane bias values tended to
decrease as the test gas concentration increased. The best bias values were
-6% and 10% for Detector F in benzene at 240 ppmv and 48 ppmv, respectively.
Bias is not applicable to qualitative-output detectors., which produce only a
binary ("on/off") output signal, and it was not calculated for vapor-phase
Detectors A, B, and C. For Detector D, benzene bias was from -56% to -38%.
For isopentane, this detector showed negative bias at 49 ppmv and 253 ppmv and
positive bias at 499 ppmv and 991 ppmv. The bias magnitudes for both test
gases were similar. For Detector E, bias values ranged from 108% to 512%.
Isopentane bias values (RA = 233% to 512%) were higher than benzene bias
values (RA = 108% to 254%). Benzene bias values for Detector F were from -34%
27
-------
to 10%, and the bias decreased with increasing test gas concentration! Detec-
tor F isopentane bias results ranged closely from -97% to -93%. As with rela-
tive accuracy results. Detector F bias results followed an almost opposite
trend from bias results for Detector G. Detector G bias was -99% to -98% for
benzene and -13% to -24% for isopentane.
In general, all four quantitative-output detectors were very precise.
All but one detector showed similar precision values for both test gases. The
poorest precision values (91% and 224%) were observed for the lowest test gas
concentrations with a single detector. All other precision values were under
50%. Precision is not applicable to qualitative-output detectors, which pro-
duce only a binary ("on/off") output signal, and it was not calculated for
vapor-phase Detectors B and C.
Detector D's precision ranged from 3% to 17% CV. Precision values for
Detector E ranged from 0% to 224% coefficient of variation (CV). showing less
precision at the lowest test concentrations for both gases than at higher
concentrations. As previously noted, test gases were at the low end of
Detector E's operating range. Precision values for Detector F ranged from 0%
to 28% CV. Even though this detector's best precision value was for 991 ppmv
isopentane, benzene precision values were better than isopentane precision
values overall. Detector G precision values were from 1% to 39% CV. Other
than a 39% benzene precision value, values were under 10% for Detector G.
This detector was slightly more precise for isopentane than for benzene.
Liquid-Phase Detectors - Accuracy Testing
Accuracy for qualitative-output liquid-phase detectors was defined in the
test procedure as the number of positive responses for a particular test
liquid at a particular thickness, expressed as a percentage of the total num-
ber of tests in the series (five in this case). For quantitative-output
liquid-phase detectors, the relative accuracy was calculated as the absolute
mean difference between the measured value and the true value, plus the 2.5%
28
-------
error confidence coefficient for each series of five tests, divided by the
true value. It is expressed as a percentage of the known thickness of hydro-
carbon layer on water. As with vapor-phase detectors, accuracy, as defined in
these methods, is neither conceptually nor numerically comparable for qualita-
tive-output versus quantitative-output detectors.
Bias is expressed as the signed (positive or negative) percent difference
between the average measured value for each series of five replicate tests and
the true value for the test liquid thickness. Precision is expressed in terms
of the percent coefficient of variation, which is equal to the standard devi-
ation for each set of five values divided by the mean of the set and multi-
plied by 100.
Each of six external liquid-phase petroleum hydrocarbon leak and release
detection systems was tested with three thicknesses (0.04, 0.32, and 0.64 cm)
each of simulated (laboratory-generated) gasoline mixture and commercial gaso-
line. The simulated gasoline mixture will provide a uniform protocol for
future testing; commercial gasolines frequently have differing product compo-
sitions that may affect test results.
The test procedure (2) required an initial background test in tap water
to'monitor the response to each test liquid and thickness as a function of
time, then repeating these steps until all of the tests were replicated five
times for each liquid product and thickness in accordance with the matrix of
180 total tests given in the project test plan (6). Each of the replicate
tests was performed under identical conditions.
Table 6 summarizes the test results for accuracy, bias, precision, and
response time testing for liquid-phase petroleum hydrocarbon detectors.
Detector B was the only detector capable of providing quantitative out-
put. Relative accuracy for this detector ranged from 1120% for the simulated
29
-------
TABLE 6. LIQUID-PHASE ACCURACY, BIAS, PRECISON, AND RESPONSE TIME "RESULTS
SUMMARY
_
Criteria Gasoline Type
Accuracy Simulated
Commercial
Bias Simulated
Commercial
Precision Simulated
Commercial
Detection Simulated
Time
(minutes)
Commercial
Hydro-
carbon
Thickness
(cm)
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
Detector
A
100%a
100%a
100%a
100%a
100%a
100%a
c
c
c
c
c
c
d
d
d
d
d
d
6.18
0.83
0.58
1.37
1.30
0.70
B
11202?'
147%?
61?
444%?
87%?
42?
606%
114%
45%
. 102%
57%
42%
59%
12%
9%
137%
18%
0%
e
<0.02
<0.02
e
<0.02
<0.02
C
100%a
ioo%a
100%a
100%a
100%a
100%a
c
c
c
c
c
c
d
d
d
d
d
d
2.73
0.32
0.32
2.42
0.23
0.27
D
ioo%a
100%a
100%a
100%a
100%a
100%a
c
c
c
c
c
c
d
d
d
d
d
d
104.60
72.83
57.63
121.55
64.22
73.10
E
100%a
ioo%a
100%a
100%a
ioo%a
100%a
c
c
c
c
c
c
d
d
d
d
d
13.30
0.08
0.32
8.13
0.13
0.12
F
«:
100%a
100%a
0%*
100%a
100%a
c
c
c
c
c
. c
d
d
d
d
d
f
0.08
0.17.
f
0.18
0.10
30
-------
TABLE 6. (Continued)
Criteria Gasoline Type
Fall Time Simulated
(minutes)
Commercial
Hydro-
carbon
Thickness
(cm)
0.04
0.32
0.6A
0.04
0.32
0.64
Detector
A
g
g
g
g
g
g
B
h
k
k
h
k
k
C
i
i
2.
±
i
i
D
g
g
g
g
g
g
E
g
g
g
g
g
g
F
j
k
k
j
k
k
Qualitative Accuracy - For qualitative-output detectors, the number of positive responses to the
test liquid, expressed as a percentage of the number of tests (five) performed. A value of 100%
means the detector responded appropriately for all five tests in the series; 0% means the detector
failed to respond during any of the tests.
^Relative Accuracy - For quantitative-output detectors, the absolute mean difference between the
measured value and the true value for the test liquid, plus the 2.5% error confidence coefficient.
expressed ae a percentage of the true value. A value of 100% for relative accuracy means that.
within the 95% confidence interval, the measurement bias can be expected not to exceed +100% of the
true value.
Bias not applicable because device is qualitative.
Precision was not applicable because the device was qualitative.
elmmediate response when any hydrocarbon was detected.
Not applicable because no response was recorded.
^Not applicable because the probe was destroyed during the test.
immediate when activation occurs.
^Tfo deactivation after 24 hours in water. Will deactivate in air.
applicable because there was no response.
Immediate deactivation (fall time <1 second).
31
-------
gasoline mixture at 0.04 cm (1/64 in.,) to 42% at 0.64 cm (1/4 in.) for commer-
cial gasoline. Poor accuracy results at the 0.04 cm (1/64 in.) thickness were
expected because the manufacturer claimed a 0.32 cm (1/8 in.) lower detection
limit. Accuracy values at 0.32 cm (1/8 in.) and 0.64 cm (1/4 in.) were much
better, ranging from 147% to 42%. Detector B showed similar accuracy for both
test liquids.
Bias for external liquid-phase petroleum hydrocarbon detectors was only
applicable to the one quantitative-output detector (Detector B) that was
tested. Bias values for this detector ranged from 606% at 0.04 cm of simu-
lated gasoline to 42% at 0.64 cm of commercial gasoline. As with the relative
accuracy values for this detector, bias values for 0.32 cm and 0.64 cm test
liquid thicknesses, which were equal to or higher than the detector's lower
detection limit, were significantly better than the values at the 0.04 cm test
liquid thickness.
Precision for liquid-phase detectors was only applicable to Detector B
because it was the only quantitative-output detector tested. Precision ranged
from 137% at 0.04 cm (1/64 in.) to 0% at 0.62 cm (1/4 in.) of commercial gaso-
line. As with relative accuracy and bias, precision values were much better
for test liquid thicknesses that were above the detector's lower detection
limit. Precision values at the 0.04 cm thickness were 59% and 137% for the
synthetic gasoline mixture and commercial gasoline, respectively. Precision
values at test thicknesses of 0.32 cm and 0.64 cm ranged from 18% to 0%.
Only one of five qualitative-output detectors failed to respond to test
liquids at each test thickness within 24 hours. Detector F, whose manufac-
turer reported a detection limit of 0.32 cm (1/8 in.), failed to respond to
either test liquid at 0.04 cm (1/64 in.). This same detector, however, did
respond in all tests to both test liquids at the two higher test liquid thick-
nesses. In other words, this detector was 0% accurate at a test liquid thick-
ness of 0.04 cm but 100% accurate at test liquid thicknesses of 0.32 cm and
0.64 cm.
32
-------
Vapor-Phase Detectors - Response Time Testing
Detection time and fall time results for vapor-phase detectors were given
in Table 5. Vapor-phase detectors typically responded faster than their
liquid-phase counterparts. The range of average detection times ranged from
less than 15 seconds to 1 hour and 24 minutes. The vast majority of responses
occurred in under 1 minute. Vapor-phase fall time values were smaller than
detection time values. All fall time values were less than 1C minutes, and
most values were under 1 minute.
Liquid-Phase Detectors - Response Time Testing
Detection time for liquid-phase detectors, as shown in Table 6, was gen-
erally 2 hours or less. For all but one detector, responses occurred in less
than 15 minutes if the test liquid was above the detector's lower detection
limit.
Detection times for liquid-phase Detectors B and F were very short (<1 to
11 seconds) at all test liquid thicknesses at which these detectors responded.
These detectors did not respond at the 0.04 cm test liquid thickness, which
was below the detection limit of both devices.
Detectors A, C, and E responded at all three thicknesses of both test
liquids in under 15 minutes. The longest average detection time for a set of
five replicate tests from this group of detectors was 13.3 minutes for the
simulated gasoline mixture at 0.04 cm. The shortest average detection time
was 7 seconds for commercial gasoline at 0.64 cm. Detection times were much
higher for test liquid thicknesses at 0.04 cm (1 minute, 22 seconds to 13
minutes, 18 seconds) than for 0.32 cm and 0.64 cm test liquid thicknesses (7
seconds to 1 minute, 18 seconds). There did not appear to be significant
differences between simulated versus commercial gasoline or test liquid thick-
nesses of 0.32 cm versus 0.64 cm. Detector D averaged from 57 minutes to 2
33
-------
hours for responses. Response times between tests with simulated gasoline
were not significantly different from those for commercial gasoline.
Fall time was generally not a significant factor for liquid-phase detec-
tors. Fall time was not applicable to Detectors A. D. and E because they
required replacement of a product-soluble component that was destroyed during
contact with test liquid. Detector C would not return to an inactivated state
when tested for fall time in water. This detector would return to an inacti-
vated state if the probe was tested for fall time in air. Detectors B and F
had fall times that were less than 1 second if the detectors responded to the
test liquid. If these detectors did not respond to a test liquid, fall time
was not applicable.
34
-------
REFERENCES
1. Eklund, B.M. and W.L. Crow. Survey of Vendors of External Petroleum Leak
Monitoring Devices for Use with Underground Storage Tanks. EPA Contract
No. 68-02-3994, Work Assignment 44. March 11, 1987.
2. Eklund, A.G., Achord,-R.D., Kienbusch, M.R., et al. Development of
Procedures to Assess the Performance of External Leak Detection Devices:
Performance Test Procedures, Draft Report. EPA Contract No. 68-03-3409,
Work Assignment No. 02. March 11, 1988.
3. Eklund, B.M. and W.L. Crow. Development of Interim Performance Criteria
for External Petroleum Leak Monitoring Devices Used with Underground
Storage Tanks. Draft Technical Note. EPA Contract No. 68-02-3994, Work
Assignment 70. November 14, 1986.
4. Eklund, A.G., Kienbusch, M.R., Achord, R.D.* et al. Development of
Procedures to Assess the Performance of External Petroleum Leak Detection
Devices: Draft Performance Test Procedures. EPA Contract No. 68-02-
3994, Work Assignment 92. March, 1987.
5. Kienbusch, M.R. and Randall, J.R. Device Acquisition Plan. Technical
Note. EPA Contract No. 68-03-3409, Work Assignment 02, Task 02. August
1987.
6. Behrens, G.P. and Adams, K.M. Development of Performance Criteria for
External Petroleum Leak Detection Devices: Quality Assurance Project
"Plan and Test Plan for Laboratory Testing. EPA Contract No. 68-03-3409,
Work Assignment No. 02. Plan for Laboratory Testing. August 1987.
35
-------
APPENDIX A
MATHEMATICAL FORMULAS
Vapor-Phase Detectors
Relative accuracy—The relative accuracy (RA) of a set of data was calcu-
;
lated as follows:
RA =
CC
100
(1)
where: V = reference (theoretical) value;
d = arithmetic mean of the difference of a data set. Equation 1.1;
and
cc = 2.5% error confidence coefficient (one tailed. Equation 1.2).
Mean difference—The arithmetic mean of the difference (d) of a data set
was calculated as follows:
n
d = - X d.
(1.1)
where: d. = measured response - theoretical response.
Confidence coefficient—The one-tailed 2.5% confidence coefficient (cc)
was calculated as follows:
cc = t
0.975
(1.2)
36
-------
where: S = the standard deviation (n-1) of the data set; and
t0.975 =2.5% (one-tailed) t value = 2.776 for n=5.
Accuracy for qualitative-output detectors—For qualitative-output detec-
tors, accuracy for a particular test atmosphere at a particular concentration
was calculated as the number of positive responses expressed as a percentage
of the total number of tests (five in this case) at that concentration with
that test atmosphere, according to the following formula:
A = 100 x (r /5)- (2)
q p
where: A = accuracy in percent; and
r = number of positive responses.
Bias—Bias is a measure of how much, on the average, the quantitative-
output detector responds high or low with respect to the true (theoretical)
response. Bias was calculated as follows:
Bias = 100 x [(V - V )/V ] (3)
o r r
! 5
where: V = the average observed value, r* Z V.;
0 D i=l 1
V. = the individual response to a test atmosphere; and
Vr = the reference (theoretical) value.
Detection time—Detection time was calculated according to the following
formula:
ET = T _ T (4)
/, 1
37
-------
where: DT = elapsed detection time;
T = clock time when the test atmosphere was first added to the test
chamber; and
T = clock time when the detector output went from an inactivated
2 state to an activated state for a qualitative-output detector or
from a baseline reading to 95% of stable high level output for a
quantitative-output detector.
Fall time—Fall time was calculated according to the following formula:
** = T2 ~ Tl
where: T?T = elapsed fall time;
T = clock time when the test atmosphere was switched to ultrahigh-
purity air; and
T = clock time when the detector output went from an activated state
to an inactivated state for a qualitative-output detector or from
a high level reading to within 95% of stable baseline level
output for a quantitative-output detector.
Relative percent difference—The relative percent difference is a measure
of variation between two observations. It is their absolute difference
divided by their average, expressed as a percentage, and was calculated as
follows:
RPD = 200 x l(Vl - V2)/(V1 + V2)] (6)
where: EPD = relative percent difference;
V1 = larger value; and
V = smaller value.
38
-------
Correlation coefficient—The correlation coefficient was calculated as
follows:
cc =
V s ' s
V xx yy
where: cc = correlation coefficient;
2
S = n£x. - (Zx.)
2Q£ J- J-
(7)
x. = i x value; and
x
.th -
y. = x y value.
x
Coefficient of variation (%) — The percent coefficient of variation is
equal to the standard deviation for a set of values, divided by the mean of
the set, times 100 and was calculated as follows:
CV(%) = (S/X) x 100 (8)
where: CV = coefficient of variation;
S = standard deviation for n values (n-1 degrees of freedom) ; and
X = the arithmetic mean of n values.
Response factor (RF) — Response factors were calculated as follows:
RF = C /C - (9)
m t
where: C = the vapor-phase detection system response (ppm-v, %, etc.); and
m
C = the concentration of the test atmosphere gas (ppm-v, %, etc.).
39
-------
Liquid-Phase Detectors . :
Water-miscible substances content — The fraction of water-miscible sub-
stances in commercial gasoline test product was calculated with the following
equation for a 20-mL sample:
(m - m.)
where: w = water-miscible substances content;
m = upper meniscus volume in mL; and
m. = lower meniscus volume in mL.
Area from immersing in water — The cross-sectional area of a probe was
measured by determining the volume occupied by the probe when immersed in
water. The following equation was used to calculate the cross-sectional area:
a. = (V. - V )/1.27 (11)
Q X p
2
where: a, = cross-sectional area in cm ;
a
V. = volume between marks without probe in mL;
V = volume between marks with probe in inL; and
P
1.27 = height of column of water displaced in cm.
Product volume — Before any testing began, a product layer of accurately
known thickness was formed on water in the test container. The volume of test
product to add to the test container to create a desired thickness of product
was calculated with the following equation:
t x (a - a,)
_ c d
v = - ; -
1 - w
40
-------
where: v = volume of product to add to container in mL;
t = desired product thickness in cm;
2
a = test container cross-sectional area (193.77 cm );
c 2
a, = estimated detector cross-sectional area in cm ; and
w = test product water content (w = 0 for synthetic gasoline).
Product thickness—The thickness of a test product layer on water was not
measured .but was determined from the volume of product added to the test con-
tainer, test container dimensions, and detector dimensions. The thickness of
product on water was determined with the following formula:
t = v/(a - a.) (13)
c d
where: t = test product thickness in cmj
3
v = volume.of test product in cm ;
2
a = container cross-sectional area (193.77 cm ); and
c 2
a, = detector cross-sectional area in cm .
d
Output at 95% of stable high level output—Detection time for
quantitative-output detectors was measured from the time the detector con-
tacted test product until it reached 95% of its final stable output. The
following formula was used to calculate the 95% of stable high level output, •
based on the stable baseline level output and the stable high level output.
HB = BL + (HL - BL) x 0.95 (14)
where: HB = output at 95% of stable high level outputj
BL = stable baseline output; and
HL = stable high level output.
Output within 95% of stable baseline output—Fall time for quantitative-
output detectors was measured from the time the detector was no longer in
41
-------
contact with test product until it reached 95% of its final stable baseline
output. The following formula was used to calculate the output within 95%
stable baseline level output based on the stable baseline level output and the
stable high level output.
OB = BL + (HL - BL) x 0.05
(15)
where: OB = output within 95% of stable baseline output;
BL = stable baseline output; and
HL = stable high level output.
Relative percent difference—The relative percent difference is a measure
of variation between two observations, neither of which is considered a refer-
ence value. It is their absolute difference divided by their average.
expressed as a percentage, and was calculated as follows:
RPD = 200 x [(V, - V_)/(V.. + V.)]
(16)
where: KPD = relative percent difference;
V- = larger value; and
X H
V0 = smaller value.
£* e
Coefficient of variation—The coefficient of variation indicates the
relative degree of variation associated with two or more values. It is often
reported as "Relative Standard Deviation." The coefficient of variation was
calculated as follows:
CV (%) = (S/X) x 100
(17)
where: CV = coefficient of variation;
S = standard deviation of n values (n-1 degrees of freedom); and
X = mean of n values.
42
-------
Accuracy—Calculation for accuracy of liquid-phase detectors depended on
the type of output that the detectors produced. Some detectors produced
quantitative output from which the thickness of product on water was deter-
mined. Other liquid-phase detectors only produced an alarm or other qualita-
tive signal that signified that petroleum product was present.
Quantitative-output liquid-phase detectors—Accuracy for quantitative-
* output detectors is a function of systematic error (hias) and random
error (precision). Accuracy was calculated as the sum of the absolute
value of the arithmetic mean of differences, |d|, and the absolute value
of the confidence coefficient, cc. Relative accuracy for liquid-phase
detectors, which is the accuracy at a given hydrocarbon thickness divided
by the hydrocarbon thickness and multiplied by 100, was calculated for
each test product at every hydrocarbon thickness.
Relative accuracy—The relative accuracy (RA) of a set of data was calcu-
lated as follows:
Jli_L_i££i x 100
r
where: V = reference (theoretical) value;
d = arithmetic mean of the difference of a data set. Equation
18.1; and
cc = 2.5% error confidence coefficient. Equation 18.2.
Mean difference—The arithmetic mean of the difference (d) of a data set
was calculated as follows:
d= - d (18.1)
where: d. = measured response - theoretical.
43
-------
Confidence coefficient—The one-tailed 2.5% confidence coefficient (cc)
was calculated as follows:
cc = t ' — (18.2)
cc ^.975
n
where: S = the standard deviation (n-1) of the data set; and
tn 0,c = 2.5% t value = 2.776 for n=5.»
o •
The relative accuracy was calculated for both commercial gasoline and
synthetic gasoline at every test product thickness. Relative accuracy
values were plotted against hydrocarbon thickness.
Qualitative-output detectors — For qualitative-output detectors, accuracy
for a particular test product at a particular thickness was calculated as
the number of positive responses expressed as a percentage of the total
number of tests at that thickness with that test product, according to
the following formula:
A = 100 x (r /5) .
I P
where: A = accuracy in percent; and
q
r = number of positive responses.
P
Bias — Bias is a measure of how much, on the average, the quantitative-
output detector responded high or low with respect to the true (theoretical)
response. Bias was calculated as follows:
Bias = 100 x [(V - V )/V ] (20)
o r r
44
-------
1 5
where: V = the average observed value, •=• Z V.;
o 5 i=1 i
V. = the individual response to a test atmosphere; and
Vr = the reference (theoretical) value.
Bias at a particular thickness for a particular test product was the
percent difference between the average detector output for a series of tests
and the actual thickness of test product. In this calculation, the average
detector output reading is used as the observed value (V ) in Equation 13, and
the actual test product thickness is the reference value (V ). Bias was
reported in percent.
Detection time—Detection time was the elapsed time between introduction
of the detector probe into the test container and when the detector reached a
positive response. The nature of a positive response was dependent on whether
the detector had quantitative or qualitative output; however, the calculation
for detection time was the same for both types of detectors. Detection time
was calculated according to the following formula:
DT = T2 - TI (21)
where: DT = elapsed detection time;
TI = clock time when liquid was first added to test container; and
T_ = clock time when detector output went from an inactivated state to
£•
an activated state for a qualitative-output detector or from a
baseline reading to 95% of stable high level output for a
quantitative-output detector.
Fall time—Fall time was the amount of elapsed time between removal of
test product from the test container and when the detector reached a negative
response. The nature of a negative response was dependent on whether the
45
-------
detector had quantitative or qualitative output; however, the calculation for
detection time was the same for both types of detectors. Fall time was calcu-
lated according to the following formula:
FT = T2 - TX
where: FT = elapsed fall time;
T = clock time when detector was removed from test container; and
T = clock time- when detector output went from an activated state to
an inactivated, state for a qualitative-output detector or from a
high level reading to within 95% of stable baseline level output
for a quantitative-output detector.
Specificity — Determination of specificity was dependent on whether a
detector's output was qualitative or quantitative.
Quantitative-output detectors — Specificity for quantitative-output detec
tors was defined as the ratio of detector output, or measured thickness, to
the actual thickness of hydrocarbon test product expressed as a percentage.
The following equation was used to calculate specificity for quantitative-
output detectors:
S = 100 x m,/t
Q
where: S = specificity in percent;
m, = detector's output reading in cm; and
d
t = hydrocarbon thickness in cm (t = 1.27 cm).
Specificity for qualitative-output devices was reported as "activated" if
the detector responded within 24 hours. Otherwise, specificity was reported
as "inactivated."
46
-------
GLOSSARY
Accuracy - The degree of agreement between measured values and the true
values. Accuracy in a single measurement reflects both systematic and random
error. Accuracy in multiple measurements estimates of the same parameter,
i.e., average accuracy, reflects systematic error, or bias.
Activated - Refers to the state of a qualitative detector's response when
indicating the presence of hydrocarbons.
Bias - The systematic error inherent in a method. Bias may be positive
or negative.
Continuous Petroleum Hydrocarbon Detection Systems (Detectors) - A class
of detectors that monitor in a constant, real-time mode, without interruption.
Detection Time - Sum of lag time and rise time.
Fall Time - The elapsed time after the test atmosphere is displaced by
ultrahigh-purity air until its output returns to within 95% of its original
baseline level or there is no detectable signal output.
Intermittent Petroleum Hydrocarbon Detection Systems (Detectors) - A
class of detectors that monitor on a. periodic basis. Interruptions may be due
to sample transfer, sample analysis, system removal, installation, detection
response time, etc. A continuous detector may be operated in an intermittent
mode.
Lag Time - The elapsed time from a detector's first contact with the test
atmosphere or test liquid to the first detectable signal.
47
-------
Memory Effects - An effect typically encountered when tests using test
atmospheres or test liquids of different concentrations or compounds are
performed in succession. The effect is caused by residual hydrocarbon vapors
from the previous test significantly interfering with the succeeding test.
Non-activated - Refers to the state of a qualitative output detector's
response when indicating that no hydrocarbons are detected.
Petroleum Hydrocarbon Detection System (Detector) - Detection system
(probes, lines, control bo*, readouts, etc.) for hydrocarbon compounds speci-
fically designed or used for vapor-phase petroleum product detection.
Precision - The degree of agreement of repeated measurements of the same
parameter. Precision estimates reflect random error and are not affected by
bias.
Probe - Component of a detection system that must come into contact with
petroleum vapor before the vapor can be detected.
Qualitative Response - A type of detector response that indicates only
the presence or absence of hydrocarbons without determining the hydrocarbon
concentration.
Quantitative Response - A type of petroleum hydrocarbon leak detection
system response that quantifies the concentration of the hydrocarbon present.
Relative Accuracy - The absolute mean difference between a group of mea-
sured values and the true value, plus the 2.5% error confidence coefficient,
divided by the true value. Relative accuracy is a measure of the maximum
(upper 95% limit) expected bias for a series of measurements.
Responses - The detector's indications of the presence or concentration
of petroleum hydrocarbons. These can be qualitative or quantitative.
48
-------
Response Factor - The ratio of the detector response (ppmv, %, e'tc.) to
* the test atmosphere or liquid concentration (ppmv, %, etc.).
» Response Time - A general term that refers to the more specific terms of
lag time and rise time, which together constitute detection time, and fall
time.
Rise Time - The elapsed time from a detector's first detectable signal in
response to hydrocarbon vapor to an output that is 95% of full scale.
Specificity (gas-phase) - The ability of a detector to detect a
particular chemical or class of chemicals at 250 ppm-v.
Specificity (liquid-phase) - The ability of a detector to respond to
different substances. For the purposes of this procedure, these substances
include commercial gasoline, synthetic gasoline, diesel fuel, jet fuel,
n-hexane, xylene(s), and toluene.
Test Atmospheres - The standard gases (Section 6.0) used to evaluate the
performance of detectors.
Test Chamber - The gas-tight or liquid-tight chamber where a detector is
exposed to ultrahigh-purity air or water and hydrocarbon test gases or
liquids.
Test System - The arrangement of instrumentation and equipment required
to perform this test procedure.
For a larger glossary of terms consistent with the field of petroleum and
hydrocarbon detection, consult the Survey of Vendors of External Petroleum
s
Leak Monitoring Devices for Use with Underground Storage Tanks Glossary (2).
49
-------
-------
-------
------- |