United States      Solid Waste And      EPA 510-S-92-801
           Environmental Protection   Emergency Response    May 1988
          • Agency        5403W
vxEPA     Development of Procedures
           To Assess The Performance
           Of External Leak Detection
           Devices

           Executive Summary - Draft
                                  ) Printed on Recycled Paper

-------

-------
                   FINAL REPORT DUE MAY,  1988


DCN 88-231-121-05
                          DEVELOPMENT OF PROCEDURES
                         TO ASSESS THE PERFORMANCE OF
                        EXTERNAL LEAK DETECTION DEVICES

                               EXECUTIVE SUMMARY

                                     DRAFT
                          EPA Contract No. 68-03-3409
                             Work Assignment No. 2
              EPA EMSL Technical Project Monitor:  J. R. Worlund
                                 Prepared for:

                       CDM Federal Programs Corporation
                 13135 Lee Jackson Memorial Highway, Suite 200
                            Fairfax, Virgina 22030
                 Deputy Project Officer:  Dr. M. S. Rosenberg
                                 Prepared by:

                              Radian Corporation
                               P. 0. Box 201088
                           Austin, Texas  78720-1088

                    Program Manager:          T. R. Blair
                    Project Director:         A. G. Eklund

                    Principal Investigators:  R. D. Achord
                                              M. R. Kienbusch
                                              D. B. Burrows
                                              G. P. Behrens
                                              W. L. Crow
                                              B. M. Eklund

-------

-------
                                   CONTENTS

     1.    Introduction.	    1
               Ob j ect ives	•    2
               Report Organization	    3

     2.    Preliminary Summary Conclusions	    4

     3.    Summary of Technical Approach	    6
               Vendor Survey & Performance Criteria	    6
               First Draft Performance Evaluation Test Procedures	    6
               Device Acquisition	   12
               Second Draft Performance Evaluation Test Procedures	   13
               Preliminary Leak Detection Performance Data	   13
               Validation of Performance Evaluation Test Procedures.....   13
               Completion of Proposed Performance Evaluation Test
               Procedures	   14

     4.    Results	   15
               Results of Specificity Testing	   16
               Results of Accuracy and Response Time Testing.	   21

References	«	   35
Appendix

     A.    Mathematical Formulas.	   36
Glossary.
                                                                          47

-------

-------
                                    SECTION 1
                                  INTRODUCTION


     Underground storage systems  comprised of tank, piping, and ancillary
components that contain hydrocarbons or other hazardous materials represent a
potential source of environmental contamination.  Proper system design, in-
stallation, operation, and maintenance, along with a leak detection program,
can minimize the detrimental effects of leaking underground storage tanks and
piping.  Devices capable of detecting petroleum hydrocarbons lost from under-
ground storage tank (UST) systems can be used inside an UST system, i.e.,
in-tank, or external to an UST system, i.e., out-of-tank.  In-tank (internal)
UST leak detectors generally detect losses with liquid-level sensors.  Out-
of-tank (external) UST detection  systems measure the presence of liquid- or
vapor-phase hydrocarbons.  Early detection of liquid- or vapor-phase petroleum
hydrocarbons allows leaking UST systems or components to be removed from
service and repaired or replaced, thereby
impairment and economic loss of product.
service and repaired or replaced, thereby minimizing both environmental
     Historically, external (out-of-tank) petroleum hydrocarbon leak and
release detection devices have not been extensively used and, therefore, are '
used primarily in conjunction with new underground storage tank installations.
However, most external leak and release detection systems can be retrofitted
at existing facilities.  Since existing UST facilities are potentially at
greater risk of failure (due to age), it is extremely important that leak and
release detection devices be used to detect releases of petroleum hydrocarbons
from UST installations.

     There are numerous commercially available external leak and release
detecting devices (for petroleum hydrocarbons) designed exclusively for use
with UST systems; however, there are no established performance specifications

-------
or certification procedures for assessing the capabilities of these devices.
The U.S. Environmental Protection Agency (EPA) is currently promulgating
regulations and guidelines that address underground storage tanks.   One aspect
of this program is to develop simple, benchmark test procedures that can be
used to assess the performance of external petroleum hydrocarbon leak and
release detection devices.  Performance test results for external leak and
release detectors in actual field use may vary from performance measured by
these benchmark tests.

     Radian Corporation,  as a subcontractor to Camp Dresser McKee under con-
tract to EPA/EMSL, is providing technical support in developing test proce-
dures for  performance evaluation of  out-of-tank  (external) leak or  release
detection  equipment  (components) for petroleum hydrocarbons.   Several  tasks
critical to this  program are complete  or currently underway.   Results  of a
survey  of  vendors of external petroleum leak  monitoring  devices for use with
underground storage  tanks have  already been published  (1).   Standard test
procedures are under development  to  allow objective  comparisons of  external
leak and release  detector performance. Draft performance methods  designed  to
be simple  benchmarks for comparison  of external  leak and release  detectors
under prescribed conditions have been submitted  to EPA (2).
 OBJECTIVES
      The overall objective of Radian's UST work is to support EPA's regulatory
 development.  Radian's key contributions are to identify parameters that
 affect leak detector performance and to develop standard test procedures for
 assessing external petroleum hydrocarbon detector performance within the
 meaningful ranges for each performance parameter.  The objective of the cur-
 rent task (Work Assignment 02) was  to develop and execute draft standard test
 procedures for performance evaluation of external detection  systems for leak
 and release monitoring.   Procedures that address specificity, accuracy, and
 response time for both vapor-phase  and liquid-phase  external detectors have
 been submitted to EPA in a draft  report  (2).

-------
     The performance evaluation test procedures submitted to EPA for review
were written as a laboratory test plan for Radian personnel involved in devel-
oping formal procedures.  Each procedure will require validation before it is
used by the public.  These performance evaluation procedures will also require
revision to provide further clarification, example calculations, and an
increased level of detail for performance by an operator.  A final collection
of performance evaluation test procedures for external leak and release detec-
tion components will be submitted in a single document at a future date for
EPA review and validation.

REPORT ORGANIZATION

     Preliminary conclusions and a summary of results from initial execution
of the performance -evaluation test procedures for a limited number of external
leak and release detection devices are given in Section  2.  Section 3 provides
a description of the technical approach for this work assignment in the con-
text of prior work.  Section A gives a preliminary discussion of the results
from initial testing of external leak and release detection equipment using
the test procedures for evaluating performance.  References, mathematical
formulas used for vapor-phase and liquid-phase detector  performance calcu-
lations  (Appendix A),  and a glossary are given at the end of this summary
report.

-------
                                   SECTION 2
                        PRELIMINARY SUMMARY CONCLUSIONS
     Test procedures used to date to assess the performance of external
petroleum hydrocarbon leak and release detection systems were developed to
provide a uniform, inexpensive, benchmark protocol for comparing parameters
such as specificity, accuracy, bias, precision, and detection time.   Although
only 13 detectors have been tested by Radian, the procedures appear to be
applicable to a wide variety of external petroleum hydrocarbon detectors.
Categorization of external detectors provided adequate separation of operating
principles for ease of performance testing.  Intermittent and continuous
detectors, separated into liquid-phase and vapor-phase categories, were easily
tested with the same procedures.

     No major problems or obstacles were reported by Radian personnel per-
forming these initial test runs, and it is likely that these procedures could
be performed easily by manufacturers or technicians knowledgeable in UST
technology.

     Preliminary test results  of the performance evaluation procedures indi-
cate that external vapor-phase and  liquid-phase detectors  respond to most
petroleum hydrocarbon components; however, specific detection systems  (espe-
cially vapor-phase  sensors) may have better  response factors for  certain com-
pound classes  (aromatics or aliphatics).   These results  also indicate  that the
250 ppmv concentration used for specificity  testing of vapor-phase  detectors
may be too low for  evaluation of some  types  of qualitative-output detectors.
All the liquid-phase detectors responded within 24 hours or less  to a  0.5-inch
thickness of petroleum product layer  floating  on water.

-------
     Liquid-phase accuracy, bias, precision, and response time results appear
to be independent of the composition of the hydrocarbon test liquid;  there-
fore, the simulated standard gasoline mixture probably provides a good repre-
sentation of a commercial product while offering the advantages of a uniform,
laboratory-generated test liquid.  In general, the liquid-phase detectors that
were tested responded in the presence of a hydrocarbon.  A larger range of
accuracy was observed for the vapor-phase detectors that were tested.

     Detection times for vapor-phase detectors ranged from less than 15
seconds to 24 minutes, with most of the responses occurring in less than 1
minute.  Vapor-phase fall time values were all under 15 minutes, and most
values were under 1 minute.  Detection times for liquid-phase detectors were
generally less than 2 hours and were less than 15 minutes for most detectors
when the hydrocarbon layer thickness was above the vendor's specifications for
lower detection limit.

     Future testing of a wider array of external leak and release detection
devices may require modification to test containers and chambers to accommo-
date all detector probe configurations.  New performance evaluation procedures
may be required to address the scientific principles of operation of future
detection equipment as technology presents new modes of external leak and
release monitoring.

     For test procedures to progress past the draft stage, validation tests
will be required to evaluate each procedure's acceptability,  and possibly, the
relationship of leak  detector performance in  the laboratory to  leak  detector
performance in the field.

-------
                                   SECTION 3
                         SUMMARY OF TECHNICAL APPROACH
VENDOR SURVEY AND PERFORMANCE CRITERIA (WORK ASSIGNMENTS 44 AND 70)

     The external petroleum hydrocarbon leak and release detection devices
that were identified in a prior study (1) were categorized by function into
four groups  (i.e., intermittent and continuous devices for both vapor-phase
and liquid-phase petroleum hydrocarbon detection).  Separate performance eval-
uation test  procedures are being developed for vapor-phase and liquid-phase
petroleum hydrocarbon detection systems, but intermittent and continuous
detection systems of each type are being tested using identical procedures.
Parameters that describe petroleum hydrocarbon leak and release detection
device performance have been  identified  (3) and are being used to develop
performance  evaluation test methods.  Parameters related to long-term perfor-
mance  (reliability), safety,  and cost were not considered in the development
of the present test procedures.

FIRST DRAFT  PERFORMANCE EVALUATION  TEST  PROCEDURES (WORK ASSIGNMENT 92)

     In March 1987.  a first draft  of example performance  evaluation procedures
was submitted to EPA for review (4).  These procedures addressed:   1)  speci-
 ficity of vapor-phase leak and release detection equipment and 2)  accuracy and
 response time for liquid-phase detectors.   These test procedures were based
 upon information from the literature, vendor and manufacturer testing, and
 engineering and scientific knowledge of the equipment to be tested.

      Prior  to writing the first draft, performance parameters were grouped
 together, where possible, to save time and resources and to decrease the

-------
variability of reproduced test procedure performances.  The performance param-
eters were combined in some cases, as shown in Table 1. so that initially a
total of 10 performance evaluation test procedures were to be written.  Proce-
dures have been written in EPA's standard format.  The generic performance
procedure outline is given in Table 2.  The following paragraphs describe each
of the outlined sections' objectives.

     The Scope and Application section of each procedure names and describes
the performance parameter(s) that the procedure was designed to measure, such
as accuracy, signal-to-noise ratio, or specificity.  The applicability and
limitations of the procedure are discussed.  Procedures are specific to either
vapor-phase or liquid-phase detectors but are usually applicable to both
intermittent and continuous external leak and release detectors.  Technical
limitations of the method are given.

     The Stunmary section provides a brief overview of operational concepts and
principles that are necessary for performing the procedure.

     The Interferences section is a detailed guide on how to detect and mini-
mize potential interferences.  Specific recommendations on how to measure the
effects of contamination are made in this section.

     The Safety section provides information and guidelines that are needed to
perform a procedure safely.  This section discusses minimum safety practices
that must be implemented to reduce the possibilities of an accident or
exposure.

     The Apparatus and Materials section describes equipment, materials, and
specifications for each.  Descriptions of test containers or chambers,  gas
rotameters, manometers, and timers are included  in this section.  Materials
such as tubing and fittings are covered also.

-------
    TABLE 1   PROPOSED PERFORMANCE EVALUATION TEST PROCEDURE MATRIX FOR
              EXTERNAL LEAK AND RELEASE DETECTION EQUIPMENT
1)  Test Procedure for Measuring Accuracy and Response Time for Liquid-Phase
    Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors

2)  Test Procedure for Measuring Accuracy and Response Time for Vapor-Phase
    Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors

3)  Test Procedure for Determining the  Specificity  of Liquid-Phase  Out-of-
    Tank Petroleum Hydrocarbon Leak and Release Detectors

4)  Test Procedure for Determining the  Specificity  of Vapor-Phase Out-of-Tank
    Petroleum Hydrocarbon Leak and Release Detectors

5)  Test Procedure for Determining the  Detection  Limit and the Signal-to-
    Noise  Ratio  of Liquid-Phase  Out-of-Tank  Petroleum Hydrocarbon Leak and
    Release Detectors

6)  Test Procedure  for Determining the  Detection  Limit and the Signal-to-
    Noise  Ratio  of Vapor-Phase Out-of-Tank Petroleum Hydrocarbon Leak and
    Release Detectors

7)  Test  Procedure  for Measuring Drift  for Liquid-Phase  Out-of-Tank Petroleum
    Hydrocarbon Leak and Release Detectors

 8)   Test  Procedure  for Measuring Drift  for Vapor-Phase  Out-of-Tank Petroleum
     Hydrocarbon Leak and Release Detectors

 9)   Test  Procedure for Determining the Effect of Potential Interferences of
     Liquid-Phase Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors

10)   Test  Procedure for Determining the Effect of Potential Interferences of
    Vapor-Phase Out-of-Tank Petroleum Hydrocarbon Leak and Release Detectors
 ^Submitted to EPA in draft form for review in March 1988.

-------
  TABLE 2.  OUTLINE FOR PERFORMANCE EVALUATION TEST PROCEDURES FOR--EXTERNAL
            LEAK AND RELEASE DETECTION EQUIPMENT
1.  Scope and Application
         Method Covers (e.g.. detection limit)
         Application (e.g., continuous gas-phase detectors)
         Limitations
         Experienced Personnel to Employ Method

2.  Summary of Method
         Brief Discussion of Method Principles
         Optional Method(s)
                        :
3.  Interferences
         Sources of Contamination
         Practices/Materials to Avoid
         QC Checks for Contamination

4.  Safety
         Compliance to OSHA Regulations
         Special Chemical Concerns
         Safety Procedures

5.  Apparatus and Materials
         Major Systems
              Testing equipment
              Functioning ranges
              Specifications
              Alternative systems
                              V-
6.  Test Gases or Liquids
         Chemicals/Gases
              Define if necessary
              Purchasing information
              Procedures to make solutions
              Storage requirements
              Expiration
              QC reagents discussed in QC section

7.  Calibration
         Per Instrument Used
              Stepwise procedures
         Alternative Procedures (include equations/define variables)
                                                                   (Continued)

-------
                             TABLE 2.   (Continued)
 8.   Quality Control
          Established QC Program
               Minimum requirements
               Record keeping
               Special procedures
               Accuracy and precision
               Blanks
               Spikes
               Ongoing system control

 9.   Procedure           ••  .
          Stepwise Guide Through Method, Including:
               Operating conditions
               Estimated responses
               Calibration
               QC
               Atypical procedures

10.   Calculations
          Discussion of All Calculations
          Definition All Variables Used

11.   Method Performance (i.e., how reliable is this method)
          Reliability of Method
               Specific ranges (if applicable)
               Operator
               Overall
          Method Accuracy

12.   Definitions
          For terms that have meanings  that are particular to the method or
          are ambiguous

13.   References
                                       10

-------
     The Reagent section describes requirements for all chemicals that are
used in a procedure.  This section is entitled "Test Gases" or "Test Liquids"
for vapor-phase or liquid-phase methods, respectively.  Requirements include
concentrations, tolerances, and quality or purity.  Storage recommendations
and necessary mixing instructions are also given.

     The Calibration section provides a stepwise guide for calibrating equip-
ment.  This equipment may include rotameters, timers, thermocouples, etc.
Frequency and acceptance criteria for calibrations are provided.  Calibration
of actual petroleum hydrocarbon detectors is only briefly discussed in this
section.  Complete calibration instructions for detectors should be provided
by the manufacturer or vendor.

     The Quality Control section describes minimum steps and acceptance cri-
teria that are necessary to establish that procedure results are reliable.
Quality control steps may include replicate analyses, record keeping, and
blank testing.  Frequency, acceptance criteria, and corrective actions for
quality control methods are listed in this section.

     The Procedure section includes detailed step-by-step instructions of how
to perform the procedure.  This section guides the technician through equip-
ment assembly, quality control procedures, calibration, testing, and expected
results.

     The Calculations section provides all the equations necessary to perform
the procedure and reduce resulting data.  Equations for quality control, data
manipulation, and data reporting are provided.

     The Method Performance section gives the technician an impression of the
procedure's reliability.  This section addresses procedure accuracy, repeata-
bility, reproducibility, range, applicability, ruggedness, testing duration,
and limitations.  (This section can be completed only after validation testing
                                      11

-------
and is incomplete in the methods that have thus far been submitted for
review.)

     The Definitions section is a list of terms and their meanings that apply
specifically to these methods.

     References provide a list of references that were used in writing the
procedure.

DEVICE ACQUISITION  (WORK ASSIGNMENT 02)

     After sets of  representative vapor-phase and liquid-phase detection
devices were identified and categorized, suppliers were contacted.  In August
1987. Radian Corporation began acquiring a limited number of external leak and
release detection devices from vendors to undergo initial execution of newly
drafted performance evaluation test procedures.  Efforts were made to acquire
several devices from each category of  sensor type established in  the prior
vendor  survey  (1).

      Several selection criteria  were  established to  guide selection of the
external  leak  and release detection devices to undergo  initial performance
testing.  The  selection criteria were:   detector type,  number of  devices  in
service,  specificity for UST  leak  detection market,  and commercial availabil-
ity.  Vendors  were  contacted; agreements with  Radian to borrow the sensor
systems for testing were executed;  and the devices were sent to  the Radian
laboratories for performance  evaluation testing.   A more detailed description
 of the device  acquisition  plan was presented  in a draft technical note to EPA
 (5) and subsequently updated in monthly progress reports.

      Devices that were acquired represented several categories of scientific
 operating principles within the larger categories of liquid-phase and vapor-
 phase leak and release detection devices.  The following types of external
 petroleum hydrocarbon liquid-phase leak and release detection devices were
                                       12

-------
acquired:  electrical conductivity sensors, interface probes, product-
permeable probes, and product-soluble sensors.  External petroleum hydrocarbon
vapor-phase leak and release detection devices were representative of the
following sensor classes:  metal oxide semiconductors, diffusion sensors,
catalytic sensors, photoionization sensors, and infrared sensors.

SECOND DRAFT PERFORMANCE EVALUATION TEST PROCEDURES (WORK ASSIGNMENT 02)

     After the draft performance evaluation test procedures were executed on a
limited basis in the Radiap laboratories, these procedures were modified from
the procedures given in the Quality Assurance Project Plan for Work Assignment
02 (6).  The second draft performance evaluation test procedures were sub-
mitted to EPA in March 1988 (2).  Modifications included minor changes to the
test apparatuses to accommodate different sensor shapes and sizes.

PRELIMINARY LEAK DETECTION PERFORMANCE DATA

     Only 13 external petroleum hydrocarbon leak and release detectors were
tested using the draft procedures.  No more than two detectors representing
any one operating principle were tested.  Therefore, the discussion of results
(Section 4) of this initial phase of testing using draft procedures is
limited.  No specific conclusions have been made about the adequacy of opera-
ting principles from performance of the draft test procedures for performance
evaluation.  Since these methods have not been validated, these data should be
treated as preliminary.

VALIDATION OF PERFORMANCE EVALUATION TEST PROCEDURES

    •Validation tests will be developed and performed in the next phase of
this program.  Individual tests for validating each of the draft performance
evaluation test procedures will be developed.  The major objective of these
validation test procedures will be to evaluate the ability of each of the
draft procedures to produce valid test data within acceptable error limits.
                                      13

-------
The results from performing the validation tests (in a future work assignment)
will be used to identify any limitations in the performance evaluation test
procedures.

     The exact details of each validation test will depend on the associated
performance test procedure.  The overall approach to be used in developing the
validation process will be the same for all cases and is patterned after the
successful approach developed by the EPA for the new source performance test
methods.  The overall approach consists of the following steps:

     •    Technical examination;
     •    Laboratory evaluation;
     •    Ruggedness testing; and
     •    Validation testing.

     Subsequent to the validation testing, final versions of performance eval-
uation  test procedures will be submitted for EPA review.  Final validation may
require establishing the  relationship  of laboratory performance to field per-
formance for network leak detection design.

COMPLETION OF PROPOSED PERFORMANCE EVALUATION  TEST PROCEDURES

     The purpose  of developing performance evaluation test  procedures  is to
provide a means of establishing  a uniform, consistent database of performance
characteristics for external  leak and  release  detection and monitoring
devices.  Over the long-term  course  of this  project,  it is  likely that other
procedures for new technologies  and  additional performance  parameters  to  those
given  in Table  1  will be identified.   As  performance evaluation procedures are
refined-and validated,  the procedures  document will be updated.
                                       14

-------
                                   SECTION 4
                                    RESULTS
     This section is a summary of results and observations made during the
preliminary execution of performance evaluation test procedures to determine
specificity, accuracy, bias, precision, and response time of external
petroleum hydrocarbon leak and release detection systems.  The test matrix
given in the project plan (6) and procedures given in the draft methods manual
(2) were followed.

     None of the results presented in this section have been validated or
approved for disclosure by EPA or any of its contractors.  The performance
evaluation results obtained for specific devices were secondary to the task of
developing test procedures.  Although all due diligence was taken in the exe-
cution of these procedures, the results summarized below must be considered
preliminary for several reasons.  First, the draft procedures performed have
not yet been scientifically validated.  Changes in the procedures may alter
the responses observed for some of the devices.  Secondly, the number of
devices tested was extremely small compared to the universe of devices on the
market.  Therefore, no correlation trends assigned to specific operating prin-
ciples can be made at this time.  Although a wide variety of operating princi-
ples, were represented, no more than two detectors from a single category were
tested.  Again, the major emphasis of this work was to develop test proce-
dures, not to test the devices.

     The following discussion is organized by the performance categories:
specificity, accuracy, and response time.  Within each performance category,
the test matrix, test procedure, and results are organized under subcategories
for vapor-phase and liquid-phase detectors.  Special efforts, have been made in
                                      15

-------
the data presentation to maintain the anonymity of devices and associated
principles of operations, as agreed upon by the participating vendors.  Radian
Corporation, and EPA Environmental Monitoring Services Laboratory in Las
Vegas, Nevada.

RESULTS OF SPECIFICITY TESTING

     The significance of specificity  (degree of response) of an external leak
detector is twofold.  If a detector's response is not specific, it may respond
to compounds in the environment that are not associated with UST petroleum
hydrocarbon releases, causing a false alarm  (false positive) situation.  On
the other hand, if the detector is highly specific  (selective), then a low
probability of detection may result  (false negative).  Therefore, one of the
most  important performance evaluation parameters  for external petroleum hydro-
carbon leak detection systems is  specificity.  Specificity  answers the ques-
tion,  "What will  the detector detect?"   Specificity  is especially important
when  the  stored product  is a single-component  hydrocarbon.

      Proper network  design, site-specific  factors,  and calibration  for  exter-
nal detectors will affect  field  specificity.   These issues  were not  addressed
 in these  laboratory  benchmark  test procedures.

Vapor-Phase Detectors  - Specificity Response

      The specificity of a vapor-phase detector was defined for the procedure
 as a system's ability to indicate either the presence or concentration of a
 pure chemical or a mixture of petroleum hydrocarbons at 250 ppmv (certified
 gas concentration in ultrahigh-purity air).  The test gases were monocomponent
 n-hexane, benzene,  toluene,  n-butane, isobutane, 2-methylpentane. and 3-
 methylpentane gas standards.   Each of the seven vapor-phase leak detection
 systems was randomly tested with each gas near 250  ppmv with 20% replicate
 measurements.  The replicate quality control  test gas was  chosen at random for
                                        16

-------
each leak detector.  Detector responses were initially measured in ultrahigh-
purity air, monitored for response to a test gas, then purged with ultrahigh-
purity air according to the test matrix given in the project test plan (6)
using the second draft test procedure (2).  Results from these tests were used
to calculate response factors for quantitative-output detectors for each test
gas, and activated versus nonactivated responses for qualitative-output detec-
tors for each test gas.  These results are summarized in Table 3.

     Three qualitative-output vapor-phase detectors (A, B, and C) failed to
respond to any of the test .gases at a concentration of 250 ppmv.  These detec-
tors were subsequently tested with n-hexane at 500 ppmv, and only Detector C
responded at the higher concentration.  These detectors had similar modes of
operation and do not represent the wide variety of qualitative-output vapor-
phase detectors on the market.

     The four quantitative-output vapor-phase detectors (D, E, F, and G)
showed specificity for the test gases, with differences in response factors.
Some detectors were specific to the chemical class of compound  (aliphatic
versus aromatic), while others showed specificity response factors that
appeared to be a combination of chemical class and other factors such as
molecular weight.  For example, Detector D was most specific  (had high
response factors) for 2-methylpentane, isobutane, n-hexane, and isopentane,
respectively.  It was less specific (had lower response factors) for n-butane
and toluene.  While Detector E's specificity was consistent,  response factors
exceeded 100% for four of the test gases.  The gas concentrations tested were
at the lower end of Detector E's operational range.  Detector F was most spe-
cific for benzene and toluene  (aromatic compounds) but demonstrated very low
specificity response factors for the other test gases consisting of aliphatic
compounds.  Detector G had accurate specificity for n-hexane  and 2-methyl-
pentane but lower response factors for the other aliphatic compounds  (n-
butane, isobutane, and isopentane) and very low response factors for benzene
and toluene  (common aromatic components in fuels).
                                      17

-------
oo
                         TABLE 3.  SUMMARY OF VAPOR-PHASE LEAK DETECTORS SPECIFICITY TEST RESULTS8
                                   (Percent Response to Test Gases)
Bact
Detector
Identification Ul
:ground
n-
1P Air Butane Isobutane
Test Gases
n- 2-Methyl-
Isopentane Hexane pentane

Benzene Toluene
Qualitative-Output Detectors
A
B
C
Quantitative-Output
D
E
F
G
NA NA NA
I
NA NA NA
NA NA NA .
Detectors
NRd 11 63
NRd 100 200
NRd 1 1
NRd 68 63
NA — b NA .
•»
NA ~b NA
NA ~° NA
57 61 65
200 200 200
6 10 12
77 96 92
..
NA NA
NA NA
NA NA
42 17
100 150
100 102
2 14
            Specificity,  the ability of a detector to indicate the  presence  or concentration of a test gas  at
             250  ppmv.  is  reported in terms of response factors calculated  as the  percent  of  the detection
             system's  response compared to the certified concentration of  the test gas  for quantitative-output
             detectors  and as "activated" (A)  or "non-activated" (NA)  for  qualitative-output  detectors.

            bNo response to either 250 ppmv or 500 ppmv of n-hexane.

            CActivated response to 500 ppmv of n-hexane but no response to 250 ppmv of  n-hexane.

            dNR = no response for quantitative-output detectors.

-------
Liquid-Phase Detectors - Specificity Response

     For liquid-phase petroleum hydrocarbon leak and release detection sys-
tems, specificity was defined in the performance evaluation test procedure as
the response to a 1.27-cm (1/2-in.) layer on water of different chemical
substances representing components or types of petroleum hydrocarbons commonly
stored in UST systems.  The test liquids used in the performance evaluation
procedure were:  n-hexane, xylene(s), toluene, commercial gasoline, a
laboratory-mixed "standard" gasoline, commercial diesel fuel, and commercial
jet fuel.  Each of six external liquid-phase petroleum hydrocarbon leak and
release detection devices was randomly tested using each test liquid with 20%
replicate measurements.  The replicate quality control test liquids were
chosen at random for each liquid-phase detector.  Detectors were measured for
background responses in tap water, then for responses in each test liquid with
a complete cleaning of the test apparatus between tests.  The matrix from the
project test plan (6) and the second draft test procedure (2) were followed.
Results from these tests were used to determine if there was a response for
specific compounds within 24 hours.

     Test results indicated five of the six liquid-phase detectors responded
to a 1.27-cm (1/2-in.) thick layer for all seven test liquids on water within
24 hours.  The sixth liquid-phase detector responded to all test liquids
within 24 hours, except commercial diesel fuel, to which it responded in 26
hours.

     The ranges of observed response times during specificity testing are
summarized by test liquid in Table 4.  The longest time until response for all
test liquids was observed for the same detector (Detector D).  Likewise, the
shortest time required until response to all test liquids was observed for a
single detector (Detector B).

     Detector D required over 16 hours to respond to n-hexane, xylene(s),
diesel fuel, and jet fuel; it responded to toluene, commercial gasoline, and
synthetic gasoline mixture in under 2 hours.  Detector B showed no specificity
                                      19

-------
  TABLE 4.   SUMMARY OF  RESPONSE  TIMES  TO TEST LIQUIDS DURING LIQUID-PHASE
             SPECIFICITY TESTING
             Test Liquid
,  Ranges  of  Observed^
 Time Until  Response'
         n-Hexane

         Xylene(s)

         Toluene

         Simulated Gasoline-

         Commercial Gasoline

         Diesel Fuel

         Jet Fuel
   <1 sec - 17.5  hr

   <1 sec - 21.5  hr

   <1 sec - 1.5 hr

   <1 sec - 1 hr

   <1 sec - 1 hr

   <1 sec - 26 hr

   <1 sec - 16.5  hr
aThe range is based on the response times for six detectors.
                                        20

-------
toward any of the test liquids, responding to all of them in less than 1
second.  Detector A required over 9 hours to respond to diesel fuel but
responded to all other test liquids in less than 20 minutes.  Detector E
responded to xylene(s), toluene, synthetic gasoline mixture, and commercial
gasoline in 2 minutes or less; it required 6 to 7 hours to respond to n-
hexane, diesel fuel, and jet fuel.  Detectors C and F responded to all test
liquids in under 10 minutes.
                               i
RESULTS OF ACCURACY AND RESPONSE TIME TESTING

     Accuracy, in general, is the degree of agreement between measured values
and the true values.  In these methods, the description of accuracy is quite
different for qualitative-output versus quantitative-output detectors.  Quali-
tative accuracy in these methods is a measure of positive responses, where
100% accurate means the detector responded positively 100% of the time.  Quan-
titative accuracy is defined as "relative accuracy," following the convention
used for continuous emission monitoring systems where 100% relative accuracy
means the maximum expected bias (at the 95% confidence level) is +100% of the
true value.  Thus, qualitative and quantitative accuracy in these methods can-
not be directly compared.  Relative accuracy has no meaning for qualitative-
output detectors; however, relative accuracy may serve as the prime comparator
for quantitative-output detectors and may be determined in the laboratory
against known test conditions or in the field versus a reference method.

     Bias is the term used for the systematic error inherent in a method, and
it can be positive or negative.  Precision is the degree of agreement between
repeated measurements of the same parameter, and it reflects random errors
that are unaffected by bias.  Neither bias nor precision are applicable to
qualitative devices, which only produce a binary ("on/off") signal.

     Detection time is the sum of lag time  (the elapsed time from a detector's
first contact with the test gas or liquid to the first detectable signal) and
rise time  (the elapsed time from a detector's first detectable signal  in
response to hydrocarbon liquid or gas to an output that is within 95%  of  full
                                      21

-------
scale or "activated").  Fall time is the elapsed time from a hydrocarbon-free
condition after exposure to hydrocarbon until a detector's output returns to
within 95% of its original baseline level or there is no detectable signal
output.

     Several parameters measured in this suite of tests describe accuracy;
others describe time-related statistics that are important to the owner of
underground storage tanks.  Accuracy answers the question, "How close is the
hydrocarbon value measured by a leak detector to the actual hydrocarbon level
present?"  Bias answers the question, "Is the detection device higher or lower
than the real value?"  Precision answers the question. "Is the test result
repeatable and is the uncertainty  in the test result large enough to adversely
influence any decision process?"   Detection time answers  the question, "How
fast will the detector signal an alarm  after it contacts  hydrocarbons?"  Fall
time answers the  question. "How long will it take after a positive response
for a  sensor placed in a  clean environment  to cease  its positive alarm sig-
nal?"  Answers to these questions  are important because they indicate whether
an alarm will occur at an acceptable performance level.   Again,  the following
test results pertain to laboratory performance  and not field performance.

Vapor-Phase Detectors - Accuracy Testing

      Accuracy  for qualitative-output vapor-phase detectors was defined in the
 test procedure as the number of positive responses expressed as a percentage
 of the total number of tests (five in this case).   For quantitative-output
 detectors,  the relative accuracy was calculated as the absolute mean diffe-
 rence between the measured value and the true value, plus the 2.5% error con-
 fidence coefficient for each series of five replicate tests, divided by the
 true value.  It is expressed as a percentage of the reference (certified gas)
 standard concentration.  Accuracy for qualitative-output detectors is neither
 conceptually nor numerically comparable to relative accuracy for quantitative-
 output detectors.
                                        22

-------
     Bias, for this method, was expressed as the signed (positive or negative)
percent difference between the average measured value for a series of five
replicate tests and the true value.  Precision was expressed in terms of the
percent coefficient of variation (CV), which is equal to the standard devia-
tion for each set of five values, divided by the mean of the set, and multi-
plied by 100.

     Each of six external vapor-phase petroleum hydrocarbon leak and release
detection systems was tested with certified gas standards of benzene and
isopentane.  Four concentration ranges (approximately 50, 250, 500, and 1000
ppmv) were tested for both compounds.  Benzene and isopentane were used as
test gases because they are common components in petroleum products and many
types of vapor-phase detectors are sensitive to them.  Concentration levels
were chosen based on preliminary information of the working concentration
ranges for total hydrocarbons in which vapor release detection might be most
appropriate.

     The test procedure (2) required an initial background test with
ultrahigh-purity air to monitor the response of a test gas; record detection
time, response value, and fall timej then repeat these steps until all of the
tests were replicated five times for each test gas concentration, according to
the matrix of 240 total tests in the project test plan (6).  Each of the rep-
licate tests was performed under equivalent test conditions.  Duplicate analy-
ses consisted of immediate repetition of test conditions for 10% of the tests
chosen at random.

     Table 5 summarizes vapor-phase accuracy, bias, precision, and response
time test results.  In general, some of the vapor-phase detectors were very
accurate, while other detectors were less accurate.  Some of the vapor-phase
detection systems were more accurate for a particular test gas, while others
showed similar accuracy for both test gases.  Some detectors were more accu-
rate at higher test gas concentrations, while others were just as accurate at
the lower test gas concentrations as they were at the higher concentrations.
                                      23

-------
                 TABLE 5.  VAPOR-PHASE ACCURACY, BIAS,  PRECISION, AND RESPONSE TIME  RESULT SUMMARY
to
	 Accuracy 	 "
Detector
Test Gas Concentration
(ppmv)
Qualitative
Accuracy
(%r
Relative
Accuracy Bias
(%)c (%)
Precision
(%)
Detection
Time
(minutes)
Fall
Time
(minutes)
Qualitative Detectors
B
B
B
B
B
B
B
B
C
C
C
C
C
C
C
C
quantitative
D
D
D
D
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
Detectors
Benzene
Benzene
Benzene
Benzene
48
240
500
990
49
253
499
991
48
240
500
990
49
253
499
991

48
240
500
990
NTd
NT
NT
0
NT
NT
0
0
NT
0
100
100
0
100
100
100

—
NLe
NL
NL
NL
NL
NL
NL
	 NL
	 NL
NL
	 NL
	 NL
	 NL
NL
	 NL
NL

44 -38
58 -55
61 -56
64 -55
NL
NL
.. NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL
NL

7
6
8
16
NDf
ND
ND
ND
ND
ND
ND
ND
ND
ND
42.40
9.60
ND
24.00
42.00
42.40

8.00
8.00
8.00
16.60
ND
ND
ND
ND
ND
ND
ND
ND
ND
ND
3.40
4.40
ND
3.20
7.80
3.20

8.00
8.00
8.00
8.00

-------
                                                           TABLE 5.    (Continued)
to
Cn

Detector3
Test Gas
Qualitative
Concentration Accuracy
(ppmv) (%)
Relative
Accuracy
Bias
Precision
Detection
Time
(minutes)
Fall
Time
. (minutes)
Quantitative Detectors (Cont.)
G
G
G
G
G
G
G
G
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
48
240
500
990 	
49 — — —
253
499 	
991 	
99
98
98
98
16
17
20
25
-99
-98
-98
-98
-13
-16
-18
-24
39
8
- 5
5
3
1
2
1
1.15
0.25
0.60
0.35
0.30
0.35
0.30
0.50
0.30
0.25
0.30
0.35
0.45
1.05
2.30
0.70
         Tletector A was not  tested because it did not respond during specificity tests.

          Qualitative  Accuracy - For qualitative-output detectors, the number of positive responses to the test gas.  expressed as a
          percentage of the number of tests (five) performed.  A value of 100% means the  detector responded appropriately for all five
          tests  in the series; 0% means the detector failed to respond during any of the  tests.

          Relative Accuracy - For quantitative-output detectors, the absolute mean difference between the measured value and the true
          value  for  the test  gas. plus the 2.5% error confidence coefficient, expressed as a percentage of the true value.   A value of
          100% for relative accuracy means that, within the 95% confidence interval, the  measurement bias can be expected not to
          exceed +100% of  the true value.

          NT = not tested  at  this concentration.
         g
          NL = not applicable because the detector's output was qualitative.

          ND = not applicable because the test gas was not detected.

         ""Detection  time and  fall time values are only for test gas responses that were above baseline responses with ultrahigh-purity
          air.

-------
                                                     TABLE 5.   (Continued)
to
Detector
Test Gas Concentration
(ppmv)
Quantitative Detectors (Cont.!
D
D
D
D '
E
E
E
.E
E
E
E
E
E
E
E
E
F
F
F
F
F
F
F
F
Isopentane
Isopentane
Isopentane
Isopentane
Benzene
Benzene
Benzene
Benzene
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
Benzene
Benzene
Benzene
Benzene
Isopentane
Isopentane
Isopentane
Isopentane
>
49
253
499
991
48
240
500
990
48
240
500
990
49
253
499
991
48
240
500
990
49
253
499
991

Qualitative Relative
Accuracy Accuracy
(%)B tt)c

46
54
29
42
687
370
150
173
687
370
150
173
1206
	 295
543
267
13
	 g
20
35
95
97
98
97
Bias
(%)

-33
-42
24
61
108
254
150
132
108
254
150
132
512
295
321
233
10
-6
-17
-34
-93
-96
-96
-97
Precision
(%)

16
17
3
- 7
224
26
0
14
224
26
0
14
91
0
43
8
3
3
4
1
28
20
25
0
Detection
Time
(minutes)

6.40
8.00
4.00
6.40
0.25B
0.45
0.50
0.90
0.25s
0.45
0.50
0.90
0.55
0.30
0.65
2.40
0.25
0.30
0.25
0.25
0.25
0.30
0.30
0.25
Fall
Tine
(minutes)

7.20
8.00
7.20
8.00
0.258
0.80
1.00
2.15
0.25s
0.80
1.00
2.15
0.30
0.70
1.70
1.85
0.40
0.55
0.30
0.95
0.30
0.25
0.25
0.20
                                                                                                            (Continued)

-------
     Vapor-phase Detectors A, B, and C were qualitative-output detectors.
Detector A was not tested because it did not respond to any of the specificity
test gases at 250 ppmv or to hexane at 500 ppmv.  Its mode of operation was
almost identical to Detector B.  Detector B did not respond during accuracy
testing to either test gas at any concentration tested.  Detector C responded
in all of the accuracy tests for 500 ppmv and 990 ppmv of benzene and for 253
ppmv, 499 ppmv. and 991 ppmv of isopentane.  (The fact that Detector C re-
sponded at 253 ppmv in these tests and did not respond at the same concen-
tration during the specificity tests indicates that this concentration may be
very close to the detector's detection limits.)

     Vapor-phase Detectors D, E, F, and G were quantitative-output detectors.
Detector D's relative accuracy ranged from 29% to 64% for benzene and isopen-
tane. - Relative accuracy  (RA) values were similar for benzene and isopentane
at all concentration levels.  Detector E demonstrated poorer relative accuracy
(RA = 150% to 1206%).  Detector F was more accurate for benzene  (RA = 9% to
35%) than for isopentane  (RA = 95% to 98%).  Detector G's relative accuracy
values were more accurate for isopentane  (RA = 16% to 25%) than  for benzene
(RA = 98% to 99%).

     Average bias values  for quantitative-output vapor-phase detectors ranged
from -99% to a high of 512%.  In general, isopentane bias values tended to
decrease as the test gas  concentration increased.  The best bias values were
-6% and 10% for Detector  F  in benzene at  240 ppmv and 48 ppmv, respectively.
Bias is not applicable to qualitative-output detectors., which produce  only a
binary  ("on/off") output  signal, and it was not  calculated for vapor-phase
Detectors A, B, and C.  For Detector D, benzene  bias was from -56% to  -38%.
For  isopentane, this detector  showed negative bias at  49 ppmv and 253  ppmv and
positive bias  at 499 ppmv and  991  ppmv.   The bias magnitudes  for both  test
gases were  similar.  For  Detector  E, bias values ranged  from  108% to 512%.
Isopentane  bias values  (RA  = 233%  to 512%) were higher than benzene  bias
values  (RA  =  108% to 254%). Benzene bias values for Detector F  were from -34%
                                       27

-------
to 10%, and the bias decreased with increasing test gas concentration!   Detec-
tor F isopentane bias results ranged closely from -97% to -93%.   As  with rela-
tive accuracy results. Detector F bias results followed an almost opposite
trend from bias results for Detector G.  Detector G bias was -99% to -98% for
benzene and -13% to -24% for isopentane.

     In general, all four quantitative-output detectors were very precise.
All but one detector showed similar precision values for both test gases.  The
poorest precision values (91% and 224%) were observed for the lowest test gas
concentrations with a single detector.  All other precision values were under
50%.  Precision is not applicable to qualitative-output detectors, which pro-
duce only a binary  ("on/off") output signal, and it was not calculated for
vapor-phase Detectors B and C.

     Detector D's precision ranged from 3% to 17% CV.  Precision values for
Detector E ranged from 0% to 224% coefficient of variation  (CV). showing less
precision at the lowest test concentrations for both gases than at higher
concentrations.  As previously noted,  test gases were at the low end of
Detector E's operating range.  Precision values for Detector F ranged from 0%
to  28%  CV.  Even though this detector's best precision value was for 991 ppmv
isopentane, benzene precision values were better than isopentane precision
values  overall.  Detector G precision  values were  from  1% to 39% CV.  Other
than a  39% benzene  precision value, values were under  10% for Detector  G.
This detector was slightly more  precise for isopentane  than for benzene.

Liquid-Phase Detectors - Accuracy Testing

     Accuracy  for qualitative-output  liquid-phase  detectors was  defined in the
test procedure as the number of  positive responses for a particular test
liquid at  a particular thickness,  expressed as a percentage of the  total num-
ber of tests  in the series  (five in this case).   For quantitative-output
 liquid-phase  detectors,  the relative accuracy was  calculated as  the absolute
mean difference between the measured value and the true value,  plus the 2.5%
                                       28

-------
error confidence coefficient for each series of five tests, divided by the
true value.  It is expressed as a percentage of the known thickness of hydro-
carbon layer on water.  As with vapor-phase detectors, accuracy, as defined in
these methods, is neither conceptually nor numerically comparable for qualita-
tive-output versus quantitative-output detectors.

     Bias is expressed as the signed (positive or negative) percent difference
between the average measured value for each series of five replicate tests and
the true value for the test liquid thickness.  Precision is expressed in terms
of the percent coefficient of variation, which is equal to the standard devi-
ation for each set of five values divided by the mean of the set and multi-
plied by 100.

     Each of six external liquid-phase petroleum hydrocarbon leak and release
detection systems was tested with three thicknesses (0.04, 0.32, and 0.64 cm)
each of simulated (laboratory-generated) gasoline mixture and commercial gaso-
line.  The simulated gasoline mixture will provide a uniform protocol for
future testing; commercial gasolines frequently have differing product compo-
sitions that may affect test results.

     The test procedure (2) required an initial background test in tap water
to'monitor the response to each test liquid and thickness as a function of
time, then repeating these steps until all of the tests were replicated five
times for each liquid product and thickness in accordance with the matrix of
180 total tests given in the project test plan (6).  Each of the replicate
tests was performed under identical conditions.

     Table 6 summarizes the test results for accuracy, bias, precision, and
response time testing for liquid-phase petroleum hydrocarbon detectors.

     Detector B was the only detector capable of providing quantitative out-
put.  Relative accuracy for this detector ranged from 1120% for the simulated
                                      29

-------
TABLE 6.  LIQUID-PHASE ACCURACY, BIAS, PRECISON, AND RESPONSE TIME "RESULTS
          SUMMARY
_
Criteria Gasoline Type
Accuracy Simulated
Commercial
Bias Simulated


Commercial


Precision Simulated

Commercial


Detection Simulated
Time
(minutes)
Commercial

Hydro-
carbon
Thickness
(cm)
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
0.04
0.32
0.64
Detector
A
100%a
100%a
100%a
100%a
100%a
100%a
c
c
c
c
c
c
d
d
d
d
d
d
6.18
0.83
0.58
1.37
1.30
0.70
B
11202?'
147%?
61?
444%?
87%?
42?
606%
114%
45%
. 102%
57%
42%
59%
12%
9%
137%
18%
0%
e
<0.02
<0.02
e
<0.02
<0.02
C
100%a
ioo%a
100%a
100%a
100%a
100%a
c
c
c
c
c
c
d
d
d
d
d
d
2.73
0.32
0.32
2.42
0.23
0.27
D
ioo%a
100%a
100%a
100%a
100%a
100%a
c
c
c
c
c
c
d
d
d
d
d
d
104.60
72.83
57.63
121.55
64.22
73.10
E
100%a
ioo%a
100%a
100%a
ioo%a
100%a
c
c
c
c
c
c
d
d
d
d

d
13.30
0.08
0.32
8.13
0.13
0.12
F
«:
100%a
100%a
0%*
100%a
100%a
c
c
c
c
c
. c
d
d
d
d

d
f
0.08
0.17.
f
0.18
0.10
                                      30

-------
                                   TABLE  6.   (Continued)

Criteria Gasoline Type

Fall Time Simulated
(minutes)

Commercial


Hydro-
carbon
Thickness
(cm)
0.04
0.32
0.6A
0.04
0.32
0.64
Detector
A

g
g
g
g
g
g
B

h
k
k
h
k
k
C

i
i
2.
±
i
i
D

g
g
g
g
g
g
E

g
g
g
g
g
g
F

j
k
k
j
k
k
Qualitative Accuracy - For qualitative-output detectors,  the  number of  positive responses to the
 test liquid, expressed as a percentage of the number of  tests (five)  performed.  A value of 100%
 means the detector responded appropriately for all  five  tests in the series; 0% means the detector
 failed to respond during any of the tests.

^Relative Accuracy - For quantitative-output detectors,  the absolute mean  difference between the
 measured value and the true value for the test liquid,  plus the 2.5% error  confidence coefficient.
 expressed ae a percentage of the true value.  A value of 100% for relative  accuracy means that.
 within the 95% confidence interval, the measurement bias can  be expected  not to exceed +100% of the
 true value.

 Bias not applicable because device is qualitative.

 Precision was not applicable because the device was qualitative.

elmmediate response when any hydrocarbon was detected.

 Not applicable because no response was recorded.

^Not applicable because the probe was destroyed during the test.

immediate when activation occurs.

^Tfo deactivation after 24 hours in water.  Will deactivate in  air.

     applicable because there was no response.
 Immediate deactivation (fall time <1 second).
                                               31

-------
gasoline mixture at 0.04 cm (1/64 in.,) to 42% at 0.64 cm (1/4 in.)  for commer-
cial gasoline.  Poor accuracy results at the 0.04 cm (1/64 in.)  thickness were
expected because the manufacturer claimed a 0.32 cm (1/8 in.) lower detection
limit.  Accuracy values at 0.32 cm (1/8 in.) and 0.64 cm (1/4 in.)  were much
better, ranging from 147% to 42%.  Detector B showed similar accuracy for both
test liquids.

     Bias for external liquid-phase petroleum hydrocarbon detectors was only
applicable to the one quantitative-output detector  (Detector B)  that was
tested.  Bias values for this detector ranged from 606% at 0.04 cm of simu-
lated gasoline to 42% at 0.64 cm of commercial gasoline.  As with the relative
accuracy values for this detector, bias values for 0.32 cm and 0.64 cm test
liquid thicknesses, which were equal  to or higher than the detector's lower
detection limit, were significantly better than the values at the 0.04 cm test
liquid thickness.

     Precision for liquid-phase  detectors was only  applicable to Detector B
because  it was the only quantitative-output  detector tested.  Precision  ranged
from  137% at 0.04 cm  (1/64  in.)  to 0% at 0.62 cm  (1/4 in.) of commercial gaso-
line.  As with relative accuracy and  bias,  precision values  were much better
for test liquid thicknesses that were above the detector's lower detection
limit.   Precision values  at the  0.04  cm thickness were  59% and  137%  for  the
synthetic gasoline mixture  and commercial  gasoline,  respectively.  Precision
values at test thicknesses  of 0.32 cm and  0.64  cm ranged from 18%  to 0%.

      Only  one of  five qualitative-output detectors  failed to respond to  test
liquids  at  each  test  thickness within 24 hours.   Detector F, whose manufac-
turer reported a detection limit of  0.32 cm (1/8  in.),  failed to  respond to
either test liquid at 0.04 cm (1/64  in.).   This same detector,  however,  did
respond in all tests  to both test liquids at the two higher test  liquid thick-
nesses.   In other words,  this detector was 0% accurate at a test  liquid thick-
ness of 0.04 cm but 100% accurate at test liquid thicknesses of 0.32 cm and
 0.64 cm.
                                       32

-------
Vapor-Phase Detectors - Response Time Testing

     Detection time and fall time results for vapor-phase detectors were given
in Table 5.  Vapor-phase detectors typically responded faster than their
liquid-phase counterparts.  The range of average detection times ranged from
less than 15 seconds to 1 hour and 24 minutes.  The vast majority of responses
occurred in under 1 minute.  Vapor-phase fall time values were smaller than
detection time values.  All fall time values were less than 1C minutes, and
most values were under 1 minute.

Liquid-Phase Detectors - Response Time Testing

     Detection time for liquid-phase detectors, as shown in Table 6, was gen-
erally 2 hours or less.  For all but one detector, responses occurred in less
than 15 minutes if the test liquid was above the detector's lower detection
limit.

     Detection times  for  liquid-phase Detectors B and F were very short  (<1 to
11  seconds) at all test liquid  thicknesses  at which these detectors responded.
These detectors did not respond at the 0.04 cm test liquid thickness, which
was below  the detection limit of both devices.

     Detectors A, C,  and  E responded at  all three thicknesses  of both  test
liquids  in under  15 minutes.  The longest  average detection  time for a set  of
five replicate tests  from this  group of  detectors was  13.3 minutes  for the
simulated  gasoline mixture at 0.04 cm.   The shortest  average detection time
was 7  seconds  for commercial  gasoline at 0.64 cm.   Detection times  were much
higher for test  liquid thicknesses at 0.04 cm (1 minute,  22  seconds to 13
minutes,  18 seconds)  than for 0.32 cm and  0.64 cm  test liquid thicknesses  (7
 seconds to 1 minute,  18 seconds).  There did not  appear to be significant
 differences between simulated versus commercial gasoline or  test liquid thick-
 nesses of 0.32 cm versus 0.64 cm.  Detector D averaged from  57 minutes to  2
                                       33

-------
hours for responses.  Response times between tests with simulated gasoline
were not significantly different from those for commercial gasoline.

     Fall time was generally not a significant factor for liquid-phase detec-
tors.  Fall time was not applicable to Detectors A. D. and E because they
required replacement of a product-soluble component that was destroyed during
contact with test liquid.  Detector C would not return to an inactivated state
when tested for fall time in water.  This detector would return to an inacti-
vated state if the probe was tested for fall time in air.  Detectors B and F
had fall times that were less than 1 second if the detectors responded to the
test liquid.  If these detectors did not respond to a test liquid, fall time
was not applicable.
                                       34

-------
                                  REFERENCES


1.   Eklund, B.M. and W.L. Crow.  Survey of Vendors of External Petroleum Leak
     Monitoring Devices for Use with Underground Storage Tanks.  EPA Contract
     No. 68-02-3994, Work Assignment 44.  March 11, 1987.

2.   Eklund, A.G., Achord,-R.D., Kienbusch, M.R., et al.  Development of
     Procedures to Assess the Performance of External Leak Detection Devices:
     Performance Test Procedures, Draft Report.  EPA Contract No. 68-03-3409,
     Work Assignment No. 02.  March 11, 1988.

3.   Eklund, B.M. and W.L. Crow.  Development of Interim Performance Criteria
     for External Petroleum Leak Monitoring Devices Used with Underground
     Storage Tanks.  Draft Technical Note.  EPA Contract No. 68-02-3994, Work
     Assignment 70.  November 14, 1986.

4.   Eklund, A.G., Kienbusch, M.R., Achord, R.D.*  et al.  Development of
     Procedures  to Assess the Performance  of External Petroleum Leak Detection
     Devices:  Draft Performance Test Procedures.  EPA  Contract No. 68-02-
     3994,  Work Assignment 92.  March,  1987.

5.   Kienbusch,  M.R. and  Randall, J.R.  Device Acquisition  Plan.  Technical
     Note.  EPA  Contract  No. 68-03-3409, Work Assignment 02, Task 02.   August
     1987.

6.   Behrens,  G.P.  and Adams, K.M.   Development  of Performance Criteria for
     External  Petroleum Leak Detection Devices:   Quality Assurance  Project
     "Plan and  Test  Plan for Laboratory Testing.   EPA Contract  No. 68-03-3409,
     Work Assignment No.  02.  Plan  for Laboratory Testing.   August  1987.
                                       35

-------
                                  APPENDIX A
                             MATHEMATICAL FORMULAS
Vapor-Phase Detectors
     Relative accuracy—The relative accuracy  (RA) of a set of data was calcu-
                          ;
lated as follows:
RA =
                      CC
                             100
                                                                 (1)
where:  V  = reference  (theoretical) value;
         d = arithmetic mean of the difference of a data set. Equation 1.1;
             and
        cc = 2.5% error confidence coefficient  (one tailed. Equation  1.2).

     Mean difference—The arithmetic mean  of  the difference  (d) of a  data  set
was calculated as follows:
                   n
          d =  -   X   d.
                                                                (1.1)
where:  d. = measured response - theoretical  response.

     Confidence  coefficient—The one-tailed 2.5% confidence coefficient (cc)
was  calculated as  follows:
           cc  = t
                 0.975
                                                                          (1.2)
                                       36

-------
where:       S = the standard deviation (n-1) of the data set; and
        t0.975 =2.5% (one-tailed) t value = 2.776 for n=5.

     Accuracy for qualitative-output detectors—For qualitative-output detec-
tors, accuracy for a particular test atmosphere at a particular concentration
was calculated as the number of positive responses expressed as a percentage
of the total number of tests (five in this case) at that concentration with
that test atmosphere, according to the following formula:

          A  = 100 x (r /5)-                                                 (2)
           q           p

where:  A  = accuracy in percent; and
        r  = number of positive responses.

     Bias—Bias is a measure of how much, on the average, the quantitative-
output detector responds high or low with respect to the true (theoretical)
response.  Bias was calculated as follows:

          Bias = 100 x  [(V  - V )/V ]                                       (3)
                          o    r   r

                                         !  5
where:  V  = the average observed value, r*  Z V.;
         0                               D i=l 1
        V. = the individual response to a test atmosphere; and
        Vr = the reference  (theoretical) value.

     Detection time—Detection time was calculated according to the  following
formula:

          ET = T  _ T                                                      (4)
                /,    1
                                       37

-------
where:  DT = elapsed detection time;
        T  = clock time when the test atmosphere was first added to the test
             chamber; and
        T  = clock time when the detector output went from an inactivated
         2   state to an activated state for a qualitative-output detector or
             from a baseline reading to 95% of stable high level output for a
             quantitative-output detector.

      Fall time—Fall time was  calculated according  to the following formula:

          ** = T2 ~ Tl

 where:  T?T  = elapsed  fall  time;
        T  = clock  time when the test  atmosphere was switched to ultrahigh-
             purity air; and
        T  = clock  time when the detector output went from  an activated state
             to an  inactivated state for a qualitative-output detector or from
              a high level reading to within 95% of stable baseline level
              output for a quantitative-output detector.

      Relative percent difference—The relative percent difference is a measure
 of variation between two observations.  It is their absolute difference
 divided by their average, expressed as a percentage, and was calculated as
 follows:

           RPD  = 200 x  l(Vl -  V2)/(V1 + V2)]                                 (6)

 where:  EPD = relative  percent  difference;
          V1 = larger value;  and
          V = smaller value.
                                        38

-------
     Correlation coefficient—The correlation coefficient was calculated as
follows:
          cc =
                V s   ' s
                V   xx    yy
where:  cc  = correlation coefficient;
                   2
        S   = n£x.  -  (Zx.)
         2Q£       J-        J-
                                                                            (7)
        x.  = i   x value; and
         x
              .th     -
        y.  = x   y value.
         x

     Coefficient of variation  (%) — The percent coefficient of variation is
equal to the standard deviation for a set of values, divided by the mean of
the set, times 100 and was calculated as follows:

          CV(%) =  (S/X) x 100                                               (8)

where:  CV = coefficient of variation;
         S = standard deviation for n values  (n-1 degrees of freedom) ; and
         X = the arithmetic mean of n values.

     Response factor  (RF) — Response factors were calculated as follows:

          RF = C /C                                  -                       (9)
                m  t

where:  C  = the vapor-phase detection system response  (ppm-v, %,  etc.); and
         m
        C  = the concentration of  the test atmosphere gas  (ppm-v,  %,  etc.).
                                       39

-------
Liquid-Phase Detectors               .                               :

     Water-miscible substances content — The fraction of water-miscible sub-
stances in commercial gasoline test product was calculated with the following
equation for a 20-mL sample:

                   (m  - m.)
where:   w = water-miscible substances content;
        m  = upper meniscus volume in mL; and
        m. = lower meniscus volume in mL.

     Area from immersing in water — The cross-sectional area of a probe was
measured by determining the volume occupied by the probe when immersed in
water.  The following equation was used to calculate the cross-sectional area:

          a. = (V. - V )/1.27                                             (11)
           Q     X    p
                                         2
where:    a, = cross-sectional area in cm ;
           a
          V. = volume between marks without probe in mL;
          V  = volume between marks with probe in inL; and
           P
        1.27 = height of column of water displaced in cm.

     Product volume — Before any testing began, a product layer of accurately
known thickness was formed on water in the test container.  The volume of test
product to add to the test container to create a desired thickness of product
was calculated with the following equation:

               t x  (a  - a,)
              _ c    d
          v = - ; -
                  1 - w
                                       40

-------
where:   v = volume of product to add to container in mL;
         t = desired product thickness in cm;
                                                           2
        a  = test container cross-sectional area (193.77 cm );
         c                                                2
        a, = estimated detector cross-sectional area in cm ; and
         w = test product water content (w = 0 for synthetic gasoline).

     Product thickness—The thickness of a test product layer on water was not
measured .but was determined from the volume of product added to the test con-
tainer, test container dimensions, and detector dimensions.  The thickness of
product on water was determined with the following formula:

          t = v/(a  - a.)                                                 (13)
                  c    d

where:   t = test product thickness in cmj
                                         3
         v = volume.of test product in cm ;
                                                      2
        a  = container cross-sectional area  (193.77 cm ); and
         c                                      2
        a, = detector cross-sectional area in cm .
         d

     Output at 95% of stable high level output—Detection time for
quantitative-output detectors was measured from the time the detector con-
tacted test product until it reached 95% of its final stable output.  The
following formula was used to calculate the 95% of stable high level output,  •
based on the stable baseline level output and the stable high level output.

          HB = BL + (HL - BL) x 0.95                                      (14)

where:  HB = output at 95% of stable high level outputj
        BL = stable baseline output; and
        HL = stable high level output.

     Output within 95% of stable baseline output—Fall time for quantitative-
output detectors was measured from the time the detector was no longer in
                                      41

-------
contact with test product until it reached 95% of its final stable baseline
output.  The following formula was used to calculate the output within 95%
stable baseline level output based on the stable baseline level output and the
stable high level output.
          OB = BL + (HL - BL) x 0.05
(15)
where:  OB = output within 95% of stable baseline output;
        BL = stable baseline output; and
        HL = stable high level output.

     Relative percent difference—The relative percent difference is a measure
of variation between two observations, neither of which is considered a refer-
ence value.  It is their absolute difference divided by their average.
expressed as a percentage, and was calculated as follows:
          RPD = 200 x  [(V, - V_)/(V.. + V.)]
(16)
where:  KPD = relative percent difference;
         V- = larger value; and
           X                                           H
         V0 = smaller value.
           £*             e

     Coefficient of variation—The coefficient  of variation  indicates the
relative degree of variation associated with  two or more values.   It is  often
reported as "Relative Standard Deviation."  The coefficient  of variation was
calculated as follows:
           CV (%)  = (S/X) x 100
                                                                           (17)
where:   CV = coefficient  of variation;
          S = standard deviation of n values (n-1  degrees of freedom);  and
          X = mean of n values.
                                       42

-------
     Accuracy—Calculation for accuracy of liquid-phase detectors depended on
the type of output that the detectors produced.  Some detectors produced
quantitative output from which the thickness of product on water was deter-
mined.  Other liquid-phase detectors only produced an alarm or other qualita-
tive signal that signified that petroleum product was present.

     Quantitative-output liquid-phase detectors—Accuracy for quantitative-
   *  output detectors is a function of systematic error (hias) and random
     error (precision).  Accuracy was calculated as the sum of the absolute
     value of the arithmetic mean of differences, |d|, and the absolute value
     of the confidence coefficient, cc.  Relative accuracy for liquid-phase
     detectors, which is the accuracy at a given hydrocarbon thickness divided
     by the hydrocarbon thickness and multiplied by 100, was calculated for
     each test product at every hydrocarbon thickness.

     Relative accuracy—The relative accuracy  (RA) of a set of data was calcu-
     lated as follows:
               Jli_L_i££i  x 100
                   r
where:  V  = reference  (theoretical) value;
         d = arithmetic mean of the difference of a data set. Equation
             18.1; and
        cc = 2.5% error confidence coefficient. Equation 18.2.

     Mean difference—The arithmetic mean of the difference  (d) of a data set
     was calculated as  follows:
          d=  -         d                                              (18.1)
where:  d. = measured response - theoretical.
                                       43

-------
     Confidence coefficient—The one-tailed 2.5% confidence coefficient (cc)
     was calculated as follows:
          cc = t      '  —                                            (18.2)
          cc   ^.975
                          n
where:       S = the standard deviation (n-1) of the data set; and
        tn 0,c = 2.5% t value = 2.776 for n=5.»
         o •
     The relative accuracy was calculated for both commercial gasoline and
     synthetic gasoline at every test product thickness.  Relative accuracy
     values were plotted against hydrocarbon thickness.

     Qualitative-output detectors — For qualitative-output detectors, accuracy
     for a particular test product at a particular thickness was calculated as
     the number of positive responses expressed as a percentage of the total
     number of tests at that thickness with that test product, according to
     the following formula:

          A  = 100 x  (r /5)                                              .
           I           P

where:  A  = accuracy in percent; and
         q
        r  = number of positive  responses.
         P
     Bias — Bias is a measure of  how  much,  on the average,  the  quantitative-
output  detector responded high or low with respect  to  the  true (theoretical)
response.  Bias was calculated as follows:

          Bias =  100 x  [(V  -  V  )/V  ]                                      (20)
                          o    r  r
                                       44

-------
                                               1   5
where:        V  = the average observed value, •=•   Z  V.;
               o                               5  i=1  i

        V. = the individual response to a test atmosphere; and
        Vr = the reference (theoretical) value.

     Bias at a particular thickness for a particular test product was the
percent difference between the average detector output for a series of tests
and the actual thickness of test product.  In this calculation, the average
detector output reading is used as the observed value (V ) in Equation 13, and
the actual test product thickness is the reference value (V ).  Bias was
reported in percent.

     Detection time—Detection time was the elapsed time between introduction
of the detector probe into the test container and when the detector reached a
positive response.  The nature of a positive response was dependent on whether
the detector had quantitative or qualitative output; however, the calculation
for detection time was the same for both types of detectors.  Detection time
was calculated according to the following formula:

          DT = T2 - TI                                                    (21)

where:  DT = elapsed detection time;
        TI = clock time when liquid was first added to test container; and
        T_ = clock time when detector output went from an inactivated state to
         £•
             an activated state for a qualitative-output detector or from a
             baseline reading to 95% of stable high level output for a
             quantitative-output detector.

     Fall time—Fall time was the amount of elapsed time between removal of
test product from the test container and when the detector  reached a negative
response.  The nature of a negative response was dependent  on whether the
                                      45

-------
detector had quantitative or qualitative output; however, the calculation for
detection time was the same for both types of detectors.  Fall time was calcu-
lated according to the following formula:
          FT = T2 - TX

where:  FT = elapsed fall time;
        T  = clock time when detector was removed from test container; and
        T  = clock time- when detector output went from an activated state to
             an inactivated, state for a qualitative-output detector or from a
             high level reading to within 95% of stable baseline level output
             for a quantitative-output detector.

      Specificity — Determination of specificity was dependent on whether a
detector's output was qualitative or quantitative.

      Quantitative-output detectors — Specificity for quantitative-output detec
tors  was defined as the ratio  of detector output, or measured thickness, to
the actual thickness of hydrocarbon test product expressed as a percentage.
The following  equation was used to calculate  specificity  for quantitative-
output  detectors:

          S =  100 x m,/t
                     Q

where:   S  =  specificity  in percent;
         m, =  detector's output  reading  in  cm;  and
          d
         t  =  hydrocarbon  thickness  in cm  (t  =  1.27  cm).

      Specificity  for qualitative-output  devices was  reported  as  "activated"  if
the detector responded  within  24 hours.  Otherwise,  specificity  was reported
as "inactivated."
                                       46

-------
                                   GLOSSARY

     Accuracy - The degree of agreement between measured values and the true
values.  Accuracy in a single measurement reflects both systematic and random
error.  Accuracy in multiple measurements estimates of the same parameter,
i.e., average accuracy, reflects systematic error, or bias.

     Activated - Refers to the state of a qualitative detector's response when
indicating the presence of hydrocarbons.

     Bias - The systematic error inherent in a method.  Bias may be positive
or negative.

     Continuous Petroleum Hydrocarbon Detection Systems (Detectors) - A class
of detectors that monitor in a constant, real-time mode, without interruption.

     Detection Time - Sum of lag time and rise time.

     Fall Time - The elapsed time after the test atmosphere is displaced by
ultrahigh-purity air until its output returns to within 95% of its original
baseline level or there is no detectable signal output.

     Intermittent Petroleum Hydrocarbon Detection Systems  (Detectors) - A
class of detectors that monitor on a. periodic basis.  Interruptions may be due
to sample transfer, sample analysis, system removal,  installation, detection
response time, etc.  A continuous detector may be operated in an intermittent
mode.

     Lag Time - The elapsed time from a detector's  first contact with the test
atmosphere or test liquid to the first detectable signal.
                                       47

-------
     Memory Effects - An effect typically encountered when tests using test
atmospheres or test liquids of different concentrations or compounds are
performed in succession.  The effect is caused by residual hydrocarbon vapors
from the previous test significantly interfering with the succeeding test.

     Non-activated - Refers to the state of a qualitative output detector's
response when indicating that no hydrocarbons are detected.

     Petroleum Hydrocarbon Detection System (Detector) - Detection system
(probes, lines, control bo*, readouts, etc.) for hydrocarbon compounds speci-
fically designed or used for vapor-phase petroleum product detection.

     Precision - The degree of agreement of repeated measurements of the same
parameter.  Precision estimates reflect random error and are not affected by
bias.

     Probe - Component  of a detection system that must come into contact with
petroleum vapor before  the vapor can be detected.

     Qualitative Response - A type of detector response that indicates only
the presence or absence of hydrocarbons without determining the hydrocarbon
concentration.

     Quantitative Response - A type of petroleum hydrocarbon leak detection
system response that quantifies the concentration of  the  hydrocarbon present.

     Relative Accuracy  - The absolute mean difference between a group of mea-
sured values and the true value, plus the  2.5% error  confidence coefficient,
divided by the true value.  Relative  accuracy  is a measure of the maximum
 (upper 95% limit)  expected bias for a series  of measurements.

     Responses - The detector's indications of the presence or  concentration
of petroleum hydrocarbons.  These  can be qualitative  or quantitative.
                                       48

-------
              Response Factor - The ratio of the detector response (ppmv,  %,  e'tc.)  to
 *        the test atmosphere or liquid concentration (ppmv,  %,  etc.).

 »             Response Time - A general term that refers  to  the more specific terms of
         lag time and rise time,  which together constitute detection time,  and fall
         time.

              Rise Time - The elapsed time from a detector's first detectable signal in
         response to hydrocarbon vapor to an output that  is  95% of full  scale.

              Specificity (gas-phase)  - The ability of a  detector to detect a
         particular chemical or class of chemicals at 250 ppm-v.

              Specificity (liquid-phase)  - The ability of a  detector to  respond to
         different substances.   For the purposes of this  procedure,  these  substances
         include commercial gasoline,  synthetic gasoline, diesel fuel, jet fuel,
         n-hexane,  xylene(s),  and toluene.

              Test Atmospheres - The standard gases (Section 6.0) used to  evaluate  the
         performance of detectors.

              Test Chamber - The gas-tight or liquid-tight chamber where a detector is
         exposed to ultrahigh-purity air or water and hydrocarbon test gases or
         liquids.

              Test System - The arrangement of instrumentation  and equipment required
         to perform this test procedure.

              For a larger glossary of terms consistent with the field of  petroleum and
         hydrocarbon detection, consult the Survey of Vendors of External  Petroleum
s
         Leak Monitoring Devices for Use with Underground Storage Tanks  Glossary (2).
                                               49

-------

-------

-------

-------