v>EPA
United States
Environmental Protection
Agency
Solid Waste And
Emergency Response/
Research And Development
5403W
EPA/530/UST-90/006
March 1990
Standard Test Procedures
for Evaluating Leak
Detection Methods
Automatic Tank
Gauging Systems
s\% ^f^t ^N.W^*'^'^'^"
-0'-X.X^ V'-'S^'X.X:
s •• ».s s; * ^ ^\^<:\-.s ^
''" '- >S";>" -\ -"-"" ^s^\>./
ss -K\Os% X f s •" s * "S >** j. S f
--v^x -5?-" -'4,^
-s.^s^- ^
**' *^xv \ ^
*< S *X»S XV. S ' *A*f y*Si>\ * \ff -f' ff
v-f**->f $ *^^x A *& ^y&?'» s»
s '•'•/•X;-' < •• «• sW''?^ sy^ s *\ *$ff*-
-------
-------
Standard Test Procedures for
Evaluating Leak Detection Methods:
Automatic Tank Gauging Systems
Final Report
U.S. Environmental Protection Agency
Office of Underground Storage Tanks
March 1990
-------
-------
FOREWORD
How to Demonstrate That Leak Detection Methods Meet EPA's Performance
Standards
The Environmental Protection Agency's (EPA's) regulations for
underground storage tanks require owners and operators to check for leaks
on a routine basis using one of a number of detection methods (40 CFR
Part 280, Subpart D). In order to ensure the effectiveness of these
methods, EPA set minimum performance standards for equipment used to
comply with the regulations. For example, after December 22, 1990, all
automatic tank gauging (ATG) systems must be capable of detecting a
0.20 gallon per hour leak rate with a probability of detection of at
least 95% and a probability of false alarm of no more than 5%. It is up
to tank owners and operators to select a method of leak detection that
has been shown to meet the relevant performance standard.
Deciding whether a method meets the standards has not been easy,
however. Until recently, manufacturers of leak detection methods have
tested their equipment using a wide variety of approaches, some more
rigorous than others. Tank owners and operators have been generally
unable to sort through the conflicting sales claims that are made based
on the results of these evaluations. To help protect consumers, some
state agencies have developed mechanisms for approving leak detection
methods. These approval procedures vary from state to state, making it
difficult for manufacturers to conclusively prove the effectiveness of
their method nationwide. The purpose of this policy is to describe the
ways that owners and operators can check that the leak detection equip-
ment or service they purchase meets the federal regulatory require-
ments. States may have additional requirements for approving the use of
leak detection methods.
EPA will not test, certify, or approve specific brands of commercial
leak detection equipment. The large number of comrnercially available
leak detection methods makes it impossible for the Agency to test all the
equipment or to review all the performance claims. Instead, the Agency
is describing how equipment should be tested to prove that it meets the
standards. Conducting this testing is left up to equipment manufacturers
in conjunction with third-party testing organizations. The manufacturer
will then provide a copy of the report showing that the method meets
EPA's performance standards. This information should be provided to
customers or regulators as requested. Tank owners and operators should
keep the evaluation results on file to satisfy EPA's record keeping
requirements.
iii
-------
, EPA recognizes three distinct ways to prove that a particular brand
of leak detection equipment meets the federal performance standards:
1. Evaluate the method using EPA's standard test procedures for
leak detection equipment;
2. Evaluate the method using a national voluntary consensus code
or standard developed by a nationally recognized association or
independent third-party testing laboratory; or,
3. Evaluate the method using a procedure deemed equivalent to an
EPA procedure by a nationally recognized association or
independent third-party testing laboratory.
The manufacturer of the leak detection method should prove that the
method meets the regulatory performance standards using one of these
three approaches. For regulatory enforcement purposes', each of the
approaches is equally satisfactory. The following sections describe the
ways to prove performance in more detail.
EPA Standard Test Procedures
EPA has developed a series of standard test procedures that cover
most of the methods commonly used for underground storage tank leak
detection. These include:
1. "Standard Test Procedures for Evaluating Leak Detection
Methods: Volumetric Tank Tightness Testing Methods"
2. "Standard Test Procedures for Evaluating Leak Detection
Methods: Nonvolumetric Tank Tightness Testing Methods"
3. "Standard Test Procedures for Evaluating Leak Detection
Methods: Automatic Tank Gauging Systems"
4. "Standard Test Procedures for Evaluating Leak Detection
Methods: Statistical Inventory Reconciliation Methods"
5. "Standard-Test Procedures for Evaluating Leak: Detection
Methods: Vapor-Phase Out-of-tank Product Detectors"
6. "Standard Test Procedures for Evaluating Leak: Detection
Methods: Liquid-Phase Out-of-tank Product Detectors"
7. "Standard Test Procedures for Evaluating Leak: Detection
Methods: Pipeline Leak Detection Systems"
Each test procedure provides an explanation of how to conduct the test,
how to perform the required calculations, and how to report the
results. The results from each standard test procedure provide the
iv
-------
information needed by tank owners and operators to determine if the
method meets the regulatory requirements.
The EPA standard test procedures may be conducted directly by equip-
ment manufacturers or may be conducted by an independent third party
under contract to the manufacturer. However, both state agencies and
tank owners typically prefer that the evaluation be carried out by an
independent third-party in order to prove compliance with the regula-
tions. Independent third-parties may include consulting firms, test
laboratories, not-for-profit research organizations, or educational
institutions with no organizational conflict of interest. In general
EPA believes that evaluations are more likely to be fair and objective
the greater the independence of the evaluating organization.
National Consensus Code or Standard
A second way for a manufacturer to prove the performance of leak
detection equipment is to evaluate the system following a national volun-
tary consensus code or standard developed by a nationally recognized
association (e.g., ASTM, ASME, ANSI, etc.). Throughout the technical
regulations for underground storage tanks, EPA has relied on national
voluntary consensus codes to help tank owners decide which brands of
equipment are acceptable. Although no such code presently exists for
evaluating leak detection equipment, one is under consideration by the
ASTM D-34-subcommittee. The Agency will accept the results of evalua-
tions conducted following this or similar codes as soon as they have been
adopted. Guidelines for developing these standards may be found in the
U.S. Department of Commerce "Procedures for the Development of Voluntary
Product Standards" (FR, Vol. 51, No. 118, June 20, 1986) and" OMB Circular
No. A-119,,
Alternative Test Procedures Deemed Equivalent to EPA's
In some cases, a specific leak detection method may not be ade-
quately covered by EPA standard test procedures or a national voluntary
consensus code, or the manufacturer may have access to data that makes it
easier to evaluate the system another way. Manufacturers who wish to
have their equipment tested according to a different plan (or who have
already done so) must have that plan developed or reviewed by a
nationally recognized association or independent third-party testing
laboratory (e.g., Factory Mutual, National Sanitation Foundation,
Underwriters Laboratory, etc.). The results should include an accredita-
tion by the association or laboratory that the conditions under which the
test was conducted were at least as rigorous as the EPA standard test
procedure. In general this will require the following:
-------
1. The evaluation tests the system both under the no-leak condi-
tion and an induced-leak condition with an induced leak rate as
close as possible to (or smaller than) the pesrformance stan-
dard. In the case of ATG systems., for example, this will mean
testing under both 0.0 gallon per hour and 0,20 gallon per hour
leak rates. In the case of ground-water monitoring, this will
mean testing with 0.0 and 0.125 inch of free product.
2. The evaluation should test the system under at least as many
different environmental conditions as the corresponding EPA
test procedure.
3. The conditions under which the system is evaluated should be at
least as rigorous as the conditions specified in the corre-
sponding EPA test procedure. For example, in the case of ATGS
testing, the test should include a temperature difference
between the delivered product and that already present in the
tank, as well as the deformation caused by filling the tank
prior to testing.
4, The evaluation results must contain the same information and
should be reported following the same general format as the EPA
standard results sheet.
5. The evaluation of the leak detection method must include
physical testing of a full-sized version of the leak detection
equipment, and a full disclosure must be made of the experi-
mental conditions under which (1) the evaluation was performed,
and (2) the method was recommended for use. An evaluation
based solely on theory or calculation is not sufficient.
vi
-------
ACKNOWLEDGMENTS
This document was written by Jairus D. Flora Jr., Ph.D., and
Karin M. Bauer for the U.S. Environmental Protection Agency's Office of
Underground Storage Tanks (EPA/OUST) under contract No. 68-01-7383. The
Work Assignment Manager for EPA/OUST was Thomas Young and the EPA/OUST
Project Officer was Vinay Kumar. Technical assistance and review were
provided by the following people:
Russ Brauksieck - New York Department of Environmental Conservation
Tom Clark - Minnesota Pollution Control Agency
Allen Martinets - Texas Water Commission
Bill Seiger - Maryland Department of Environment
American Petroleum Institute
Leak Detection Technology Association
Petroleum Equipment Institute
vii
-------
-------
CONTENTS
Foreword
)wle
1. Introduction
Acknowledgments !!!!!!!!!! «i}
1.1 Background. i
1.2 Ob ject i ves „ !!!!!!!!!! 2
1.3 Approach „ !!!!!!!!!!!!! 2
1.4 Organization of this document !!!!!!!!! 3
2. Scope and Applications !*"* 4
3. Summary „. 7
4. Safety !!!!!!!!!!!!!!!!!"** 9
5. Apparatus and Materials !!!!!!!!!!!!!! 11
5.1 Tanks !.!!!!!!!!!!!! n
5.2 Test equipment !!!!!!!!!!!!! 12
5.3 Leak simulation equipment !!!!! 12
5.4 Product !!!!!!! 13
5.5 Water sensor equipment !!!!!!! 13
5.6 Miscellaneous equipment ! 13
6. Testing Procedure !!! 15
6.1 Environmental data records .....!!!!!!!!!* 17
6.2 ATGS leak detection mode !!!!!! 17
6.3 Testing problems and solutions '.'.'.'. 23
6.4 ATGS evaluation protocol for water detection ! 23
7. Cal cu 1 ati ons 29
7.1 ATGS leak detection mode !!!!!!!!!!!!!!!!!!!!! 29
7.2 ATGS water detection mode '.'.'.'.'. 36
7.3 Supplemental calculations and data analyses
(optional) 41
7.4 Outline of calculations for alternative approach!! 51
8. Interpretation 55
8.1 Leak test function evaluation !!!!!!!!!!!!!!! 55
8.2 Water level detection function 55
8.3 Minimum water level change measurement 56
9. Reporting of Results 59
Appendices
A. Definitions and Notational Conventions A-l
B. Reporti ng Forms ! g-1
-------
-------
SECTION 1
INTRODUCTION
1.1 BACKGROUND
The regulations on underground storage tanks (40 CFR Part 280
Subpart D) specify performance standards for leak detection methods that
are internal to the tank. For automatic tank gauging (ATG) systems the
system must be capable of detecting a leak of 0.20 gallon per hour with a
probability of (at least) 95%, while operating at a false alarm rate of
5% or less..
The regulations for ATG systems require (1) that automatic product
level monitor test be able to detect a 0.20 gallon per hour leak from any
portion of the tank that routinely contains product and (2) that its
automatic inventory function meet the requirements for inventory con-
trol. That is, the equipment must be capable of:
• measuring the height of the liquid to the nearest one-eighth of
an inch.
measuring any water in the bottom of the tank at least once a
month to the nearest one-eighth of an inch.
conducting daily reconciliation of the inventory.
declaring a leak on the basis of the inventory reconciliation if
the discrepancy exceeds 1% of the flow-through plus 130 gallons
on a monthly basis.
A large number of test devices and systems are reaching the market,
but little evidence is available to support their performance claims.
Advertising literature for these systems can be confusing. Owners and
operators need to be able to determine whether a vendor's ATGS meets the
EPA performance standards. The implementing agencies (state and local
regulators) need to be able to determine whether a tank facility is fol-
lowing the UST regulations, and vendors of ATG systems need to know how
to evaluate their systems.
-------
1.2 OBJECTIVES
The objectives of this protocol are twofold. First, it provides a
procedure to test ATG systems in a consistent and objective manner.
Secondly, it allows the regulatory community and regulators to verify
compliance with regulations. This protocol provides a standard method
that can be used to estimate the performance of an ATGS. Tank owners and
operators are required to demonstrate that the method of leak detection
they use meets the EPA performance standards of operating at (no more
than) a 5% false alarm rate while having a probability of detection of
(at least) 95% to detect a leak of 0.20 gallon per hour. This demon-
stration must be made no later than December 22, 1990. The test
procedure described in this protocol is one example of,how this level of
performance can be proven. The test procedure presented here is
specific, based on reasonable choices for a number of factors.
Information about other ways to prove performance is provided in the
Foreword of this document.
It should be noted that this protocol only evaluates the leak test
function and the water sensing function of the ATGS since they are
considered the primary leak detection modes. The protocol does not
address the inventory function of the ATGS. Also, this protocol does not
address the issue of safety testing of equipment or operating
procedure. The vendor is responsible for conducting the testing
necessary to ensure that the equipment is safe for use with the type of
product being tested.
1.3 APPROACH
In general, the protocol calls for using the ATGS on a tight tank
and estimating the leak rate both under the no-leak conditions and under
induced leak conditions. The leak rate measured by the ATGS is then
compared with the induced leak rate for each test run. To estimate the
performance of the ATGS, the differences are summarized arid used with the
normal probability model for the measurement errors. The results are
applicable to tanks of the size used in the evaluation or to tanks of no
more than 25% greater capacity than the test tank.
The testing also includes conditions designed to check the system's
ability to deal with some of the more important sources of inter-
ference. A number of cycles of filling and partially emptying the tank
are incorporated to test the system's ability to deal with tank
deformation. During some of the cycles of filling the tank, the product
used to refill the tank is conditioned to have a temperature different
from that of the product in the tank. This allows a check on the
adequacy of the system's temperature compensation. Four different
nominal leak rates (including the no-leak condition) are used. This
demonstrates how closely the system can actually measure leak rates as
well as demonstrates the size of the measurement error for a tight
tank. The complete experimental design is given in Section 6 of this
document.
-------
An important function of an ATGS is its ability to detect water in
the product and to track the water level in the tank as a means of
detecting leaks when a high water table is present. Since the ATGS acts
as a continuous monitor with the tank in a normal operating condition
the relation of the product height to the height of the ground-water
level outside the tank varies,-.producing different relative pressures as
the product level changes during use. One part of most ATG systems is to
detect the possible incursion of water. In evaluating the water sensor
the minimum water level that the system can detect, and the smallest
change in water level that the system can reliably measure, are deter-
mined. The performance of the ATGS is evaluated on its ability to detect
a hole in the tank by measuring the incursion of water into the
product.
1.4 ORGANIZATION OF THIS DOCUMENT
The next section presents the scope and applications of this proto-
col. Section 3 presents an overview of the approach, and Section 4 pre-
sents a brief discussion of safety issues. The apparatus and materials
needed to conduct the evaluation are discussed in Section 5 The step-
by-step procedure is presented in Section 6. Section 7 describes the
data analysis and Section 8 provides some interpretation of results
Section 9 describes how the results are to be reported.
Two appendices are included in this document. Definitions of some
technical terms are provided in Appendix A. Appendix B presents a com-
pendium of forms: a standard reporting form for the evaluation results
a standard form for describing the operation of the ATGS, data reportina
forms, and individual test logs.
-------
-------
. SECTION 2
SCOPE AND APPLICATIONS
This document presents a standard protocol for evaluating ATG
systems. It is designed to evaluate systems that are installed in the
tank and monitor product volume changes on a continuous basis during the
test period. The protocol is designed to evaluate the leak detection
functions of an ATGS. These functions are the test mode, water detec-
tion, and water level monitoring. The evaluation will estimate the
performance of the system's test mode and compare it with the EPA
performance standards of a false alarm rate of (no more than) 5% and the
Probability of detecting a leak of 0.20 gallon per hour of (at least)
The protocol provides tests to determine the threshold of water
detection for the ATGS. In addition, the protocol tests the ability of
the water sensor to measure changes in the water level and compares the
results to the EPA performance standard of 0.125 inch. These are
evaluated over a range of a few inches in the bottom of the tank. The
threshold and height resolution of the water detector are converted to
gallons using the geometry of the tank.
Subject to the limitations listed on the Results of U.S. EPA Stan-
dard Evaluation form (see Appendix B), the results of this evaluation can
be used to prove that an ATGS meets the requirements of 40 CFR Part 280
Subpart D. The standard results form lists the test conditions. In
particular, the results reported are applicable for the stabilization
times (or longer) used in the tests and for temperature conditions no
more severe than those used in the evaluation.
-------
-------
SECTION 3
SUMMARY
The evaluation protocol for ATG systems calls for conducting the
testing on a tight tank. The organization performing the evaluation
should have evidence that the tank used for testing is tight, independent
of the system currently being tested. The evidence that the tank is
tight may consist of any of the following:
1. A tank tightness test in the 6 months preceding testing that
indicates a tight tank.
2. At least three ATGS records with a different AT6S than that
being tested within a 3-month period with inventory and test
modes indicating a tight tank.
3. A continuous vapor or liquid monitoring system installed that
indicates a tight tank.
Any of the above, verified by a tight test result on the initial test
(trial run) of the system under investigation, constitutes acceptable
evidence. This information should be reported on the data reporting form
(see Appendix B).
The protocol calls for an initial test (trial run) under stable
conditions to ensure that the equipment is working and that there are no
problems with the tank, associated piping, and the test equipment. If
the tank fails the trial run test, however, then testing should not
proceed until the problem is identified and corrected. Only if the
evaluating organization has strong evidence that the tank is tight,
should testing proceed.
The ATGS is installed in the test tank and used to measure a leak
rate under the no-leak condition and with three induced leak rates of
0.10, 0.20, and 0.30 gallon per hour. A total number of at least
24 tests is to be performed. The tank must be 50% full for half the
tests. It is refilled to about 90% to 95% full for the other 12 tests.
When filling the tank, product at least 5°F warmer than that in the tank
is used for one third of the fillings and product at least 5°F cooler
than that in the test tank is used for one third of the fillings. The
other third of the fillings uses product at the same temperature. The
ATG system's ability to track actual volume change is determined by the
-------
difference between the volume change rate measured by the test device and
the actual, induced, volume change rate for each test run. These
differences are then used to calculate the performance of the method.
Performance results are reported on the Results of U.S. EPA Standard
Evaluation form included in Appendix 8 of this document.
The ability of the system to measure water in the bottom of a tank
is tested by placing the system in a standpipe containing product. Mea-
sured amounts of water are added and the ability of the system to sense
the water at given depths is determined experimentally. These results
are also reported on the standard form in Appendix B.
8
-------
SECTION 4
SAFETY
This discussion does not purport to address all the safety consider-
ations involved in evaluating leak detection equipment and methods for
underground storage tanks. The equipment used should be tested and
determined to be safe for the products it is designed for. Each leak
detection system should have a safety protocol as part of its standard
operating procedure. This protocol should specify requirements for safe
installation and use of the device or method. This safety protocol will
be supplied by the vendor to the personnel involved in the evaluation.
In addition, each institution performing an evaluation of a leak detec-
tion device should have an institutional safety policy and procedure that
will be supplied to personnel on site and will be followed to ensure the
safety of those performing the evaluation.
Since the evaluations are performed on actual underground storage
tanks, the area around the tanks should be secured. As a minimum, the
following safety equipment should be available at the site:
Two class ABC fire extinguishers
One eyewash station (portable)
• One container (30 gallons) of spill absorbent
Two "No Smoking" signs
Personnel working at the underground storage tank facility should
wear safety glasses when working with product and steel-toed shoes when
handling heavy pipes or covers. After the safety equipment has been
placed at the site and before any work can begin, the area should be
secured with signs that read "Authorized Personnel Only" and "Keep Out."
All safety procedures appropriate for the product in the tanks
should be followed. In addition, any safety procedures required for a
particular set of test equipment should be followed.
This test procedure only addresses the issue of the system's ability
to detect leaks. It does not address testing the equipment for safety
hazards. The manufacturer needs to arrange for other testing for con-
struction standards to ensure that key safety hazards such as fire,
shock, intrinsic safety, product compatibility, etc., are considered.
The evaluating organization should check to see what safety testing has
been done before the equipment is used for testing to ensure that the
test operation will be as safe as possible.
9
-------
-------
SECTION 5
APPARATUS AND MATERIALS
5.1 TANKS
The evaluation protocol requires the use of an underground storage
tank known to be tight. A second tank or a tank truck is required to
store product for the cycles of emptying and refilling. As discussed
before, the tank should have been tested and shown to be tight by any of
the three methods described in Section 3. The tank should not have any
history of problems. In addition, the protocol calls for an initial
trial run with the test equipment under stable conditions. This test
should indicate that the tank is tight; if it does not, there may be a
problem with the tank and/or the test equipment that should be resolved
before proceeding with the evaluation.
The tank facility used for testing is required to have at least one
monitoring well. The primary reason for this is to determine the ground-
water level. The presence of a ground-water level above the bottom of
the tank would affect the leak rate in a real tank, that is, the flow of
product through an orifice. The flow would be a function of the
differential pressure between the inside and outside of the tank.
However, in a tight tank with leaks induced to a controlled container
separate from the environment, the ground-water level will not affect the
evaluation testing. Consequently, it is not necessary to require that
testing against the evaluation protocol be done in a tank entirely above
the ground-water level. The monitoring well can also be used for leak
detection at the site, either through liquid monitoring (if the ground-
water level is within 20 feet of the surface) or for vapor monitoring.
Because performance of internal tank test methods is generally worse
for large tanks, the size of the test tank is important. An 8,000-gallon
tank is recommended because this appears to be the most common tank in
use. However, testing may be done in tanks of any size. The results of
the evaluation would be applicable to all smaller tanks. The results are
also applicable to larger tanks with the restriction that the tanks be no
more than 25% larger in capacity than the test tank. That is, results
from a 6,000-gallon tank can also be applied to tanks of up to
7,500 gallons in capacity. Results from 8,000-gal tanks can be applied
to tanks up to 10,000 gallons, those from 10,000 gallons to up to
12,500 gallons, etc. If the method is intended to test larger tanks,
e.g., 20,000 gallons, it must be evaluated in a tank within 25% of that
size.
11
-------
Because the protocol calls for filling or emptying the tank a number
of times, a second tank or a tank truck is needed to hold reserve prod-
uct. A pump and associated hoses or pipes to transfer the. product from
the test tank to the reserve product tank or truck are also needed.
5.2 TEST EQUIPMENT
The equipment for each ATGS will be supplied by the vendor or manu-
facturer. Consequently, it will vary by system. In general, the ATGS
equipment will consist of some system for monitoring product volume or
level, for compensating for temperature, and for detecting and monitoring
•water in the product. It will also typically include instrumentation for
collecting and recording the data and procedures for using the data to
calculate a leak rate and interpret the result as a pass or fail for the
tank.
Since ATG systems are installed permanently and left to the tank
owner to be operated, it is recommended that the ATGS equipment being
tested be operated by the evaluating organization personnel. The ATGS
equipment is normally operated by the station owner, so the evaluating
organization should provide personnel to operate the equipment after the
customary training.
5.3 LEAK SIMULATION EQUIPMENT
The protocol calls for inducing leaks in the tank. The method of
inducing the leaks must be compatible with the leak detection system
under test. This is done by removing product from the tank at a constant
rate, measuring the amount of product removed and the time of collection,
and calculating the resulting induced leak rate. The experimental design
described in Section 6 gives the nominal leak rates that are to be
used.
A method that has been successfully used for inducing leaks in pre-
vious testing is based on a peristaltic pump. An explosion-proof motor
is used to drive a peristaltic pump head. The sizes of the pump head and
tubing are chosen to provide the desired flow rates. A variable speed
pump head is used so that different flow rates can be achieved with the
same equipment. The flow is directed through a rotameter so that the
flow can be monitored and kept constant. One end of the tubing is
inserted into the product in the tank. The other end is placed in a con-
tainer. Typically, volatile products are collected into a closed
container in an ice bath. The time of collection is monitored, the
amount of product weighed, and the volume at the temperature of the tank
is determined to obtain the induced leak rate. While 1t is not necessary
to achieve the nominal leak rates exactly, the induced leak rates should
be within ±30% of the nominal rates. The induced leak rates should be
carefully determined and recorded. The leak rates measured by the ATGS
will be compared to the induced leak rates.
12
-------
5.4 PRODUCT
The most common products in underground storage tanks are motor
fuels, particularly gasoline and diesel fuel. Analysis of tank test data
based on tanks containing a variety of products has shown no evidence of
difference in test results by type of product, if the same size tank is
considered. The only exception to this observation is that one tank test
method did produce better results when testing tanks with pure chemicals
(e.g., benzene, toluene, xylene) than when testing gasoline. This
difference was attributed to better test conditions, longer stabilization
times, and better cooperation from tank owners.
Any commercial petroleum product of grade number 2 or liqhter mav be
used for testing, depending on the availability and restrictions of the
test tanks. The choice of the product used is left to the evaluating
organization, but it must be compatible with the test equipment.
j-^ The test plan recluires some testing with addition of product at a
different temperature from that of the fuel already in the tank This
requirement is to verify that the method can accommodate the range of
temperature conditions that routinely occur. The procedure requires that
I?5V??tS.^9ln ?y the tank bei"9 filled from about ha^ full to 90% to
95% full with fuel that is 5°F warmer than the product in the tank, and
some tests using fuel 5°F cooler than the product in the tank. This
procedure requires that some method of heating and cooling the fuel be
provided, such as pumping the fuel through a heat exchanger or by placinq
heating and cooling coils in the supply tank or tank truck before the
fuel is transferred to the test tank.
5.5 WATER SENSOR EQUIPMENT
The equipment to test the water sensor consists of a vertical
cylinder with an accurately known (to ±0.001 inch) inside diameter. This
S !S5c sh°uld be larqe enough to accommodate the water sensor part of
the ATGS. Thus, it should be approximately 4 inches in diameter and 8 or
more inches high. A means of mounting the ATGS so that its water sensor
is in the same relation to the bottom of the cylinder as it would be to
the bottom of a tank is needed. In addition, a means of repeatedly
adding a small measured amount of water to the cylinder is needed This
can be accomplished by using a pipette.
5.6 MISCELLANEOUS EQUIPMENT
As noted, the test procedure requires the partial emptying and
tilling of the test tank. One or more fuel pumps of fairly larqe
capacity will be required to accomplish the filling in a reasonably short
time. Hoses or pipes will be needed for fuel transfer. In addition,
containers will be necessary to hold the product collected from the
induced leaks. A variety of tools need to be on hand for makinq the
necessary connections of equipment.
13
-------
-------
SECTION 6
TESTING PROCEDURE
The evaluation protocol for AT6 systems consists of two parts. The
first evaluates the leak detection function of the ATGS. The second
evaluates its water detection function and the system's resolution of
water sensing.
The overall performance of the ATGS is estimated by a comparison of
the system's measured (or detected) leak rates and the actual induced
leaks. Performance is measured over a variety of realistic conditions,
including temperature changes and filling effects. The range of condi-
tions does not represent the most extreme cases that might be encoun-
tered. Extreme conditions can cause any method to give misleading
results. If the system performs well overall, then it may be expected to
perform well in the field. The test procedures have been designed so
that additional analyses can be done to determine whether the system's
performance is affected by the stabilization time, temperature of added
product, the amount of product in the tank, or the size of the leak.
The test procedure introduces four main factors that may influence
the test: size of leak, amount of product in the tank, temperature dif-
ferentials, and tank deformation. An additional factor is the method's
ability to deal with ground-water level effects. This factor is
evaluated when determining the system's water sensing threshold and
resolution.
The primary consideration is the size of the leak. The system is
evaluated on its ability to measure or detect leaks of specified sizes.
If a system cannot closely measure a leak rate of 0.20 gallon per hour or
if the system demonstrates excessive variability on a tight tank, then
its performance is not adequate. The ability of the system to track the
leak rates can be compared for the different leak rates.
The second consideration is the temperature of product added to fill
a tank to the level needed for testing. Three conditions are used:
added product at the same temperature as the in-tank product, added
product that is warmer than that already in the tank, and added product
that is cooler. The temperature difference should be at least 5°F and
should be measured and recorded to the nearest degree F. The temperature
difference is needed to ensure that the system can adequately test under
15
-------
realistic conditions. The performance under the three temperature condi-
tions can be compared to determine whether these temperature conditions
have an effect on the system's performance.
The third consideration is the tank deformation caused by pressure
changes that are associated with product level changes. This considera-
tion is addressed by requiring"several empty-fill cycles. One test is
conducted at the minimum stabilization time specified by the test
method. A second test follows to test without any change in conditions
(except leak rate). Comparison of the order of the test pairs can deter-
mine if the additional stabilization improves performance. The actual
times between completing the fills and starting the tests are recorded
and reported.
The fourth consideration is the amount of product in the tank.
Since ATG systems work at different levels of product in the tank, the
required monthly test may be done at various levels. Two levels have
been chosen to represent these product levels. One is half full, which
requires the most sensitive level measurement. The other is 90% to 95%
full, which requires the most sensitive temperature compensation.
In addition to varying these factors, environmental data are
recorded to document the test conditions. These data may explain one or
more anomalous test results.
The ground-water level is a potentially important variable in tank
testing, and the system's means of dealing with it is to be documented.
A system that does not determine the ground-water level and take it into
account is not adequate. Ground-water levels are above the bottom of the
tank at approximately 25% of underground storage tank sites nationwide,
with higher proportions in coastal regions. The water sensing function
of the ATGS is used to detect leaks in the presence of a ground-water
level above the bottom of the tank. If the ground-water level is high
enough so that there is an inward pressure through most levels of product
in the tank, then water will come into the tank if there is a hole below
the ground-water level. Since an ATGS must operate at normal operating
levels of product in the tank, it uses water incursion to detect leaks if
there is a high ground-water level. This protocol evaluates two aspects
of the system's water sensing function: the minimum detectable water
level and the minimum detectable change in water level. Together, these
can be used with the dimensions of the tank to determine the ability of
the system's water sensing device to detect inflows of water at various
rates.
16
-------
6.1 ENVIRONMENTAL DATA RECORDS
In general, the evaluation protocol requires that the conditions
during the evaluation be recorded. In addition to all the testing
conditions, the following measures should be reported (see the Individual
Test Log form in Appendix B):
eimbient temperature, monitored hourly throughout each test
barometric pressure, monitored hourly throughout each test
weather conditions such as wind speed; sunny, cloudy, or
partially cloudy sky; rain; snow; etc.
ground-water level if above bottom of tank
any special conditions that might influence the results
Both normal and "unacceptable" test conditions for each system should be
described in the operating manual for the ATGS and should provide a
reference against which the existing test conditions can be compared.
The evaluation should not be done under conditions outside the vendor's
recommended operating conditions.
Pertaining to the tank and the product, the following items should
be recorded on the Individual Test Log (see Appendix B):
type of product in tank
tank volume
tank dimensions and type
amount of water in tank (before and after each test)
temperature of product in tank before filling
temperature of product added each time the tank is filled
temperature of product in tank immediately after filling
Temperature of product in tank at start of test
6.2 ATGS LEAK DETECTION MODE
The following presents the test conditions and schedule to determine
the performance of the ATGS.
6.2.1 Induced Leak Rates, Temperature Differentials, and Product
Volume
Following a trial run in the tight tank, 24 tests will be performed
according to the experimental design exemplified in Table 1. The actual
design will be randomized for each system. In Table 1, LRn- denote the
nominal leak rates and T^ denote the temperature differentials to be used
in the testing. These 24 tests evaluate the method under a variety of
conditions.
17
-------
Table 1. PRODUCT VOLUME, LEAK RATE, AND TEMPERATURE
DIFFERENTIAL TEST SCHEDULE
Trial run
Empty to 50% full (if
Fill to 90-95% full
Empty to 50% full
Fill to 90-95% full
Empty to 50% full
Fill to 90-95% full
Empty to 50% full
Fill to 90-95% full
Empty to 50% full
Fill to 90-95% full
Empty to 50% full
Fill to 90-95% full
Empty to 50% full
Test
No.
-
Pair
; NO.
-
Set
No.
-
Nominal
leak rate
(gallon
per hour)
0.00
Nominal
temperature
differential*
(degree F)
0
applicable)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
1
1
2
2
3
3
4
4
5
5
6
6
7
7
8
8
9
9
10
10
11
11
12
12
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
6
6
6
6
LRi
LR2 •
LR*
LR3
LRt
LR^
LR2
LR3
LR»
LRi
1
LR3
LR2
LR3 i
LR^
LR2
LRi
LR2
LR3
LR*
LRi
LR3
LR2
LR^
LRi
T2
T2
T2
T2
T!
T!
Tx
' T!
T3
T3
T3
T3
T2
T2
T2
T2
T!
Tt
Tx
Tj
T3
T3
T3
T3
*Note: The temperature differential is calculated as.the temperature of
the product added minus the temperature of the; product in the
tank.
18
-------
Leak Rates
The following four nominal leak rates will be induced during the
procedure:
English units . Metric units
(gallon per hour) ' (milliliters per minute)
0.00 0.00
0.10 6.3
0.20 12.6
0.30 18.9
Temperature Differentials
In addition, three nominal temperature differentials between the
temperature of the product to be added and the temperature of the product
in the tank during each fill cycle will be used. These three temperature
differentials are -5°, 0°, and +5°F (-2.8°, 0°, and +2.8°C).
Product Volumes
The tests will be run in sets of two pairs, holding the temperature
differential constant within a set of four tests but changing the leak
rate within each pair. The product volume will alternate from pair to
pair. The first pair of tests within a set will be run with the tank
filled to 90% to 95% capacity. Then the tank will be emptied to 50% full
and the second pair of tests in the set will be run.
Randomization
A total of 24 tests will be performed by inducing the 12 combina-
tions of the four leak rates (LRlt LR2, LR3, and LRJ and the three tem-
perature differentials (Tr, T2, and T3) at the two product volumes (50%
full and 90% to 95% full) as outlined in Table 1.
The randomization of the tests is achieved by randomly assigning the
nominal leak rates of 0, 0.10, 0.20 and 0.30 gallon per hour to LR1S LR2,
LR3, and LR^ and by randomly assigning the nominal temperature differen-
tials of 0°, -5°, and +5°F to T1$ T2, and T3, following the sequence of
24 tests as shown in Table 1. The organization performing the evaluation
is responsible for randomly assigning the four leak rates to LR^ LR2,
LR3, and LR% and the three temperature conditions to Tlt T2, and T3. In
addition, the evaluating organization should randomly assign the groups
of four tests to the set numbers 1 to 6, without disturbing the order of
the four tests within a set.
The vendor will install the ATGS and train the evaluating organiza-
tion to operate it. After the trial run the ATGS will be operated as it
would be in a commercial establishment. The evaluating organization will
19
-------
operate the ATGS and record its data. Note that since an ATGS operates
automatically, it is not necessary to keep the induced leak rates blind
to the operator. The operator merely starts the leak detection function
of the ATGS at the appropriate time and records the results. The random-
ization is used to balance any unusual conditions and to ensure that the
vendor does not have prior knowledge of the sequence of leak rates and
conditions to be used.
In summary, each test set consists of two pairs of tests. Each pair
of tests is performed using two induced leak rates, one induced tempera-
ture differential (temperature of product to be added •• temperature of
product in tank), and one in-tank product level. Each pair of tests
indicates the sequence in which the product volumes (in gallon per hour)
will be removed from the tank at a given product temperature
differential.
Notations! Conventions
The nominal leak rates, that is 0, 0.10, 0.20, and 0.30 gallon per
hour, after randomizing the order, are denoted by LRj.'LRj., LR3, and
LRjj. It is clear that these figures cannot be achieved exactly in the
field. Rather, these numbers are targets that should be achieved within
±30%.
The leak rates actually induced for each of the 24 tests will be
measured during each test. They will be denoted by S11( S2,».., S2tf.
These are the leak rates against which the leak rates obtained by the
vendors performing their tests will be compared.
The leak rates measured by the ATGS during each of the 24 tests will
be denoted by Llf L2,...,L2i, and correspond to the induced leak rates Slf
^ ^
O2, ... ,O2I|..
The subscripts 1,...,24 correspond to the order in which the tests
were performed (see Table 1). That is, for example, S,; arid Ls correspond
to the test results from the fifth test in the test sequence.
I
6.2.2 Testing Schedule
The first test to be done is a trial run. This test should be done
with a tight tank in a stable condition and this should be known to the
vendor. The results of the trial run will be reported along with the
other data, but are not explicitly used in the calculations estimating
the performance of the method.
There are two purposes to this trial run. One is to allow the
vendor to check out the ATGS equipment and provide instructions to the
operators before starting the evaluation. As part of this check, any
faulty equipment should be identified and repaired. A second part is to
ensure that there are no problems with the tank and the test equipment.
20
-------
Such practical field problems as leaky valves or plumbing problems should
be identified and corrected with this trial run. The results also pro-
vide current verification that the tank is tight and so provide a base-
line for the induced leak rates to be run in the later part of the
evaluation.
The testing will be performed using a randomized arrangement of
nominal leak rates, temperature differentials, and in-tank product levels
as shown in Table 1 above. The time lapse between the two tests in each
pair should be kept as short as practical. The date and time of starting
each test are to be recorded and reported in the test log. Twelve pairs
of tests will be carried out. After each pair of tests, the test proce-
dure starts anew with either emptying the tank to half full or filling it
up to 90% to 95% capacity, stabilizing, etc. The details of the testing
schedule are presented next.
Step 1: Randomly assign the nominal leak rates of 0, 0.10, 0.20, and
0.30 gallon per hour to LR1S LR2, LR3, and LRH. Also, randomly
assign the temperature differentials of 0°, -5°, and +5°F to
TU T2, and T3. Randomly assign the groups of four tests to
the 6 sets. This will be done by the evaluating organization
supervising the testing.
Step 2: Follow the vendor's instructions and install the ATGS in the
tank. Also install the leak simulation equipment in the tank
if this has not already been done, making sure that the leak
simulation equipment will not interfere with the ATGS. Perform
any calibration or operation checks needed with the
installation of the ATGS.
Step 3: Trial run. Following the test system's standard operating
procedure, fill (if needed) the tank to the recommended level
for operation in the Teak detection mode, and allow for the
stabilization period called for by the system or longer. Any
product added should be at the same temperature as that of the
in-tank product. Conduct a test on the tight tank to check out
the system (tank, plumbing, etc.) and/or the ATGS equipment.
Perform any necessary repairs or modifications identified by
the trial run.
Step 4: Empty the tank to 5Q% full if the product volume was above that
level during the trial run.
Step 5: Fill the tank to 90% to 95% capacity. Fill with product at the
temperature required by the randomized test schedule. The
temperature differential will be T2 (Table 1, Test No. 1).
Record the date and time at the completion of the fill. Allow
for the recommended stabilization period, but not longer.
Record the temperature of the product in the test tank and that of
the product added to fill the test tank. After the product has been
added to fill the test tank, record the average temperature in the test
21
-------
tank. Measuring the temperature of the product in the tank is not a
trivial task. One suggested way to measure the temperature of the
product in the tank is to use a probe with five temperature sensors
spaced to cover the diameter of the tank. The probe is inserted in the
tank (or installed permanently), and the temperature readings of those
sensors in the liquid are used, to obtain an average temperature of the
product. The temperature sensors can be spaced to represent equal
volumes or the temperatures can be weighted with the volume each
represents to obtain an average temperature for the tank.
Step 6: Continue with the system's standard operating procedure and
conduct a test on the tank, using the system's recommended test
duration. Record the date and time of starting the test,, This
test will be performed under the first nominal leak rate of the
first set in Table 1. This nominal leak rate to be induced is
L.RX.
When the first test is complete, determine and record the actual
induced leak rate, Sx, and the system's measured leak rate, LI. If
possible, also record the data used to calculate the leak rate and the
method of calculation. Save all data sheets, computer printouts, and
calculations. Record the dates and times at which the test began and
ended. Also record the length of the stabilization period. The
Individual Test Log form in Appendix B is provided for the purpose of
reporting these data and the environmental conditions for each test.
Step 7: Change the nominal leak rate to the second in the first set,
that is LR2 (see Table 1). Repeat Step 6. Note that there
will be an additional period (the time taken by the first test
and the set-up time for the second test) during which the tank
may have stabilized. When the second test of the first set is
complete, again record all results (dates and times, measured
and induced leak rates, temperatures, calculations, etc.).
Step 8: Empty the tank to 50% capacity (to within ±6 inches of the tank
midpoint). The temperature of the in-tank product will remain
unchanged.
Step 9: Change the nominal leak rate to the third in the first sest,
that is LR^. Repeat Step 6. Record all results.
Step 10: Change the nominal leak rate to the fourth in the first set,
that is LR3. Repeat Step 7. Record all results.
Step 11: Repeat Step 5. The temperature differential will be changed to
T»-
Step 12: Repeat Steps 6 through 10, using each of the four nominal leak
rates of the second set, in the order given in Table 1.
22
-------
Steps 5 through 10, which correspond to a fill and empty cycle and
one set of two pairs of tests, will be repeated until all 24 tests are
performed.
6.3 TESTING PROBLEMS AMD SOLUTIONS
Inevitably, some test runs will be inconclusive due to broken equip-
ment, spilled product used to measure the induced leak rate, or other
events that have interrupted the testing procedure. It is assumed that
in practice, the field personnel would be able to judge whether a test '
result is valid. Should a run be judged invalid during testing, then the
following rule applies.
Rule 1: The total number of tests must be at least 24. That is, if a
test is invalid, it needs to be rerun. Report the test results
as invalid together with the reason and repeat the test.
Rule 2: If equipment fails during the first run (first test of a set of
four tests) and if the time needed for fixing the problem(s) is
short (less than 20% of the stabilization time or less than
1 hour, whichever is greater), then repeat that run. Other-
wise, repeat the empty/fill cycle, the stabilization period,
etc. Record all time periods.
Note: The average stabilization time will be reported on the
results of U.S. EPA Standard Evaluation form in Appendix B. If
the delay would increase this time noticeably, then the test
sequence should be redone.
Rule 3: If equipment fails during a later test (after the first run in
a set of four has been completed successfully), and if the time
needed for fixing the problem(s) is less than 8 hours, then
repeat the test. Otherwise, repeat the whole sequence of
empty/fill cycle, stabilization, and test at the given
conditions.
6.4 AT6S EVALUATION PROTOCOL FOR WATER DETECTION
Typically the ATGS probe has a water sensor near the bottom of the
tank. A standpipe device to test the function of the water sensor con-
sists of a cylinder with an accurately known (to ±0.001 inch) inside
diameter attached to the bottom of a 4- to 6-inch diameter pipe. The
probe is mounted so that the sensor is in the same relation to the bottom
of the cylinder as to the bottom of a tank. Enough product is put into
the cylinder and pipe so that the product level sensor is high enough so
as not to interfere with the water sensor. A measured amount of water is
then added to the cylinder until the water sensor detects it, at which
time the water level is calculated and recorded. Additional measured
amounts of water are added to produce calculated level changes. The
amount of water added, the calculated level change, and the level change
23
-------
measured by the ATGS are recorded. This is done over the range of the
water sensor or 6 inches, whichever is less. When testing is complete,
the product and water are removed, separated, and the process is
repeated. The testing procedure is given in detail next.
Step 1: Install the probe temporarily in a test standpipe. The bottom
section of about 1 foot should have an accurately known (to
±0.001 inch) inside diameter. The diameter must be large enough
to accommodate the probe and must be known accurately so that
the volume of water added can be used to calculate the water
level.
Step 2: Fill the bottom section of the standpipe with the product
(typically this will require a gallon or less). Enough product
needs to be added so that the product level is high enough not
to interfere with the water sensor.
Step 3: Add water in increments to the cylinder with a pipette until the
sensor detects the presence of the water. Record the volume of
water added and the sensor reading at each increment. The
sensor reading will be zero until the first sensor response. At
that point, total the water increments and calculate the cor-
responding level, Xlf of water detected. Record all data on
page 1 of the Reporting Form for Water Sensor Evaluation Data in
Appendix B.
Step 4: Add enough water to the cylinder with a pipette to produce a
height increment, h, measured to the lesser of 1/16 inch or half
of the claimed resolution. At each increment, record the volume
of water added and the water height (denoted by W^j in Table 3
of Section 7.2) measured by the sensor. Use pages 2 to 4 as
necessary of the Reporting Form for Water Sensor Evaluation Data
in Appendix B. Repeat the incremental addition of water at
least 20 times to cover the height of about 6 inches (or, the
range limit of the sensor, if less).
Step 5: Empty the product and water from the standpipe, refill with
product (the same product can be used after separating the
water) and repeat Steps 2 and 3 20 times to obtain 20 repli-
cations. Repeat Step 4 at least 3 times or as needed to obtain
a minimum of 100 increments.
Record all data using the reporting form for ATGS water sensor data
in Appendix B. The 20 minimum detectable water levels are denoted by Xj,
0=1,..., 20. The sensor reading at the ith increment of the jth test is
denoted by
as described in Section 7.2 and Table 3.
24
-------
6.5 ALTERNATIVE EVALUATION PROCEDURE
As noted in the Foreword, EPA will accept alternative evaluation
protocols to the specific one just described. An overview of an
alternative protocol is presented next. Although it is not completely
specified, enough detail is presented so that an evaluating organization
should be able to set it up and carry it out. a«i nation
The previous sections (6.1 to 6.4) provide a test plan that can be
accomplished in about three calendar weeks. The approach described there
requires a tank that can be fully devoted to testing, which may be a
difficult requirement. The following alternative approach uses
in-service tanks. Only a limited amount of work is required that would
prohibit using the tank for dispensing product.
The alternative approach consists of installing the AT6S in a number
of tanks. Since the ATGS operates automatically, it can be programmed to
perform a test whenever the tank is out of service for a long enough
period, typically each night. With several available tanks, a large set
of tests could be performed in a relatively short time. By selecting
tanks in different climates or observing tanks over the change of sea-
sons, tests can be performed under a wide variety of conditions. Thus
with little expenditure of effort, a large data base of test results on
tight tanks can be obtained readily.
The alternative approach will provide test data under a variety of
actual conditions. In selecting the sites and times for the data collec-
tion, the evaluating organization should attempt to obtain a wide variety
of temperature conditions and to conduct the tests at a wide variety of
product levels in the tank as well as a variety of times after the tank
receives a product delivery. This alternative approach will produce data
under conditions as actually observed in the field. The primary dif-
ference between the standard and alternative procedures is how the test
conditions are attained. Both approaches attempt to conduct the evalu-
ation testing under conditions representative of the real world. The
standard approach does this by controlling the test conditions, while the
alternative tests under a variety of situations and records the test
conditions.
Next, the data base of ATGS test results on tight tanks needs to be
supplemented with a limited number of tests using an induced leak. This
is to demonstrate that the system can track an induced leak adequately,
that is, that it will respond to and identify a loss of product from the
tank of the magnitude specified in the EPA performance standard. The
combined data sets can then be analyzed to estimate the performance of
the ATGS. If the resulting performance estimate meets the performance
standard for an ATGS, that would constitute demonstration that the system
meets the EPA standard.
This alternative approach will result in a large number of tests on
tight tanks, and relatively few tests under induced leak rate condi-
tions. A suggested sample size is 100 tight tank tests and 10 induced
25
-------
leak rate tests. Larger numbers of either type of test can be used. It
should be easy to obtain the tight tank tests, however, some work will be
needed to prepare the data base, recording the ancillary data. It will
also be necessary to exclude some tests, for example those that were
started, but had a delivery or dispensing operation during the test
period thus invalidating the test.
The following steps provide an outline of this method of evaluation.
Step 1: Identify a number of tanks for installation of the AT6 sys-
tems. These tanks should be known to be tight, by meeting one
of the criteria described in Section 3. The tanks can be of
varying sizes, but the sizes used will limit the applicability
of the results. The tanks should be at several sites, with a
suggested minimum of 5 different sites and 10 different tanks.
i
Step 2: Install identical ATG systems in the tanks. Arrange to collect
and record ancillary data to document the test conditions. The
data needed are:
the average in-tank product temperature prior to a
delivery.
• the time and date of each delivery.
the average in-tank product temperature immediately after
a delivery.
the amount of product added at. each delivery.
• the date, time, and results of each test.
• the product level when the test is run.
the tank size, type of tank, product contained, etc., (see
the Individual Test Log for a form to record these data).
Step 3: Conduct tests in each tank for at least a two-week period.
Tests should be run approximately nightly or as frequently as
practical with the tank's use. Report the starting and ending
dates of the test period. Record the test result along with
the data listed in Step 2. The data above define the condi-
tions of each test in terms of the time since the last fill
(stabilization time), the product level, and the difference
between the temperature of the product added and that of the
product in the tank. Report all test results, even if some
tests must be discarded because of product delivery or dis-
pensing during the scheduled test period. Identify and report
the reason for discarding any test data on the test log.
26
-------
Step 4: Conduct tests with an induced leak at the rate between 0.10 and
0.20 gallon per hour. These induced leak tests will generally
require a person on site to monitor the induced leak rates and
measure the rates actually achieved. A minimum of 10 such
tests is suggested, with some conducted shortly after a fill
with a nearly full tank, and others conducted when the tank is
about half full. The induced leak tests should be conducted on
the largest available tanks to demonstrate the performance on
the largest tank that the ATGS is intended for.
Step 5: At some time during the evaluation period, evaluate the per-
formance of the water sensor function. This can be done at a
separate site and does not require a tank. Follow the
procedure described in Section 6.4.
Step 6: Using the resulting data, analyze the differences between the
leak rate measured by the ATGS and the induced leak rate
achieved (zero for the many tests on tight tanks) for teach
test to estimate the performance.
The data base can be used to investigate the relationship of the
error size (the leak rate differences) to each of the variables measured
for the tests. These include tank size, length of stabilization time
temperature differential, product level, and presence of induced leaks
Multiple regression techniques can be used for these analyses, most of
which would fall under the category of optional analyses. However, the
data should be analyzed with the two groups of tight tank tests and
induced leak rate tests separately to demonstrate that the system can
determine the leak rates. Otherwise, it would be possible to have such a
large number of tight tests that small errors on those would obscure
large errors on the small number of induced leak rates tests. An outline
of the data analysis approach is given in Section 7.4.
27
-------
-------
SECTION 7
CALCULATIONS
From the results obtained after all testing is completed, a series
of calculations will be performed to evaluate the system's performance.
The evaluation of the ATGS in its leak detection mode is presented
first. These calculations compare the system's measured leak rate with
the induced leak rate under a variety of experimental conditions. The
probability of false alarm and the probability of detection are estimated
using the difference between these two numbers. If the overall perfor-
mance of the ATGS is satisfactory, analysis and reporting of results
could end at this point. However, the experimental design has been con-
structed so that the effects of stabilization time, product level and
temperature can be tested to provide additional information to the
vendor.
A separate section (Section 7.2) presents the calculations to esti-
mate the minimum water level (detection threshold) and the minimum water
level change that the sensor can detect.
7.1 ATGS LEAK DETECTION MODE
After all tests are performed according to the schedule outlined in
Section 6, a total of at least n = 24 pairs (4 leak rates x 3 temperature
differentials x 2 product volumes) of measured leak rates and induced
leak rates will be available. These data form the basis for the perfor-
mance evaluation of the system. The measured leak rates are denoted by
Li,...tL2ti and the associated induced leak rates by Slt...,S2u. These
leak rates are numbered in chronological order. Table 2 summarizes the
notation used throughout this protocol, using the example test plan of
Table 1.
7.1.1 Basic Statistics
The n = 24 pairs of data are used to calculate the mean squared
error, MSE, the bias, and the variance of the method as follows.
29
-------
Table 2. NOTATION SUMMARY
Test
No.
1
2
3
4
*
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Pair
No.
1
1
2
2
3
3
4
4
5
5
6
6
7
7
8
8
9
9
10
10
11
11
12
12
Set
No.
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
6
6
6
6
Nominal
temperature
differential
(degree F)
T2
T2
T2
T2
T!
TI
T!
TI
T3
T3
T3
T3
T2
T2
T2
T2
T!
TI
T!
TI
T3
T3
T3
T3
Nominal
leak 'rate
(gallon
per hour)
LR
LR2
LR,,
LR3
LR,
LR,,
LR2
LR3
• LR,,
LR,
LR3
LR2
LR3
LRH
LR2
LRi
LR2
LR3
LR,,
LRi
LR3
LR2
LR,,
«,
Induced
leak rate
(gallon
per hour)
Si
S2
S3
s*
ss
Se
S7
Sa
S9
Sio
Si,
S12
Sis
Si*
SIB
Sis
SIT
SIB
Sis
S20
Szi
S22
S23
s.
Measured
leak rate
(gallon
(per hour)
LI
L2
L3
Ll»
L5
L6
L7
L8
L9
Lio
LH
Ll2
Ll3
Lm
LIB
Lie
LIT
LIB;
Ll9
L20
Lzi
L22
L23
Absolute
leak rate
difference
L-S|
(gallon
per hour)
d
d2
d3
d*
ds
d6
d7
d8
d9
dio
dn
d12
di3
di*
dis
die
d17
die
dig
d20
d21
d22
d23
d2i»
30
i
-------
Mean Squared Error, MSE
24
MSE -2
1-1
where Li is the measured leak rate obtained from the 1th test at the cor
responding induced leak rate, S^ with 1-1, ..., 24.
Bias, B
24
1-1
The bias, B, is the average difference between measured and induced
leak rates over the number of tests. It is a measure of the accuracy of
the system and can be either positive or negative.
Variance and Standard Deviation
The variance is obtained as follows:
24
Variance = ^ [(L. - S^ - B]2/23
Denote by SD the square root of the variance. This is the standard
deviation,
NOTE: It is recommended that the differences between the measured and
induced leak rates be plotted against the time or the order in which they
were performed. This would allow one to detect any patterns that might
exist, indicating potentially larger differences in the results from the
first test of each set of tests, among the three temperature differen-
tials, or between in- tank product levels. This could suggest that the
system calls for an inadequate stabilization time after filling, that the
system does not properly compensate for temperature differences between
in-tank product and product to be added, or that the system is influenced
by the product level. (See Sections 7.3.3, 7.3.4, and 7.3.5 for
appropriate statistical tests.)
Test for Zero Bias
To test whether the method is accurate—that is, the bias is zero—
the following test on the bias calculated above is performed.
31
-------
, Compute the t-statist1c
tB = \/24~ B/SD
From the t-table in Appendix A, obtain the critical value cor-
responding to a t with (24-1) = 23 degrees of freedom and a two-sided 5%
significance level. This value is 2.07. Note: If more than 24 tests are
done, replace 24 with the number of tests, n, throughout. A larger num-
ber will change the t-value.
Compare the absolute value of tB, abs(tB), to 2.07 (or to the
appropriate t-value if more than 24 tests were performed)., If abs(tg) is
less than 2.07, conclude that the bias is not statistically different
from zero, that is, the bias is negligible. Otherwise,, conclude that the
bias is statistically significant.
The effect of a statistically significant bias on the calculations
of the probability of false alarm and the probability of detection is
clearly visible when comparing Figures A-l and A-2 in Appendix A.
i
7.1.2 False Alarm Rate, P(FA)
The normal probability model is assumed for the errors in the
measured leak rates. Using this model, together with the statistics
estimated above, allows for the calculation of the predicted false alarm
rate and the probability of detection of a leak of 0.20 gallon per hour.
The vendor will supply the criterion (threshold) for interpreting
the results of the AT6S test function. Typically, the leak rate measured
by the ATGS is compared to that threshold and the results interpreted as
indicating a leak if the measured leak rate exceeds the threshold.
Denote the system's criterion or threshold by C. The false alarm rate or
probability of false alarm, P(FA), is the probability that the measured
leak rate exceeds the threshold C when the tank is tight. Note that by
convention, all leak rates representing volume losses from the tank are
treated as positive.
P(FA) is calculated by one of two methods, depending on whether the
bias is statistically significantly different from zero.
False Alarm Rate With Negligible Bias
In the case of a nonsignificant bias (Section 7.1.1), compute the
t-statistic
tx = C/SD
32
-------
where SD is the standard deviation calculated above and C is the system's
threshold. Using the notational convention for leak rates, C is posi-
tive. P(FA) is then obtained from the t-table, using 23 degrees of free-
dom. P(FA) is the area under the curve to the right of the calculated
value ti.
In general, t-tables are constructed to give a percentile t cor
responding to a given number of degrees of freedom, df, and a preassianed
3r^ aor'a]Pha» under the curve, to the right of t. (see Figure 1 below
and Table A-l in Appendix A). For example, with 23 Segrees of freedom
and a = 0.05 (equivalent to a P(FA) of 5%), ta = 1.714.
Figure 1. Student's t-D1stributton Function.
In our case, however, we need to determine the area under the curve
to the right of the calculated percentile, tlf with a given number of
degrees of freedom. This can be done by interpolating between the two
areas corresponding to the two percentiles in Table A-l on either side of
the calculated statistic, tj. The approach is illustrated next.
Suppose that the calculated tx = 1.85 and has 23 degrees of
freedom, l-rom Table A-l, obtain the following percentiles at df = 23:
1.714
1.85
2.069
a (alpha)
0.05
X to be determined
0.025
Calculate X by linearly interpolating between 1.714 and 2.069 correspond-
ing to 0.05 and 0.025, respectively.
33
-------
°'05 - ' X (1'714 ' 1-85) = °-
Thus the probability of false alarm corresponding to a tj of 1.85 would
be 4%.
A more accurate approach would be to use a statistical software
package (e.g., SAS or SYSTAT) to calculate the probability. Another
method would be to use a nomograph of Student's t such as the one given
by Lloyd S. Nelson in Technical Aids, 1986, American Society for Quality
Control .
'
False Alarm Rate With Significant Bias
The computations are similar to those in the case of a nonsignifi-
cant bias with the exception that the bias is included in the calcula-
tions, as shown next. Compute the t-statistic
tz = (C-B)/SD
P(FA) is then obtained by interpolating from the t-table, using 23
degrees of freedom. P(FA) is the area under the curve to the right of the
calculated value t2. (Recall that C is positive, but the bias could be
either positive or negative.)
7.1.3 Probability of Detecting a Leak Rate of 0.20 gallon per hour, P(D)
The probability of detecting a leak rate of 0.20 gallon per hour,
P(D), is the probability that the measured leak rate exceeds C when the
true mean leak rate is 0.20 gallon per hour. As for P(FA), one of two
methods is used in the computation of P(D), depending on whether the bias
is statistically significantly different from zero.
P(D) With Negligible Bias
In the case of a nonsignificant bias— that is, the bias is zero-
compute the t-statistic
t3 = (C-0.20)/SD
Next, using the t-table at 23 degrees of freedom, determine the area
under the curve to the right of t3. The resulting number will be P(D).
34
-------
P(D) With Significant Bias
The procedure is similar to the one just described, except that B is
introduced in the calculations as shown below. Compute the t-statistic
t,/= (C-B-0.20)/SD
Next, using the t-table at 23 degrees of freedom, determine the area
under the curve to the right of ti,. The resulting number will be P(D).
7.1.4 OTHER REPORTED CALCULATIONS
This section describes other calculations needed to complete the
Results of U.S. EPA Standard Evaluation form (Appendix B). Most of these
calculations are straightforward and are described here to provide
complete instructions for the use of the results form.
Size of Tank
The evaluation results are applicable to tanks up to 50% larger
capacity than the test tank and to all smaller tanks. Multiply the
volume of the test tank by 1.50. Round this number to the nearest 100
gallons and report the result on page 1 of the results form.
Maximum Allowable Temperature Difference
Calculate the standard deviation of the 6 temperature differences
actually achieved during testing (these 6 tests are the first in each of
the 6 sets). Multiply this number by the factor ± 1.5 and report the
result as the temperature range on the limitations section of the results
form.
The nominal temperature difference of 5°F used in the design was
obtained from data collected on the national survey (Flora, J. D., Jr.
and J. E. Pelkey, "Typical Tank Testing Conditions," EPA Contract
No. 68-01-7383, Work Assignment 22, Task 13, Final Report, December
1988). This difference was approximately the standard deviation of the
temperature differences observed in the tank tests conducted during the
national survey. The factor 1.5 is a combination of two effects. One
effect results from scaling up the standard deviation of the design
temperature differences to 5°F. The second effect results from using the
rule that about 80% of the temperature differences on tank tests are
expected to be within ± 1.282 times the standard deviation.
35
-------
Average Waiting Time After Filling
Calculate the average of the time intervals between the end of the
le>and fuart °f-tl?e test for the 6 tests that started
after the specified waiting time (first test in each set}
lip1"6 ^ 6 teS? *r* d°ne 1"«d1«te^ ««er the filing, use
all such tests. However, do not use the time to the start of the
JJSpTTJj ^IL™ a Set as.th1s would 9ive a »1slead1ng waiting
time.) Report this average time as the waiting time after adding product
on the results form. Note: The median may be used as the average
instead of the mean if there are atypical waiting times.
Average Data Collection Time Per Test
durat1on of the data collection phase of the tests to
« aDerdg? ?.a*a f?llect1on «« for the total number (at least
24) of tests. Report this time as the average data collection time per
7.2 ATGS WATER DETECTION MODE
thp Jn?mnSramf TJf111 5e eft1mated for the water detection sensor:
dP?pJiyi!T. fj?ablei?at!r level °r threshold t^t the sensor can
determine, and the smallest change in water level that the device can
sXn^H ?e?e :?SUlŁS w111 also be reP°rted on the Results of 5?S??PA
Standard Evaluation form in Appendix B.
7.2.1 Minimum Detectable Water Level
The data obtained consist of 20 replications of a determination of
the minimum detectable water level (see test schedule, Section 6 4)
S level d50SLS i4'j;iv20\are used to esti;ate
water level, or threshold, that can be detected reliably.
Step 1: Calculate the mean, X, of the 20 observations-
20 !
X =
X.,/20
Step 2: Calculate the standard deviation, SD, of the 20 observations:
SD =
20
_. 2
- x)
20-1
1/2
36
-------
Step 3: From a table of tolerance coefficients, K, for one-sided normal
tolerance intervals with a 95% probability level and a 95%
coverage, obtain K for a sample size of 20. This coefficient
is K - 2.396. (Reference: Lieberman, Gerald F. 1958.
"Tables for One-Sided Statistical Tolerance Limits." industrial
Quality Control. Vol. XIV, No. 10.)
Step 4: Calculate the upper tolerance limit, TL, for 95% coverage with
a tolerance coefficient of 95%:
TL = X + K SD,
or
TL = X + 2.396 SD
TL estimates the minimum level of water that the sensor can
detect. That is, with 95% confidence, the ATGS should detect water at
least 95% of the time when the water depth in the tank reaches TL.
7.2.2 Minimum Mater Level Change
The following statistical procedure provides a means of estimating
the minimum water level change that the water sensor can detect, based on
the schedule outlined in Section 6.4.
Denote by W^j the sensor reading (in inches) at the jth replicate
and the ith increment (i=l,...,nj, with nj being 20 or more in each
replicate). Note that the number of steps in each replicate need not be
the same, so the sample sizes are denoted by n,-.
Denote by h (measured to the lesser of 1/16 inch or half the claimed
resolution) the level change induced at each increment. Let m (greater
than or equal to 3) be the number of replicates.
Step 1: Calculate the differences between consecutive sensor read-
ings. The first increment will be W1 j-X-j^ for the first
replicate (j=l); more generally, W-^j-Xj, for the jth
replicate. The second increment will be W2 1-W1 ± for the
first replicate; more generally, w^j-W^j for the jth
replicate, etc.
37
-------
Step 2: Calculate the difference, at each incremental step, between h,
the .level change induced during testing, and the difference
obtained in Step 1. Denote these differences by di it where i
• v J
and j represent increment and replicate numbers, respec-
tively. Table 3 below summarizes the notations.
Table 3. NOTATION SUMMARY FOR WATER SENSOR READINGS
AT THE jth REPLICATE
Calculated
level
change
Increment (inch)
No. A
1 + h
2 + h
3 + h
* «
• •
n-j ' + h
Sensor
reading
(inch)
B
H1.J
W2,j
W3,j
*
o "
*
V
Measured Increment
sensor difference
increment calculated-meas.
(inch) (inch)
C C-A
Wi -f-X.:* di 1
i,j j i,j
W2 •«- Wi ,- dj» ^
^•jj ^iJ '••**}
Wo -J~"9 •? Q"[ •?
J»J ^»J «*»J
• a
• «
! .
Wn ,-Wn 1 4 dri ,•
n^»j n,— i,j n.-,j
\J \J J
* X,- is the water level (inches) detected for the first time
by the sensor during the jth replication of the test.
Note that the first sensor reading, Xj, may vary from replicate to repli-
cate, so that the number of differences d^ j will also vary. Let n,- be
the number of increments necessary during replicate j.
Step 3: Calculate the average, Dn-, of the differences cL .:, i=l,...,nn-,
J ' »v,l J
separately for each replicate j, j=l,...,20.
38
-------
Step 4: Calculate the variance of the differences d^ _-, i=l,...,n
separately for each replicate j, j=l,...,m.
Step 5: Calculate the pooled variance, Var_, of the m variances
Varlt...,Varm. p
(n,-l) Var, + ••• + (n -1) Var
Var m "
P P.
I (nrl)
3-1 J
Step 6: Calculate the pooled standard deviation, SD_.
SDp = TvaTp"
Step 7: From a table of tolerance factors, K, for two-sided tolerance
intervals with 95% probability and 95% coverage, obtain K for
(Zn-j-m) degrees of freedom. For the suggested sample size, the
value corresponding to a total of 100 degrees of freedom
(K = 2.233) can be used unless the number of differences
obtained is less than 100. (Reference: CRC Handbook of Tables
for Probability and Statistics. 1966. William H. Beyer (ed.).
pp. 31-35. The Chemical Rubber Company.)
Step 8: Calculate the minimum water level change, MLC, that the sensor
can detect.
MLC - K SD
or p
MLC = 2.233 SD
The result, MLC, is an estimate of the minimum water level change
that the water sensor can detect.
7.2.3 Time to Detect a 0.20-Gallon per Hour Water Incursion (Optional)
The minimum detectable water level and the minimum detectable change
can be used to determine a minimum time needed to detect a water incur-
sion into the tank at a specified rate. This time is specific to each
tank size and geometry. The calculations are illustrated for an
8,000-gallon steel tank with a 96-inch diameter and 256 inches long.
39
-------
Suppose there are x inches of water in the tank. The tank is made
of quarter-inch steel, so the inside diameter is 95.5 inchess giving a
radius, r, of 47.75 inches. The water surface will be 2d wide, where d,
in inches, is calculated as
- (r - x) 2
where x is the water depth. The area of the water surface at depth of
x inches of water is then given by 255.5 x 2d inch2. iMultiplying this by
the minimum level change and dividing the result by 231 inch3 per gallon
gives approximately the volume change in gallons that the sensor Coin
detect reliably. This differs with the level of water in the tank. (For
a somewhat more accurate approximation, calculate d at level x and at
level x + MLC and average the two readings for the d to be used to calcu-
late the change in volume of water that can be detected.)
To determine how long the AT6S will take to detect a water incursion
at the rate of 0.20 gallon per hour, divide the minimum volume change
that the water sensor can detect by 0.20 gallon per hour. As a numerical
example, suppose the depth of the water were 1 inch and the minimum
detectable change were 1/8 inch. In an 8,000-gallon tank with inside
diameter 95.5 inches and length 255.5 inches, the water surface width, d,
is calculated as
d = >/(47775)2 - (46.75)2 = 9.72 inches
The volume, in inch3, corresponding to a 1/8-inch increase is
V = 2(9.72) x 255.5 x (1/8)
or
V = 620.94 inch3 |
i
In gallons, the volume is
V = 220.94/231 = 2.688 gallons
The time that the sensor will take to detect water incursions at the rate
of 0.20 gallon per hour will be
time = 2.688 gallons/0.20 gallon per hour = 13.44 hours
40
-------
Thus, the sensor would detect water coming in at the rate of 0.20 gallon
per hour after 13.4 hours, or about half a day. The incursion of the
water into the tank should be obvious on a day-to-day basis under these
conditions.
7.3 SUPPLEMENTAL CALCULATIONS AND DATA ANALYSES (OPTIONAL)
Other information can be obtained from the test data. This informa-
tion is not required for establishing that the AT6S meets the federal EPA
performance requirements, but may be useful to the vendor of the ATGS
The calculations described in this section are therefore optional. They
may be performed and reported to the vendor, but are not required and are
not reported on the results form. These supplemental calculations
include determining a minimum threshold, a minimum detectable leak rate
and relating the performance to factors such as temperature differential
waiting time, and product level. Such information may be particularly '
useful to the vendor for future improvements of his ATGS.
The experimental design tests the system under a variety of condi-
tions chosen to be reasonably representative of actual test conditions
The tests occur in pairs after each fill cycle. A comparison of the
results from the first of the pair with the second of that pair allows
one to determine if the additional stabilization time improved the
performance. Similarly, comparisons among the tests at each temperature
condition allow one to determine whether the temperature conditions
affected the performance. A comparison among test results performed with
a tank either full or half empty will provide an assessment of the effect
of product level on the system's performance. Finally, the performance
under the four induced leak conditions can be compared to determine
whether the system performance varies with leak rate.
The factors can be investigated simultaneously through a statistical
technique called analysis of variance. The detailed computational
formulas for a generalized analysis of variance are beyond the scope of
this protocol. For users unfamiliar with analysis of variance, equations
to test for the effect of stabilization period, temperature, and product
volume individually are presented in detail, although the evaluating
organization should feel free to use the analysis of variance approach to
the calculations if they have the knowledge and computer programs
available,
7.3.1 Minimum Threshold
The 24 test results can also be used to determine a threshold to
give a specified false alarm rate of say 5%. This threshold may not be
the same as the threshold, C, pertaining to the system as reported by the
vendor. Denote by C5%, the threshold corresponding to a P(FA) of 5%.
The following demonstrates the approach for computing C5%. Solve the
equation
41
-------
P(FA) = P(t > (C5% - B)/SD} = 0.05
for Cgeg. If the bias is not statistically significantly different from
zero (Section 7.1.1), then replace B with 0. From the t-table with
23 degrees of freedom obtain the 5th-percentile. This value is 1.714.
Solving the equation above for C5^ yields
(C5% - B)/SD = 1.714
In the case of a nonsignificant bias, this would be C5g = 1.714 SO.
7.3.2 Minimum Detectable Leak Rate
With the data available from the evaluation, the minimum detectable
leak rate, R55Ł, corresponding to a probability of detection, P(D), of 95%
and a calculated threshold, C5^, can be calculated by solving the follow-
ing equation for R$%:
P(D(R5%)) = PŁt > (C5% - R5% - B)/SD} = 0.95
where C5% is the threshold corresponding to a P(FA) of 5% as previously
calculated.
At the P(FA) of 5%, solving the equation above is equivalent to
solving |
or
- 1.714 SD + C - B
which, after substituting 1.714 SD for (C5^ - B), is equivalent to
Rco; = 2Ccv - 2B
3A> 3«
Substitute 0 for B in all calculations when the bias is not statistically
significant. Otherwise, use the value of B estimated from the data.
42
-------
Thus, the minimum detectable leak rate with a probability of detec-
tion of 95% is twice the calculated threshold, C5%, determined to give a
false alarm of 5%, minus twice the bias if .the bias is statistically
significant.
In summary, based on the 24 pairs of measured and induced leak
rates, the minimum threshold, Cg%, and the minimum detectable leak rate,
R5«g, are calculated as shown below.
If the bias is not statistically significant:
For a P(FA) of 5% Ccv = 1.714 SD
For a P(D(R)) of 95% RjjJ = 2C5%
If the bias is statistically significant:
For a P(FA) of 5% C5% = 1.714 SD + Bias
For a P(D(R)) of 95% Rg = 2C5% - 2 Bias
7.3.3 Test for Adequacy of Stabilization Period
The performance estimates obtained in Sections 7.1.2 and 7.1.3 will
indicate whether the system meets the EPA performance standards. The
calculations in this section allow one to determine whether the system's
performance is affected by the additional stabilization time the tank has
experienced by the second test after each fill cycle. These statistical
tests are designed primarily to help determine why an ATGS did not meet
the performance standards.
The procedure outlined in Section 6 allows time for the tank to
stabilize after fuel is pumped into the tank prior to the first test of
each set. Thus, additional stabilization takes place between the first
and second tests of the first pair in each set. The length of the
stabilization period following refueling as well as the time between
tests are specified by each ATGS. The following statistical test is a
means to detect whether the additional stabilization period for the
second test improves performance. If the stabilization period prior to
the first test in each set is too short, then one would expect larger
discrepancies between measured and induced leak rates for these first
tests as compared to those for the second tests.
Step 1: Calculate the absolute value of the 12 differences, d,, between
the measured (L) and induced (S) leak rates for the first
2 tests in each set (last column in Table 2).
Step 2: Calculate the average of the absolute differences for the first
and second test in each set separately.
43
-------
dg + d13 + dl7 + dai)/6
D2 = (da + d6 + d
10 llf ie 22
die + d22)/6
Step 3: Calculate the variances of the absolute differences from the
first and second test in each set separately.
= {(d2 -
(d. - D2)
(d22- D2)2} /5
Step 4: Calculate the pooled standard deviation.
10
Step 5: Calculate the t-statistic:
(D, - Da)
1 - D2)
P \Tfi
Step 6: From the t-table, obtain the critical value corresponding to a t
with (6+6-2) = 10 degrees of freedom and a two-sided 5% signifi-
cance level (a = 0.025 in the table). This value is 2.228.
Step 7: Compare the absolute value of t, abs(t), to 2.228. If abs(t) is
less than 2.228, conclude that the average difference between
measured and induced leak rates obtained from the first tests
after stabilization is not significantly different (at the 5%
significance level) from the average difference between measured
and induced leak rates obtained from the second tests after
stabilization. In other words, there has not been an additional
stabilization effect between the beginning of the testing and
the end. Otherwise, conclude that the difference is statisti-
cally significant, that is, the system's performance is differ-
ent with a longer stabilization period.
If the results are statistically significant, then the performance
of the system is different for the tests with the additional stabiliza-
tion period. If the performance is better, that is, if the absolute
differences for the testing with additional stabilization are smaller
than those for the tests with the minimum stabilization period, then the
44
-------
system would show improved performance if it increased its required
!u rl«zat1on Per1od- If the system's overall performance did not meet
the EPA performance standard, performance estimates with the additional
stabilization can be calculated using only- the 6 test results with the
additional stabilization. If the results indicate that the system does
not meet the EPA performance standard but could meet the EPA performance
standard with the additional stabilization, that finding should be
reported. Note that the system would still need to conduct the full
24 tests at the longer stabilization time before claiming to meet the EPA
performance standard.
7.3.4 Test for Adequate Temperature Compensation
,*•« Th1! !ection.a^°ws one to test whether the system's performance is
different for various temperature conditions. A total of eight tests
will have been performed with each of the three temperature differen-
tials, Tlt T2, and T3 (the nominal values of 0°, -5°, and +5°F will have
been randomly assigned to TIf T2, and T3). The 24 tests have been
ordered by temperature differential and test number in Table 4 for the
example order of sets from Table 1. In general, group the tests by
temperature condition.
The test results from the three temperature conditions are compared
to check the system's performance in compensating for temperature differ-
entials. If the temperature compensation of the system is adequate, the
three groups should give comparable results. If temperature compensation
is not adequate, results from the conditions with a temperature differen-
tial will be less reliable than results with no temperature difference.
The following statistical procedure (Bonferroni t-tests) provides a
means for testing for temperature effect on the test results. With three
temperature differentials considered in the test schedule, three compari-
sons will need to be made: Tx vs. T2, Tx vs. T3, and T2 vs. T3.
Step 1. Calculate the average of the absolute differences in each group.
Mi - Z) d./8 where gt denotes the 8 subscripts in Group 1
M2 = Z) d-j/8 where g2 denotes the 8 subscripts in Group 2
Ms = Z) d./8 where g3 denotes the 8 subscripts in Group 3
45
-------
Table 4. ORGANIZATION OF DATA TO TEST FOR
TEMPERATURE EFFECTS
Test No,
Pair No. Set No,
Nominal
temperature
differential
(degree F)
Absolute
leak rate
difference
|L-s|
(gallon per hour)
5
6
7
8
17
18
19
20
1
2
3
4
13
14
15
16
9
10
11
12
21
22
23
24
3
3
4
4
9
9
10
10
1
1
2
2
7
7
8
8
5
5
6
6
11
11.
12
12
2
2
2
2
5
5
5
5
1
1
1
1
4
4
4
4
3
3
3
3
6
6
6
6
TI
TI
TI
TI
TI
TI
TI
TI
T2
T2
T2
T2
T2
T2
T2
T2
T3
T3
T3
T3
T3
T3
T3
T3
cls
d.
d7
d8
d17
dI8
d«
d20
di
d2
d3
dH
; di,
di*
dis
di.
d9
die
dn
dia
d21
d22
d23
d21f
Group 1
Group 2
Group 3
46
-------
Step 2. Calculate the variance of the absolute differences in each
group.
(d. - ^
, - M2)2/7
Var3 -Ł (d. - M3)2/7
93
Step 3. Calculate the pooled variance of Var^ Var2, and Var3.
7Varx + 7Var_ + 7Var,
Var = - — -^ - - _ i
P 24 - 3
or
Varp = (Varj + Var2 + Var3)/3
Step 4. Compute the standard error, SE, of the difference between each
pair of the means, Mls M2, and M3.
KHI
1/2
or
Step 5. Obtain the 95th percentile of the Bonferroni t-statistic with
(24-3) = 21 degrees of freedom and three comparisons. This
statistic is t = 2.60. (Reference: Miller, Ruppert G., Jr.
1981. Simultaneous Statistical inference. Second Edition.
Springer-Verlay, New York, New York.)
Step 6. Compute the critical difference, D, against which each pairwise
difference between group means will be compared.
D = SE x t = SE x 2.60
47
-------
Step 7. Compare the absolute difference of the three pairwise
differences with D.
Compare |Mj - M2| with SE x 2.60
Compare \^ - M3j with SE x 2.60
Compare |M2 - M3| with SE x 2.60
If any difference in group means, in absolute value, exceeds the critical
value of SE x 2.60, then conclude that the system's performance is influ-
enced by the temperature conditions.
If the results are statistically significant, the system's perfor-
mance is affected by the temperature conditions. If the overall perfor-
mance evaluation met the EPA standards, the effect of a 5™F temperature
difference on the system does not degrade performance severely. However,
this does not eliminate the possibility that larger differences could
give misleading results. If the overall performance did not meet the EPA
performance standards, and the temperature effect was significant, then
the system needs to improve its temperature compensation and/or stabi-
lization time in order to meet EPA performance standards. Again, an
evaluation testing the modified ATGS would need to be conducted to docu-
ment the performance before the ATGS could claim to meet the performance
standards.
7.3.5 Test for Effect of In-Tank Product Volume
The procedure outlined in Section 6 required that the tank be either
half full or filled to between 90% and 95% capacity. As shown in
Table 1, 12 tests will have been run with the tank half full, and
12 tests with the tank full to 90% to 95% capacity. The 24 tests have
been ordered by product volume and test number in Table* 5 for the example
order of tests from Table 1.
The test results from the two volume levels are compared to check
for the effect of product volume on the system's performance. If the
effect is negligible, the two groups of results should be comparable. If
the system's performance is affected by the product level, then the ATGS
may not meet EPA performance standards at all product levels. If it does
meet the performance standards at both levels, it can be used in the test
mode at any product level. However, if there is a significant difference
in performance at the two levels, it might be advisable to recommend that
the ATGS be used in its test mode only for certain product levels. If
the performance is not adequate for one of the product levels, the per-
formance of the ATGS is probably marginal. The operation of the test
function could be restricted to the product level where the performance
was adequate.
48
-------
Table 5. ORGANIZATION OF DATA TO TEST
FOR PRODUCT VOLUME EFFECT
Test No.
1
2
5
6
9
10
13
14
17
18
21
22
3
4
7
8
11
12
15
16
19
20
23
24
Pair No.
1
1
3
3
5
5
7
7
9
9
11
11
2
2
4
4
6
6
8
8
10
10
12
12
Set No.
1
1
2
2
3
3
4
4
5
5
6
6
1
1
2
2
3
3
4
4
5
5
6
6
In-tank
product
volume
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
90-95% full
50% full
50% full
50% full
50% full
50% full
50% full
50% full
50% full
50% full
50% full
50% full
50% full
Absolute
leak rate
difference
IL-SI
(gallon per hour)
dj
d,
U2
o •-
U5
df
ug
d10 Group 1
j13
d1*
j17
do,
U21
d22
d
dif
da
U8
dn
d12 Group 2
a i •-
U15
"16
Q t **
U19
f| _ _
"20
n « -»
U23
49
-------
One of the consequences of using an ATGS to test eit various levels
of product in the tank is that the test can only find leaks below the
product level used in the test. The performance standard calls for
detecting a leak from any portion of the tank that normally contains
product. Ideally, the test should be run with the tank as full as it is
filled in practice so that leaks can be detected from any part of the
tank. If the test results were restricted to testing when the tank was
half full, for example, the test could not find leaks iin the upper half
of the tank.
The following statistical procedure (two-sample t--test) provides a
means for testing the effect of product volume on the test results.
Step 1. Calculate the average of the absolute differences in the two
groups.
where gt denotes the 12 subscripts in Group 1
9i
M2 =^}d./12 where g2 denotes the 12 subscripts in Group 2
92
Step 2. Calculate the variance of the absolute differences in the two
groups.
91
Var2 -
- M2) /ll
Step 3. Calculate the pooled variance of Va^ and \lar2
HVar
Varp '
242
or
Var = (Var1 + Var2)/2
Step 4. Compute the standard error, SE, of the difference, between Mt and
M2.
Jl/2
SE =
SE =-y/Var /6
50
-------
Step 5. Calculate the t-statistic:
SE !
Step 6. From the t-table in Appendix A, obtain the critical value
corresponding to a t with (12 + 12-2) =22 degrees of freedom
and a two-sided 5% significance level. This value is 2.074.
Step 7. Compare the absolute value of t, abs(t), to 2.074. If abs(t) is
less than 2.074, conclude that the average difference between
measured and induced leak rates obtained with a tank half full
is not significantly different (at the 5% significance level)
from the average difference between measured and induced leak
rates obtained with a tank filled to 90% to 95% capacity. In
other words, the amount of product, in this given range, has no
significant impact on the leak rate results. Otherwise, con-
clude that the difference is statistically significant, that is,
the system's performance depends on the amount of product in the
tank.
7.4 OUTLINE OF CALCULATIONS FOR ALTERNATIVE APPROACH
This section describes the data analysis required for the alterna-
tive protocol described in Section 6.5.
The water sensor data will be identical to that obtained with the
standard protocol outlined in Section 6.4. Consequently, the same data
analysis will be used. Refer to Section 7.2 for the details.
7.4.1 Calculation of P(FA) and P(D)
Using the leak rate reported by the ATGS and the actual leak rate
(zero for tight tank tests, measured for the induced leak rate tests),
calculate the differences between the measured and actual leak rates.
Calculate the mean and standard deviation of these differences as in
Section 7.1.1. Perform the test for significant bias and estimate the
P(FA) and the P(D) as described in that section.
Calculate the variances of the differences separately for the data
from the tests on the tight tanks and those from the tests on tanks with
induced leak rates. This can be done as in Section 7.3.3, except that
the two groups are now defined by the leak status of the tanks and the
sample sizes will not be equal. Let the subscript "1" denote the tight
tank data set and "2" denote the data from the tests with induced leaks.
51
-------
Let P! be the number of test results from tight tanks and n2 be the
number of test results from induced leak rate tests. Denote by d^ the
difference between measured and induced leak rates for each test, where
j-1 or 2, and' i-l,...,n1 or n2. Then calculate
I WH - di)2/K - D
and
s2 =
n.
(d2. - d2)2/(n:
where .the summations are taken over the appropriate groups; of data, and
where d. denotes the mean of the data in group j, and is given by
n.
d_. =
Form the ratio
and compare this statistic to the F statistic with (n2-l) and (n^l)
degrees of freedom for the numerator and denominator, respectively, at
the 5% significance level. (The F statistic can be obtained from the
F-Table found in any statistical reference book.) If the calculated F
statistic is larger than the tabulated F value, conclude that the data
from the induced leak rate tests are significantly more variable than
those from the tight tanks. If this is the case, it might impair the
ability of the AT6S to detect leaks. Recompute the P(D) (see Sec-
tion 7.1.3) using the standard deviation calculated from just the induced
leak rate tests, S2, to verify that P(D) is still at least 95%.
7.4.2 Limitations on the Results
The limitations on the results must be calculated from the actual
test conditions. Since the conditions were not controlled,, here, but
were observed, the following approach is taken to determine the appli-
cable conditions.
52
-------
Size of Tank
List the tank sizes of the tests in the complete data set (all valid
tests). Order the sizes from smallest to largest. Determine the 80th
percent-He of these ordered sizes. (That is, the smallest size just
exceeded by 20% of the tank sizes.) Multiply that size by 1.25 and
report the result as the maximum size to which the performance results
can be extended. Note that this implies that at least 20% of the tanks
must be of at least a specified size if the ATGS is intended to work on
en at size OT tank.
Maximum Allowable Temperature Difference
Calculate the temperature difference between the product in the tank
and that of newly added product for each delivery in the data set. Note
that the temperature of the delivered product can be calculated from the
temperature of the product in the tank immediately before delivery the
temperature of the product in the tank immediately after delivery and
the volumes of product by the following formula:
T - TA VA - TB VB
TD - — V
^UuSKr]Pt V??°tes Product 1n tank after delivery, B denotes product
tank before delivery, D denotes product delivered, T denotes product
temperature, and V denotes volume. yruuuct
Calculate the standard deviation of the temperature differentials
pn^-ai JiŁ i*MM!y Irh RePo1l.th1s as ^ maximum temperature differ-
ential for which the ATGS evaluation is valid.
When the calculations are complete, enter the results on the
standard results reporting form in Appendix B. Also check the box on
that form to indicate that the evaluation was done using the alternative
approach .
Average Waiting Time After Filling
Use the time interval between the most recent fill or product
delivery and each following test as a stabilization time. Order these
times from least to greatest and determine the 20th percentile. Report
this as the minimum stabilization time. "epori
53
-------
Average Data Collection Time Per Test
The tests often have a constant or nearly constant duration
prescribed by the AT6S. If so, simply report this as the test data
collection time. If the ATGS software determines a test time from the
data, report the average test time actually taken by the test and note
that the ATGS software determines the applicable test time.
54
-------
SECTION 8
INTERPRETATION
Each function of the ATGS is evaluated separately based on data
analysis of experimental test results. This section covers the leak
detection function, water level detection function, and measurement of
minimum water level change. The entire evaluation process results in
performance estimates for the leak detection function of the ATGS. The
results reported are valid for the experimental conditions during the
evaluation, which have been chosen to represent the most common situa-
tions encountered in the field. These should be typical of most tank
testing conditions, but extreme weather conditions can occur and might
adversely affect the performance of the ATGS. The performance of the
leak detection function should be at least as good for tanks smaller than
the test tank. However, the performance evaluation results should only
be scaled up to tanks of 25% greater capacity than the test tank. The
performance of the water sensor in terms of minimum detectable level and
minimum detectable change are independent of the tank size. However, the
volume that corresponds to these heights of water does depend on tank
size. It should be emphasized that the performance estimates are based
on average results obtained in the tests. An individual test may not do
as well. Some individual tests may do better. Vendors are encouraged to
provide a measure of the precision of a test, such as a standard error
for their calculated leak rate at that site, along with the leak rate and
test results.
8.1 LEAK TEST FUNCTION EVALUATION
The relevant performance measures for proving that an ATGS meets EPA
standards are the P(FA) and P(D) for a leak rate of 0.20 gallon per
hour. The estimated P(FA) can be compared with the EPA standard of P(FA)
not to exceed 5%. In general, a lower P(FA) is preferable, since it
implies that the chance of mistakenly indicating a leak on a tight tank
is less. However, reducing the false alarm rate may also reduce the
chance of detecting a leak. The probability of detection generally
increases with the size of the leak. The EPA standard specifies that
P(0) be at least 95% for a leak of 0.20 gallon per hour. A higher esti-
mated P(D) means that there is less chance of missing a small leak.
If the estimated performance of the ATGS did not meet the EPA
performance requirements, the vendor may want to investigate the condi-
tions that affected the performance as described in Section 7.3, Supple-
mental Calculations and Data Analyses. If the stabilization time,
temperature condition, or the product level can be shown to affect the
55
-------
performance of the ATGS, this may suggest ways to improve the ATGS. It
may be possible to improve the performance simply by changing the proce-
dure (e.g., waiting longer for the tank to stabilize) or it may be neces-
sary to redesign the hardware. In either case, a new evaluation with the
modified system is necessary to document that the ATGS does meet the per-
formance standards.
The relationship of performance to test conditions is primarily of
interest when the ATGS did not meet the EPA performance standards.
Developing these relationships is part of the optional or supplementary
data analysis that may be useful to the vendor, but is not of primary
interest to many tank owners or operators.
I
8.2 WATER LEVEL DETECTION FUNCTION
The minimum water level detected by the ATGS is estimated from the
average threshold of detection, and the variability of the water level
threshold is estimated by the standard deviation of the test data. The
minimum water level that will be detected at least 95% of the time is the
level to be reported. Statistically, this is a one-sided tolerance
limit.
The tolerance limit calculated in Section 7.2.1 estimates the mini-
mum water level that the ATGS can detect above the bottom of the probe.
If the installation of the ATGS leaves the probe at a specified distance
above the bottom of the tank (for example, 1 inch), then this minimum
distance needs to be added to the reported minimum detectable water
level.
8.3 MINIMUM WATER LEVEL CHANGE MEASUREMENT
Since ATG systems operate with the product at all levels of normal
tank operation, the water sensor can be used to test for leaks in the
event of a high ground-water level. If the ground-water level is above
the bottom of the tank, there will be an inward pressure when the product
level is sufficiently low, and if there is a hole in the tank, water will
flow into the tank under these conditions. Based on the ability of the
water sensor to detect a change in the level of water in the product, one
can determine how much water must enter the tank in order for an increase
in the water level to be detected. From this information, in turns one
can determine the size of a leak of water into the tank that the ATGS can
detect at a given time.
The standard deviation of the differences between the change in
water level measured by the sensor and the change induced during the
tests is used to determine the ability of the water level sensor to
detect changes in the water level. A two-sided 95% tolerance interval is
then calculated for this detection ability (Section 7.2.2).
The minimum change in water level that can be detected is used to
compute a minimum change in water volume in the tank. This conversion is
56
-------
specific to the tank size. Using the minimum change in water volume that
the sensor can detect, the time needed for the ATGS to detect an incur-
sion of water at the rate of 0.20 gallon per hour is calculated (Sec-
tion 7.2.3). This calculation indicates the time needed for the water
detector to identify an inflow of water at the minimum leak rate and to
alert the operator that the water level has increased. If the particular
ATGS has a water alarm, and if'the conditions for activating the water
alarm are specified, the length of time for that alarm to be activated
can be calculated.
57
-------
-------
SECTION 9
REPORTING OF RESULTS
Appendix B is designed to be the framework for a standard report.
There are five parts to Appendix B, each of which is preceded by instruc-
tions for completion. The first part is the Results of U.S. EPA Standard
Evaluation form. This is basically an executive summary of the find-
ings. It is designed to be used as a form that would be provided to each
tank owner/operator that uses this system of leak detection. Conse-
quently, it is quite succinct. The report should be structured so that
this results form can be easily reproduced for wide distribution.
The second part of the standard report consists of the Description
of the ATGS. A description form is included in Appendix B and should be
completed by the evaluating organization assisted by the vendor.
The third part of the standard report contains a Reporting Form for
Leak Rate Data, also described in Appendix B. This table summarizes the
test results and contains the information on starting dates and times,
test duration, leak rate results, etc.
The fourth part of Appendix B contains a blank Individual Test
Log. This form should be reproduced and used to record data in the
field. Copies of the completed daily test logs are to be included in the
standard report. These serve as the backup data to document the perfor-
mance estimates reported.
The fifth part of Appendix B provides a form to record the test
results when evaluating the system's water sensor. The data to be
recorded follow the testing protocol (in Section 6.4) to determine the
minimum level of water and the minimum water level change that the system
can detect.
If the optional calculations described in Section 7.3 are performed,
they should be reported to the vendor. It is suggested that these
results be reported in a separate section of the report, distinct from
the standard report. This would allow a user to identify the parts of
the standard report quickly while still having the supplemental informa-
tion available if needed.
The limitations on the results of the evaluation are to be reported
on the Results of U.S. EPA Standard Evaluation form. The intent is to
document that the results are valid under conditions represented by the
test conditions. Section 7.1.4 describes the summary of the test condi-
tions that should be reported as limitations on the results form. These
59
-------
Items are also discussed below. The test conditions hiave been chosen to
represent the majority of testing situations, but do not include the most
extreme conditions under which testing could be done. The test condi-
tions were also selected to be practical and not impose an undue burden
for evaluation on the test companies.
One practical limitation of the results is the size of the tank.
Tests based on volumetric changes generally perform less well as the size
of the tank increases. Consequently, the results of the evaluation may
be applied to tanks smaller than the test tank. The results may also be
extended to tanks of 25% larger capacity than the test tank. Thus, if
testing is done in a 10,000-gallon tank, the results may be extended to
tanks up to 12,500 gallons in size. If a company wants to document that
it can test large tanks, the evaluation needs to be done in a large tank.
A second limitation on the results is the temperature differential
between the product added to the tank and that of the product already in
the tank. Often the AT6S must perform a test shortly after the tank has
been filled. The reported results apply provided the temperature differ-
ential is no more than that used in the evaluation. Testing during the
EPA national survey (Flora, J. D., Jr., and J. E. Pelkey, "Typical Tank
Testing Conditions," EPA Contract No. 68-01-7383, Work Assignment 22,
Task 13, Final Report, December 1988) found that temperature differen-
tials were no more than 5°F for at least 60% of the tests. However, it
is clear that larger differences could exist. The evaluation testing may
be done using larger temperature differentials, reporting those actually
used. The results cannot be guaranteed for temperature differentials
larger than those used in the evaluation.
A third limitation on the results is the stabilization time needed
by the ATGS. The Individual Test Logs call for recording the actual
stabilization time used during the testing. The mean of these stabiliza-
tion times is reported. The results are valid for stabilization times at
least as long as those used in the evaluation. This is viewed as an
important limitation, since shorter stabilization times can adversely
affect the system's performance. In practice, an ATGS will often test
late at night when the tank is not used. This will usually result in a
stabilization time as long or longer than that used in the evaluation.
If an ATGS is used in a very active tank and does not do daily tests,
then the required monthly test should be scheduled to have at least the
minimum stabilization time used in the evaluation.
The duration of the data collecting phase of the test is another
limitation of the ATGS. If a test shortens the data collection time and
so collects less data, this may adversely affect the system's perfor-
mance. As a consequence, the results do not apply if the data collection
time is shortened. This is primarily of concern in documenting that a
tank is tight. If results clearly indicate a leak, this may sometimes be
ascertained in less time than needed to document a tight tank, particu-
larly if the leak rate is large. Thus, while the false alarm rate may be
larger if the test time is shortened, this is not usually a problem in
that if test results indicate a leak, efforts are usually made to iden-
tify and correct the source of the leak.
60
-------
The minimum depth of water that the sensor can detect is reported.
In addition, the minimum change in water level that the sensor can detect
is reported. This minimum detectable change is compared to the EPA per-
formance standard of 0.125 inch. From this minimum detectable change in
water level, a minimum volume change can be calculated based on the tank
size and depth of the water. A minimum time for detection is calculated
and reported as the time needed for water flowing into the tank at the
rate of 0.20 gallon per hour to increase the water volume enough for the
sensor to detect.
The same reporting forms can be used for the alternative evaluation
described in Section 6.5. The data analysis for the alternative approach
is described in Section 7.4. This analysis will result in reporting
observed average conditions during the evaluation. The limitations are
based on the observed conditions instead of experimentally controlled
conditions, but the results are reported on the same form. The Individ-
ual Test Log form should be applicable to the induced leak rate tests
under the alternative evaluation procedure. However, the evaluating
organization may find it more efficient to design a different data col-
lection form for recording the data from the many tight tank tests.
61
-------
-------
APPENDIX A
DEFINITIONS AND NOTATIONAL CONVENTIONS
A-l
-------
In this protocol leaks are viewed as product lost from the tank. As
a convention, leak rates are positive numbers, representing the amount of
product loss per unit time. Thus a larger leak represents a greater
product loss. Parts of the leak detection industry report volume changes
per unit time with the sign indicating whether product is lost from the
tank (negative sign) or is coming into the tank (positive sign). We
emphasize that here, leaks refer to the direction out of the tank and the
rate to the magnitude of the flow.
The performance of a leak detection method is expressed in terms of
the false alarm rate, P(FA), and the probability of detecting a leak of
specified size, P(D(R)), where R is the leak rate. In order to under-
stand these concepts, some explanation is helpful. Generally, the volu-
metric leak detection method, either a precision tank test or the leak
test function of an automatic tank gauging system (ATGS), estimates a
leak rate. This calculated rate is compared to a criterion or threshold,
C, determined by the manufacturer. If the calculated rate is in excess
of the criterion, the tank is declared to be leaking, otherwise, the tank
is called tight.
Figure A-l represents the process of determining whether a tank is
leaking or not. The curve on the left represents the inherent vari-
ability of the measured leak rate on a tight tank (with zero leak
rate). If the measured leak rate exceeds C, the tank is declared to
leak, a false alarm. The chance that this happens is represented by the
shaded area under the curve to the right of C, denoted a (alpha).
The variability of the measured leak rates for a tank that is
actually leaking at the rate R is represented by the curve on the right
in Figure A-l. Again, a leak is declared if the measured rate exceeds
the threshold, C. The probability that the leaking tank is correctly
identified as leaking is the area under the right hand curve to the right
of C. The probability of mistakenly declaring the leaking tank tight is
denoted by B (beta), the area of the left of C under the leaking tank
curve.
Changing the criterion, C, changes both a and B for a fixed leak
rate, R. If the leak rate R is increased, the curve on the right will
shift further to the right, decreasing 8 and increasing the probability
of detection for a fixed criterion, C. If the precision of a method is
increased, the curve becomes taller and narrower, decreasing both a and
$, resulting in improved performance.
A bias is a consistent error in one direction. This is illustrated
by Figure A-2. In it, both curves have been shifted to the right by an
amount of bias, B. In this illustration, the bias indicates a greater
leak rate than is actually present (the bias is positive in this case).
This has the effect of increasing the probability of a false alarm, while
reducing the probability of failing to detect a leak. That is, the
probability of detecting a leak of size R is increased, but so is the
chance of a false alarm. A bias toward underestimating the leak rate
would have the opposite effect. That is, it would decrease both the
false alarm rate and the probability of detecting a leak.
A-2
-------
Definitions of some of the terms used throughout the protocol are
presented next.
Nominal Leak Rate:
Induced Leak Rate:
Measured Leak Rate:
Critical Level, C:
False Alarm:
Probability of
False Alarm, P(FA):
Probability of
Detection,, P(D(R));
Method Bias, B:
Mean Squared Error, MSE
Root Mean Squared Error„
RMSE:
The set or target leak rate to be achieved as
closely as possible during testing. It is a
positive number in gallon per hour.
The actual leak rate, in gallon per hour, used
during testing, against which the results from
a given test device will be compared.
A positive number, in gallon per hour, measured
by the test device and indicating the amount of
product leaking out of the tank. A negative
leak rate would indicate that water is leaking
into the tank.
The leak rate above which a method declares a
leak. It is also called the threshold of the
method.
Declaring that a tank is leaking when in fact
it is tight.
The probability of declaring a tank leaking
when it is tight. In statistical terms, this
is also called the Type I error, and is denoted
by alpha (a). It is usually expressed in
percent, say,.5%.
The probability of detecting a leak rate of a
given size, R gallon per hour. In statistical
terms, it is the power of the test method and
is calculated as one minus beta (B), where beta
is the probability of not detecting (missing) a
leak rate R. Commonly, the power of a test is
expressed in percent, say, 95%.
The average difference between measured and
induced (actual) leak rates, in gallon per
hour. It is an indication of whether the test
device consistently overestimates (positive
bias) or underestimates (negative bias) the
actual leak rate.
An estimate of the overall performance of a
test method.
The positive square root of the mean squared
error.
A-3
-------
Precision: A measure of the test method's ability in pro-
ducing similar results (i.e., in close agree-
ment) under identical test conditions.
Statistically, the precision of repeated
measurements is expressed as the standard
deviation of these measurements.
Variance: A measure of the variability of measurements.
It is the square of the standard deviation.
Accuracy: The degree to which the measured leak rate
agrees with the induced leak rate on the aver-
age. If a method is accurate, it has a very
small or zero bias.
Resolution: The resolution of a measurement system is the
least change in the quantity being measured
which the system is capable of detecting.
A-4
-------
Tight Tank
Leaking Tank
Measured Leak Rate, L,
Gallons Per Hour
C = Criterion or Threshold for declaring a leak
(a leak is declared if the measured rate exceeds C)
a m Probability of False Alarm, P(FA)
ft = Probability of not detecting a leak rate R
I - /3 = Probability of detecting a leak rate R, P(D(R))
R = Leak Rate
Figure A-l. Distribution of measurement error on a tight
and leaking tank.
A-5
-------
Tight Tank
Leaking Tank
B C
R R + B
Measured Leak Rate, L,
Gallons Per Hour
C = Criterion or Threshold for declaring a leak
(a leak is declared if the measured rate exceeds C)
a = Probability of False Alarm, P(FA)
yS = Probability of not detecting a leak rate R
I - /3 = Probability of detecting a leak rate R, P(D(R))
R = Leak Rate
B = Bias
Figure A-2. Distribution of measurement error o?i a tight and
leaking tank in the case of a positive bias.
A-6
-------
Table A-l. PERCENTAGE POINTS OF STUDENT'S t-DISTRIBUTION
f(t)
df
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
40
60
120
inf
• = .10
3.078
1.886
1.638
1.333
1.476
1.440
1.415
1.397
1.383
1.372
1.363
1.356
1.350
1.345
1.341
1.337
1.333
1.330
1.328
1.325
1.323
1.321
1.319
1.318
1.316
1.315
1.314
1.313
1.311
1.310
1.303
1.296
1.289
1.282
0 f.
• - .05 a - .025
6.314
2.920
2.353
2.132
2.015
1.943
1.895
1.860
1.833
1.812
1.796
1.782
1.771
1.761
1.753
1.746
1.740
1.734
1.729
1.725
1.721
1.717
1.714
1.711
1.708
1.706
1.703
1.701
1.699
1.697
1.684
1.671
1.658
1.645
12.706
4.303
3.182
2.776
2.571
2.447
2.365
2.306
2.262
2.228
2.201
2.179
2.160
2.145
2.131
2.120
2.110
2.101
2.093
2.086
2.080
2.074
2.069
2.064
2.060
2.056
2.052
2.048
2.045
2.042
2.021
2.000
1.980
1.960
a = .010
31.821
6.965
4.541
3.747
3.365
3.143
2.998
2.896
2.821
2.764
2.718
2.681
2.650
2.624
2.602
2.583
2.567
2.552
2.539
2.528
2.518
2.508
2.500
2.492
2.485
2.479
2.473
2.467
2462
2.457
2423
2.390
2.358
2.326
t
• -.005
63.657
9.925
5.841
4.604
4.032
3.707
3.499
3.355
3.250
3.169
3.106
3.055
3.012
2.977
2.947
2.921
2.898
2.878
2.861
. 2.845
2.831
2.819
2.807
2.797
2.787
2.779
2.771
2.763
2.756
2.750
2.704
2.660
2.617
2.576
A-7
-------
-------
APPENDIX B
REPORTING FORMS
B-l
-------
Appendix B provides five sets of blank forms. Once filled out, these
forms will provide the framework for a standard report. They consist of
the following:
1. Results of U.S. EPA Standard Evaluation—Automatic Tank Gauging
System (two pages)
2. Description—Automatic Tank Gauging System (six pages)
3. Reporting Form for Leak Rate Data—Automatic Tank Gauging System
(two pages)
4. Individual Test Log—Automatic Tank Gauging System (five pages)
5. Reporting Form for Water Sensor Evaluation Data—Automatic Tank
Gauging System (four pages)
Each set of forms is preceded by instructions on how the forms are to be
filled out and by whom. The following is an overview on various
responsibilities.
Who is responsible for filling out which form?
1. Results of U.S. EPA Standard Evaluation. The evaluating organiza-
tion is responsible for completing this form at the end of the
evaluation.
2. Description of Automatic Tank Gauging System. The evaluating
organization assisted by the vendor will complete this form by the
end of the evaluation.
3. Reporting Form for Leak Rate Data. This form is to be completed by
the evaluating organization. In general, the-statistician analyzing
the data will complete this form. A blank form can be developed on
a personal computer, the data base for a given evaluation generated,
and the two merged on the computer. The form can also be filled out
manually. The input for that form will consist of the field test
results recorded by the evaluating organization's field crew on the
Individual Test Logs (below) and the ATGS test results.
4. Individual Test Logs. These forms are to be used and completed by
the evaluating organization's field crew. These forms need to be
kept blind to the vendor during testing. It is recommended that the
evaluating organization reproduce a sufficient number (at least 24
copies) of the blank form provided in this appendix and produce a
bound notebook for the complete test period.
I
5. Reporting Form for Water Sensor Evaluation Data. These forms pro-
vide a template for the water sensor evaluation data. They are to
be used and completed by the evaluating organization's field crew.
It is recommended that the evaluating organization reproduce a
sufficient number (at least 20 copies) of the blank form provided in
this appendix and produce a bound notebook to be used in the field.
B-2
-------
JniiSo0???1!!:10!! °f the evaluation» the evaluating organization will
collate all the forms into a single Standard Report in the order listed
?K«!i +* ? Ca?cs,w!!?r?-t5e evaluating organization performed addi-
tional, optional calculations (see Section 7.3 of the protocol), these
results can be attached to the standard report. There is no reportina
requirement for these calculations, however. ^rtmy
If the alternative EPA test procedure described in Section 6.5 was
followed, then the reporting is essentially the same as that for the
standard evaluation procedure. The major difference is that the Results
of U.S. EPA Standard Evaluation form will be completed using the results
°f *he«lon«tions described in Section 7.4. A box is provided to indi-
cate which evaluation procedure was used. Individual test logs will onlv
dca ?? V?r th??6 !65tS Performed under the induced leak rate condi-
ns. All data collected on the tanks under the no-leak condition need
rTed ^a-taChl!:9 C°P.1eS Of the forms on Wh1ch the ^suHs wire
ratP rnnHit-" "? M?^ the tan* *?* results ^o-^k and induced leak
rate conditions) will be summarized on the Reporting Form for Leak Rate
uata. There will be no changes in the reporting of the water sensor
performance since only one testing procedure is presented.
Distribution of the Evaluation Test Results
The organization performing the evaluation will prepare a report to the
vendor describing the results of the evaluation. This report consists
primarily of the forms in Appendix B. The first form reports the results
of the evaluation. This two-page form is designed to be distributed
widely. A copy of this two-page form will be supplied to each tank
owner/operator who uses this method of leak detection. The owner/
operator must retain a copy of this form as part of his record keepinq
requirements. The owner/operator must also retain copies of each tank
test performed at his facility to document that the tank(s) passed the
tightness test. This two-page form will also be distributed to regula-
tors who must approve leak detection methods for use in their jurisdic-
tion.
The complete report, consisting of all the forms in Appendix B, will be
submitted by the evaluating organization to the vendor of the leak detec-
tion method. The vendor may distribute the complete report to regulators
who wish to see the data collected during the evaluation. It may also be
distributed to customers of the leak detection method who want to see the
additional information before deciding to select a particular leak detec-
tion method.
The optional part of the calculations (Section 7.3), if done, would be
reported by the evaluating organization to the vendor of the leak detec-
tion method. This is intended primarily for the vendor's use in under-
standing the details of the performance and perhaps suggesting how to
improve the method. It is left to the vendor whether to distribute this
form, and if so, to whom.
The evaluating organization of the leak detection method provides the
report to the vendor. Distribution of the results to tank owner/
operators and to regulators is the responsibility of the vendor
B-3
-------
Results of U.S. EPA Standard Evaluation
Automatic Tank Gauging System (ATGS)
Instructions for completing the formi
This 2-page form is to be filled out by the evaluating organization upon
completion of the evaluation of the ATGS. This form will! contain the
most important information relative to the ATGS evaluation. All items
are to be filled out and the appropriate boxes checked. If a question is
not applicable to the ATGS, write 'NA1 in the appropriate space.
This form consists of five main parts. These are:
1. ATGS Description
2. Evaluation Results
3. Test Conditions During Evaluation
4. Limitations on the Results
5. Certification of Results
ATGS Description
Indicate the commercial name of the ATGS, the version, and the name,
address, and telephone number of the vendor. Some vendors use different
versions of their ATGS when using it with different products or tank
sizes. If so, indicate the version used in the evaluation. If the
vendor is not the party responsible for the development and use of the
ATGS9 then indicate the home office name and address of the responsible
party.
Evaluation Results
The ATG system's threshold, C, is supplied by the vendor. This is the
criterion for declaring a tank to be leaking. Typically, a method
declares a tank to be leaking if the measured leak rate exceeds C.
P(FA) is the probability of false alarm calculated in Section 7.1.2.
Report P(FA) in percent. P(FA) may be rounded to the nearest whole
percent.
P(D) is the probability of detecting a leak rate of 0.20 gallon per hour
and is calculated in Section 7.1.3. Report P(D) in percent. P(D) may be
rounded to the nearest whole percent.
The minimum detectable water level and the minimum detectable level
change that the sensor can detect will have been obtained from the
calculations in Sections 7.2.1 and 7.2.2.
If the P(FA) calculated in Section 7.1.2 is 5% or less and if the P(D)
calculated in Section 7.1.3 is 95% or more, then check the first 'does'
box. Otherwise, check the first 'does not' box. If the minimum water
level change calculated in Section 7.2.2 is less than or equal to
1/8 inch, then check the second 'does' box. If the minimum water level
change exceeds 1/8 inch, then check the second 'does not' box.
B-4
-------
Test Conditions During Evaluation
Insert the information in the blanks provided. The nominal volume of the
tank in gallons is requested as is the tank material, steel or fiber-
glass. Also, give the tank diameter and length in inches. Report the
product used during the testing. Give the range of temperature dif-
ferences actually measured as well as the standard deviation of the
observed temperature differences. Note, if more than one tank, product,
or level was used in the testing, indicate this and refer to the data
summary form where these should be documented.
Limitations on the Results
The size (gallons) of the largest tank to which these results can be
applied is calculated as 1.50 times the size (gallons) of the test tank.
The temperature differential, the waiting time after adding the product
until testing, and the total data collection time should be completed
using the results from calculations in Section 7.1.4.
If the alternative evaluation procedures described in Section 6.5 has
been followed, then report the results obtained from the calculations in
Section 7.4.
Certification of Results
Here, the responsible person at the evaluating organization indicates
which test procedure was followed and provides his/her name and signa-
ture, and the name, address, and telephone number of the organization.
B-5
-------
-------
. Results of U.S. EPA Standard Evaluation
Automatic Tank Gauging System (ATGS)
This form tells whether the automatic tank gauging system (ATGS) described below complies with
the performance requirements of the federal underground storage tank regulation. The evaluation
was conducted by the equipment manufacturer .or a consultant to the manufacturer according to
the U.S. EPA's "Standard Test Procedure for Evaluating Leak Detection Methods: Automatic
Tank Gauging Systems." The full evaluation report also includes a form describing the method and
a form summarizing the test data.
Tank owners using this leak detection system should keep this form on file to prove compliance
with the federal regulations. Tank owners should check with State and local agencies to make sure
this form satisfies their requirements.
ATGS Description
Name
Version number
Vendor
(street address) ~
(citv) (state) (zip] (phone)
Evaluation Results ~ ~~
This ATGS, which declares a tank to be leaking when the measured leak rate exceeds the
threshold of gallon per hour, has a probability of false alarms [P(FA)J of %.
The corresponding probability of detection [P(D)] of a 0.20 gallon per hour leak is %.
The minimum water level (threshold) in the tank that the ATGS can detect is inches.
The minimum change in water level that can be detected by the ATGS is inches
(provided that the water level is above the threshold).
Therefore, this ATGS D does D does not meet the federal performance standards
established by the U.S. Environmental Protection Agency (0.20 gallon per hour at P(D) of 95%
and P(FA) of 5%), and this ATGS U does D does not meet the federal performance
standard of measuring water in the bottom of the tank to the nearest 1/8 inch.
Test Conditions During Evaluation ~ ~
The evaluation testing was conducted in a gallon D steel D fiberglass tank that
was inches in diameter and inches long.
The temperature difference between product added to fill the tank and product already in the
tank ranged from °F to °F, with a standard deviation of °F.
The tests were conducted with the tank product levels and % full.
The product used in the evaluation was
ATGS - Results Form Page 1 of 2
-------
Name of ATGS_
Version
Limitations on the Results
The performance estimates above are only valid when:
• The method has not been substantially changed.
• The vendor's instructions for installing and operating the ATGS are followed.
• The tank contains a product identified on the method description form.
• The tank is no larger than gallons.
• The tank is at least percent full.
• The waiting time after adding any substantial amount of product
to the tank is hours.
• The temperature of the added product does not differ more than
degrees Fahrenheit from that already in the tank.
• The total data collection time for the test is at least hours.
• Other limitations specified by the vendor or determined during testing: :
> Safety disclaimer: This test procedure only addresses the issue of the ATG system's
ability to detect leaks. It does not test the equipment for safety hazards.
Certification of Results
I certify that the ATGS was installed and operated according to the vendor's instructions and
that the results presented on this form are those obtained during the evaluation. I also certify
that the evaluation was performed according to one of the following:
D standard EPA test procedure for ATGS
CD alternative EPA test procedure for ATGS
(printed name) (organization performing evaluation)
(signature) (city, state, zip)
(date) (phone number)
ATGS - Results Form Page 2 of 2
-------
Description of Automatic Tank Gauging System
Instructions for completing the form
This 6-page form is to be filled out by the evaluating organization with
assistance from the vendor, as- part of the evaluation of the AT6S This
form provides supporting information on the principles behind the system
or on how the equipment works. jaw=m
To minimize the time to complete this form, the most frequently expected
answers to the questions have been provided. For those answers that are
dependent on site conditions, please give answers that apply in "tvoical"
conditions. Please write in any additional information about the testina
method that you believe is important. a
There are seven parts to this form. These are:
1. AT6S Name and Version
2. Product
> Product type
> Product level
3. Level Measurement
4. Temperature Measurement
5. Data Acquisition
6. Procedure Information
> Waiting times
> Test duration
> Total time
> Identifying and correcting for interfering factors
> Interpreting test results
7. Exceptions
Indicate the commercial name and the version of the ATGS in the first
part.
NOTE: The version is provided for AT6 systems that use different
versions of the equipment for different products or tank sizes.
For the six remaining parts, check all appropriate boxes for each
?XX ""?"; Check,niore than one box per question if it applies. If a box
Other is checked, please complete the space provided to specify or
briefly describe the matter. If necessary, use all the whfte space next
to a question for a description. H
B-8
-------
-------
Description
Automatic Tank Gauging System
system or how the equipment works.
ATGS Name and Version
Product
> Product type
For what products can this ATGS be used? (check all applicable)
D gasoline
D diesel
D aviation fuel
D fuel oil #4
D fuel oil #6
CD solvents
CH waste oil
D other (list)
> Product level
What product level is required to conduct a test?
CH greater than 90% full
D greater than 50% full
D other (specify)
Does the ATGS measure inflow of water as well as loss of product (gallon per hour)?
D yes
Dno
Does the ATGS detect the presence of water in the bottom of the tank?
D yes
Dno
ATGS - Description Page 1 of 6
-------
Level Measurement
What technique is used to measure changes in product volume?
D directly measure the volume of product change !
D changes in head pressure
CD changes in buoyancy of a probe
D mechanical level measure (e.g., ruler, dipstick)
D changes in capacitance
D ultrasonic ;
D change in level of float (specify principle, e.g., capacitance, magnetostrictive,
load cell, etc.) ' .
CD other (describe briefly)
Temperature Measurement ,
If product temperature is measured during a test, how many temperature sensors are
used?
EH single sensor, without circulation ;
HU single sensor, with circulation
D 2-4 sensors
D 5 or more sensors
D temperature-averaging probe
If product temperature is measured during a test, what type of temperature sensor is used?
D resistance temperature detector (RTD)
CI bimetallic strip
D quartz crystal
CH thermistor
D other (describe briefly)
If product temperature is not measured during a test, why not?
D the factor measured for change in level/volume is independent of temperature
(e.g., mass)
D the factor measured for change in level/volume self-compensates for changes in
temperature
D other (explain briefly) .
ATGS - Description Page 2 of 6
-------
Data Acquisition
How are the test data acquired and recorded?
EH manually
D by strip chart
[U by computer
Procedure Information
> Waiting times
What is the minimum waiting period between adding a large volume of product (i e a
delivery) and the beginning of a test (e.g., filling from 50% to 90-95% capacity)?
CH no waiting period
d less than 3 hours
D 3-6 hours
D 7-12 hours
D more than 12 hours
LJ variable, depending on tank size, amount added, operator discretion, etc.
> Test duration
What is the minimum time for collecting data?
D less than 1 hour
D 1 hour
EH 2 hours
D 3 hours
C] 4 hours
CH 5-10 hours
D more than 10 hours
D variable (explain)
> Total time
What is the total time needed to test with this ATGS after a delivery?
(waiting time plus testing time)
hours minutes
ATGS - Description Page 3 of 6
-------
What is the sampling frequency for the level and temperature measurements?
EU more than once per second
D at least once per minute ,
CH every 1-15 minutes :
D every 16-30 minutes
D every 31-60 minutes
CD less than once per hour
EH variable (explain) ;
> Identifying and correcting for interfering factors
How does the ATGS determine the presence and level of the ground water above the
bottom of the tank?
CD observation well near tank
D information from USGS, etc.
D information from personnel on-site
D presence of water in the tank
CD other (describe briefly)
D level of ground water above bottom of the tank not determined '
How does the ATGS correct for the interference due to the presence of ground water
above the bottom of the tank?
CH system tests for water incursion
D different product levels tested and leak rates compared
EH other (describe briefly)
D no action
How does the ATGS determine when tank deformation has stopped following delivery of
product?
D wait a specified period of time before beginning test
D watch the data trends and begin test when decrease in product level has stopped
CD other (describe briefly) ;
CD no procedure ,
ATGS - Description Page 4 of 6
-------
Are the temperature and level sensors calibrated before each test?
D yes
Dno
If not, how frequently are the sensors calibrated?
D weekly
D monthly
D yearly or less frequently
CH never
> Interpreting test results
How are level changes converted to volume changes (i.e., how is height-to-volume
conversion factor determined)? vuiumo
D actual level changes observed when known volume is added or removed (e a
liquid, metal bar) ^ a->
D theoretical ratio calculated from tank geometry
D interpolation from tank manufacturer's chart
D other (describe briefly)
LJ not applicable; volume measured directly
How is the coefficient of thermal expansion (Ce) of the product determined?
D actual sample taken for each test and Ce determined from specific gravity
D value supplied by vendor of product
D average value for type of product
D other (describe briefly)
How is the leak rate (gallon per hour) calculated?
D average of subsets of all data collected
D difference between first and last data collected
D from data from last hours of test period
D from data determined to be valid by statistical analysis
D other (describe briefly)
ATGS - Description _. _ ,
Page 5 of 6
-------
What threshold value for product volume change (gallon per hour) is used to declare that
a tank is leaking?
CH 0.05 gallon per hour ,
D 0.10 gallon per hour ;
D 0.20 gallon per hour
D other (list). '
Under what conditions are test results considered inconclusive?
D too much variability in the data (standard deviation beyond a given value)
D unexplained product volume increase
D other (describe briefly)
Exceptions
Are there any conditions under which a test should not be conducted?
[U water in the excavation zone
D large difference between ground temperature and delivered product temperature
D extremely high or low ambient temperature , •
D invalid for some products (specify) !
other (describe briefly) ;
What are acceptable deviations from the standard testing protocol?
EH none
D lengthen the duration of test
D other (describe briefly)
What elements of the test procedure are determined by personnel on-site?
D product level when test is conducted
C] when to conduct test
D waiting period between filling tank and beginning test :
D length of test ;
D determination that tank deformation has subsided ;
D determination of "outlier" data that may be discarded
D other (describe briefly) '
C3 none
ATGS - Description Page 6 of 6
-------
Reporting Form for Leak Rate Data
Automatic Tank Gauging System (ATGS)
Instructions for completing the form
This 1- or 2-page form is to be filled out by the evaluating organization
upon completion of the evaluation of the ATGS in its leak detection
mode. A single sheet provides for 24 test results, the minimum number of
tests required in the protocol. Use as many pages as necessary to
summarize all of the tests attempted.
Indicate the commercial name and the version of the ATGS and the period
of evaluation above the table. The version is provided for ATG systems
that use different versions of the equipment for different products or
tank sizes.
In general, the statistician analyzing the data will complete this
form. A blank form can be developed on a personal computer, the data
base for a given evaluation generated, and the two merged on the com-
puter. The form can also be filled out manually. The input for that form
will consist of the field test results recorded by the evaluating
organization's field crew on the Individual Test Logs and the ATGS test
results.
The table consists of 11 columns. One line is provided for each test
performed during evaluation of the ATGS. If a test was invalid or was
aborted, the test should be listed with the appropriate notation (e.q.
invalid) on the line.
The Test Number in the first column refers to the test number from the
randomization design determined according to the instructions in Sec-
tion 6.1 of the protocol. Since some changes to the design might occur
during the course of the field testing, the test numbers might not always
be in sequential order.
Note that the results from the trial run need to be reported here as
well.
The following list matches the column input required with its source, for
each column in the table.
B-15
-------
Column No.
1
2
3
4
5
6
7
8
9
10
11
Test number or trial run
Date at completion of last fill
Time at completion of last fill
Date test began
Time test began
Time test ended
Product temperature differential
Nominal leak rate
Induced leak rate
Measured leak rate
Measured minus induced leak rate
Source
Randomization design
Individual Test Log
Individual Test Log
Individual Test Log
Individual Test Log
Individual Test Log
Individual Test Log
Randomization design
Individual Test Log
AT6S records
By subtraction
The product temperature differential (column 7) is the difference between
the temperature of the product added and that of the product in the tank
each time the tank is filled from 50% full to between 90% :to 95% full.
This temperature differential is the actual differential achieved in the
field and not the nominal temperature differential. The difference can
be calculated by one of two methods. If the field crew measured the
temperature of the product added and that of the product in the tank just
prior to filling, then take the difference between these two tempera-
tures. If the field crew measured the temperature of the product in the
tank before and after filling and recorded the amount of product added,
then calculate the temperature differential based on volumes and tempera-
tures according to the formula in Section 7.4 The data necessary for
these calculations should all be provided on the Individual Test Log.
B-16
-------
Reporting Form for Leak Rate Data
Automatic Tank Gauging System (ATGS)
ATGS Name and Version-
Evaluation Period:
from
to
. (Dates)
Test No.
Trial Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Date at
Completion
of Last Fill
(m/d/y)
y....s. '
Time at
Completion
of Last Fill
(military)
Date Test
Began
(m/d/y)
Time Test
Began
(military)
Time Test
Ended
(military)
Product
Temperature
Differential
(deg F)
0
Nominal
Leak Rate
(gal/h)
0
Induced
Leak Rate
(gal/h)
0
, - ' ',- ' * . i
Measured
Leak Rate
(gal/h)
Meas.-lnd.
Leak Rate
(gal/h)
ATGS-Data Reporting Form
Pagel of 2
-------
Reporting Form for Leak Rate Data
Automatic Tank Gauging System (ATGS)
ATGS Name and Version:
Evaluation Period:
from
to.
.(Dates)
Test No.
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Date at
Completion
of Last Fill
(m/d/y)
Time at
Completion
of Last Fill
(military)
Date Test
Began
(m/d/y)
Time Test
Began
(military)
Time Test
Ended
(military)
Product
Temperature
Differential
(degF)
Nominal
Leak Rate
(gal/h)
Induced
Leak Rate
(gal/h)
Measured
Leak Rate
(gal/h) .
.
Meas.-lnd.
Leak Rate
(gal/h)
ATGS-Data Reporting Form
Page 2 of 2
-------
Individual Test Log
Automatic Tank Gauging System (ATGS)
Instructions for completing the form
This 5-page test log form is to be filled out by the field crew of the
evaluating organization. A separate form is to be filled out for each
individual test including the trial run (at least 25.) The information
on these forms is to be kept blind to the vendor during the period of
evaluation of the ATGS.
The form consists of eight parts. These are:
1. Header information
2. General background information
3. Conditions before testing
4. Conditions at beginning of test
5. Conditions at completion of testing
6. Leak rate data
7. Additional comments, if needed
8. Induced leak rate data sheets
All items are to be filled out and the appropriate boxes checked If a
question is not applicable, then indicate so as "NA". The following
provides -guidance on the use of this form.
Header Information
The header information is to be repeated on all five pages, if used If
a page is not used, cross it out and initial it. The field operator from
the evaluating organization needs to print and sign his/her name and note
the date of the test on top of each sheet.
The test number is the number obtained from the randomization design. It
is not the sequential running test number. If a test needs to be rerun
indicate the test number of the test being rerun and indicate that on the
test log (e.g., Test No. 5 repeat).
General Background Information
Indicate the commercial name of the ATGS. Include a version identifica-
tion if the ATGS uses different versions for different products or tank
sizes. The vendor's recommended stabilization period (if applicable) has
to be obtained from the vendor prior to testing. This is important since
it will impact on the scheduling of the evaluation. All other items in
this section refer to the test tank and product. Indicate the qround-
water level at the time of the test.
Theoretically, this information would remain unchanged for the whole
evaluation period. However, weather conditions could change and affect
the ground-water level. Also, the evaluating organization could chanqe
the test tank. 3
B-19
-------
Conditions Before Testing
Fill in all the blanks. If the information is obtained by calculation
(for example the amount of water in the tank is obtained from the stick
reading and then converted to volume), this can be done after the test is
completed. Indicate the unit of all temperature measurements by checking
the appropriate box. j
Note that the term "conditioning" refers to all activities undertaken by
the evaluating field crew to prepare for a test. As such, ;the term
refers to emptying or filling the tank, heating or cooling product, and
changing the leak rate. In some cases, all of the above is performed, in
others, only one parameter might be changed. \
Special Case Reporting :
Use the Individual Test Log form to record all data pertaining to the
trial run. Next, when emptying the tank to half full and then filling to
90% to 95% capacity before performing the first test, note on the form
that this has been done. Simply indicate on page 1 the dates and time
periods and volumes when product was removed and then added. This is the
only case where emptying and filling are performed in sequence without a
test being performed in between. Record all other information (e.g.,
temperature of product added) as applicable.
j
Conditions at Beginning of Test
The evaluating organization's field crew starts inducing the leak rate
and records the time on pages 4 and 5. All leak simulation data are to
be recorded using the form on pages 4 and 5.
Once the evaluating organization's field crew is ready with the induced
leak rate simulation, and the ATGS starts the actual testing, record the
date and time that the ATGS test data collection starts. Also, indicate
the product temperature at that time. Fill out the weather condition
section of the form. Indicate the nominal leak rate which :is obtained
from the randomization design. •
Conditions at Completion of Testing !
Indicate date and time when the test is completed.
Again, stick the tank and record the readings and the amount of water in
the tank. Record all weather conditions as requested.
Leak Rate Data
This section is to be filled out by the evaluating organization's
statistician or analyst performing the calculations. This section can
therefore be filled out as the evaluation proceeds or at the end of the
evaluation.
B-20
-------
The nominal leak rate is obtained from page 2 (Conditions at Beginning of
Test). It should be checked against the nominal leak rate in the
randomization design by matching test numbers.
The induced leak rate is obtained by calculation from the data reported
by the evaluating field crew ori page 4 (and 5, if needed) of this form.
The measured leak rate is that recorded by the ATGS for that test.
The difference is simply calculated by subtracting the induced from the
measured leak rate.
Additional Comments (If needed)
Use this page for any comments (e.g., adverse weather conditions,
equipment failure, reason for invalid test, etc.) pertaining to that
test.
Induced Leak Rate Data (pages 4 and 5)
This form is to be filled out by the evaluating organization's field
crew. From the randomization design, the crew will know the nominal leak
rate to be targeted. The induced leak rate will be known accurately at
the end of the test. However, the protocol requires that the induced leak
rate be within 30% of the nominal leak rate.
B-21
-------
-------
Name of Field Operator
Signature of Field Operator_ jest No.
Date of Test
Individual Test Log
Automatic Tank Gauging System (ATGS)
Instructions:
Use one log for each test.
Fill in the blanks and check the boxes, as appropriate.
Keep test log even if test is inconclusive.
General Background Information
ATGS Name and Version
Product Type
Type of Tank
Tank Dimensions (nominal)
Diameter inches
Length inches
Volume gallons
Ground-water level inches above bottom of tank
if applicable, recommended stabilization period before test (per vendor SOP)
hours minutes
Conditions Before Testing
Date and time at start of conditioning test tank date military time
Stick reading before conditioning test tank
Product inches gallons
Water inches gallons
Temperature of product in test tank before conditioning °F D or °C D
Stick reading after conditioning test tank
Product inches gallons
Amount of product (check one only):
CH no change in product level
D removed from tank (by subtraction) gallons
D added to tank (by subtraction) gallons
ATGS-Test Log Page 1 of 5
-------
Name of Field Operator
Signature of Field Operator_
Date of Test
Test No.
Conditions Before Testing (continued)
If product was added
Temperature of product added to fill test tank to test level
•' °FD or°cD
Temperature of product in tank immediately after filling
Date and time at completion of conditioning date
BF I
] or°cD .
..military time
Conditions at Beginning of Test
Date and time at start of ATGS test data collection
date military time ;
> Complete the induced leak rate data sheet (use attached pages 4 and 5)
Temperature of product in tank at start of test °FD or °cD '
i
Weather conditions at beginning of test
Temperature °FD or°cD :
Barometric pressure mm Hg CH or in. Hg EU i
Wind None EH Light D Moderate d Strong CD
Precipitation NoneD Light D Moderate D Heavy D
Sunny CD Partly cloudy D Cloudy D
Nominal leak rate gallon per hour I
ATGS - Test Log
Page 2 of 5
-------
Name of Field Operator
Signature of Field Operator jest No.
Date of Test
Conditions at Completion of Testing .
Date and time at completion of test data collection
date military time
Stick reading at completion of test data collection
Product inches gallons
Water inches gallons
Weather Conditions at End of Test
Temperature °F D or °C D
Barometric pressure mm Hg D or in. Hg D
w'nd NoneD Light D Moderate D Strong D
Precipitation NoneD Light D Moderate D Heavy D
Sunny D Partly Cloudy D Cloudy D
Leak Rate Data (not to be filled out by field crew)
Nominal leak rate gal/h
induced leak rate gal/h
Leak rate measured by vendor's method gal/h
Difference (measured rate minus induced rate) gal/h
Additional Comments (Use back of page if needed)
ATGS- Test Log Page 3 of 5
-------
Name of Field Operator
Signature of Field Operator.
Date of test
Test No,
Induced Leak Rate Data Sheet
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Time at
product
collection
(military)
Amount of
product
collected
(mL)
Comments (if applicable)
,
•
•
i
i
i
;
ATGS-Test Log
Page 4 of 5
-------
Name of Field Operator
Signature of Field Operator
Date of test
Induced Leak Rate Data Sheet (continued)
Test No.
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Time at
product
collection
(military)
Amount of
product
collected
(mL)
Comments (if applicable)
ATGS-Test Log
Page 5 of 5
-------
-------
Reporting Form for Water Sensor Evaluation Data
Automatic Tank Gauging System
This 4-page form is to be filled out by the field crew of the evaluating
organization when evaluating the performance of the AT6S water sensor A
separate form is to be filled out for each individual test replicate (at
least 20). The form provides a template to record the data and consists
of three parts. These are:
1. Header information
2. Template for recording the data obtained to determine the minimum
water level that the sensor can detect in each replicate (page 1)
3. Template for recording the data obtained when determining the
minimum water level change that the sensor can detect in each
replicate (pages 2-4).
Header Information
The header information is to be repeated on all four pages, if used If
a page is not used, cross it out and initial it.
Indicate the commercial name of the ATGS. Include a version identifica-
tion if the ATGS uses different versions for different products or tank
sizes. Complete the date of test and product type information. Indicate
the test (replicate) number on each sheet for each test.
The field operator from the evaluating organization needs to print and
sign his/her name and note the date of the test on top of each sheet.
Minimum Detectable Water Level Data
Follow the test protocol described in Section 6.4 and record all data on
page 1 of the form. When the sensor first detects the water, stop test-
ing for this replicate. The minimum detected water level is calculated
from the total amount of water added until the first sensor response and
the geometry of the probe and the cylinder. This calculation can be done
after all testing is completed and is generally performed by the statis-
tician or other person responsible for data analysis.
Minimum Detectable Water Level Change
After the first sensor response, continue with the test protocol as
described in Section 6.4. Record all amounts of water added and the
sensor readings at each increment using pages 2 to 4 as necessary. The
data to be entered in the third, fifth, and sixth columns on pages 2, 3,
and 4 of the form will be calculated once all testing is completed.
Again, the person responsible for the data analysis will generally
compute these data and enter the calculated minimum water level detected
in that replicate run.
B-27
-------
-------
Reporting Form for Water Sensor Evaluation Data
Automatic Tank Gauging System
ATGS Name and Version:
Date of Test:
Product Type:
Name of Field Operator:.
Signature of Field Operator:
Increment
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Total
Volume
(mL)
Volume of
Water Added
(mL)
Sensor
Reading
(inch)
Test No.
Calculated Minimum
Detectable Water Level (inches)
NOTE:
This form provides a template for data reporting. Since the number of
increments is not known from the start, the length of the report form
will vary from test to test.
ATGS-Water Sensor
Page 1 of 4
-------
ATGS Name and Version:
Date of Test:
Product Type:
Reporting Form for Water Sensor Evaluation Data
Automatic Tank Gauging System;
Name of Field Operator:
Signature of Field Operator: __
Test No.
Increment
No.
A
Volume of
Water Added
(mL)
B
Calculated
Water Height
Increment, h
(in)
C
Sensor
Reading
(in)
D
Measured
Sensor
Increment
(in)
E i
Increment
Difference
Calc.-Meas.
(in)
C-E
Minimum water level detected, X: inches (from page 1)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
I
•
'
I
i
.
i
i
i
i
\
\
, I
;
NOTE: This form provides a template for data reporting.
Use as many pages as necessary.
ATGS-Water Sensor
Page 2 of 4
-------
Reporting Form for Water Sensor Evaluation Data
Automatic Tank Gauging System
ATGS Name and Version:
Date pf Test: .
Product Type:
Name of Field Operator:.
Signature of Reid Operator:
Test No.
Increment
No.
A
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Volume of
Water Added
(mL)
B
Calculated
Water Height
Increment, h
(in)
C
Sensor
Reading
(in)
D
Measured
Sensor
Increment
(in)
E
Increment
Difference
Calc.-Meas.
(in)
C-E
NOTE: This form provides a template for data reporting.
Use as many pages as necessary.
ATGS-Water Sensor
Page 3 of 4
-------
Name of ATGS_
Version
Limitations on the Results (continued)
• The difference between added and in-tank product temperatures is no greater
than + or- _degrees Fahrenheit. ' i
• The waiting time between the end of filling and the start of the test data collection
is at least hours.
• The total data collection time for the test is at least hours.
> Safety disclaimer: This test procedure only addresses the issue of th© ATG system's
ability to detect leaks. It does not test the equipment for safety hazards.
Certification of Results
I certify that the ATGS was installed and operated according to the vendor's instructions and
that the results presented on this form are those obtained during the evaluation. I also certify
that the evaluation was performed according to one of the following:
ED standard EPA test procedure for ATGS
EH alternative EPA test procedure for ATGS
ED equivalent test procedure for ATGS (describe below or reference document)
(printed name)
(signature)
(organization performing evaluation) (city, state)
(date) (phone number)
ATGS - Results Form , Page 2 of 2
-------
-------
------- |