Batreiie
The Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Test/QA Plan for Verification of
Leak Detection and Repair Technologies
-------
Test/QA Plan
For
Verification of
Leak Detection and Repair Technologies
September 18, 2008
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 3 of 46
Version 1.0
Date: 9/18/2008
TABLE OF CONTENTS
Section Page
SECTION A PROJECT MANAGEMENT 8
Al Verification Test Organization 8
A2 Background 16
A3 Verification Test Description and Schedule 17
A4 Quality Objectives 21
A5 Special Training/Certification 22
A6 Documentation and Records 22
SECTION B MEASUREMENT AND DATA ACQUISITION 24
Bl Experimental Design 24
B2 Sampling Requirements 32
B3 Sample Handling and Custody Requirements 32
B4 Reference Method 33
B5 Quality Control Criteria for Reference Method Measurement Data 34
B6 Instrument/Equipment Testing, Inspection, and Maintenance 37
B7 Instrument Calibration and Frequency 37
B8 Inspection/Acceptance of Supplies and Consumables 37
B9 Non-Direct Measurements 38
BIO Data management 38
SECTION C ASSESSMENT AND OVERSIGHT 40
Cl Assessments and Response Actions 40
C2 Reports to Management 42
SECTION D DATA VALIDATION AND USABILITY 43
Dl Data Review, Validation, and Verification Requirements 43
D2 Validation and Verification Methods 43
D3 Reconciliation with User Requirements 44
SECTION E REFERENCES 46
El References 46
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 4 of 46
Version 1.0
Date: 9/18/2008
List of Figures Page
Figure 1. Organization Chart for the Verification Test 9
List of Tables
Table 1. Planned Verification Test Schedule 20
Table 2. Summary of Data Recording Process 23
TableS. Laboratory Testing Matrix for Each Chemical of Interest 26
Table 4. Chemicals of Interest in Laboratory Testing 26
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 5 of 46
Version 1.0
Date: 9/18/2008
Test/QA Plan for Verification of
Leak Detection and Repair Technologies
September 18, 2008
APPROVAL:
Name
Company
Date
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 6 of 46
Version 1.0
Date: 9/18/2008
ACRONYMS AND ABBREVIATIONS
AMS Advanced Monitoring Systems
COC Chain-of-custody
DQI Data quality indicator
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
FEM Fugitive emission monitoring
GC Gas chromatograph
IR Infrared
LDAR Leak detection and repair
LOD Limit of Detection
LRB Laboratory record book
MDL Method detection limit
pdf Adobe portable document format
PE Performance evaluation
QA Quality assurance
QC Quality control
QCS Quality control samples
QMP Quality management plan
r2 Coefficient of determination
SOP Standard operating procedures
TSA Technical systems audit
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 7 of 46
Version 1.0
Date: 9/18/2008
DISTRIBUTION LIST
Vendors
Paul Czerepuszko
FLIR SYSTEMS
25 Esquire Road
North Billerica. MA 01862
Michele Hinnrichs
Pacific Advanced Technology
85 Industrial Way
Buellton. CA 93427
EPA QA Officer
Lauren Drees
U.S. Environmental Protection Agency -
26 West Martin Luther King Dr.
Mail Code: N132C
Cincinnati, OH 45268
EPA Project Officer
Douglas Grosse
U.S. Environmental Protection Agency -
26 West Martin Luther King Dr.
Mail Code: 208A
Cincinnati, OH 45268
Test Collaborators
David Williams
U.S. Environmental Protection Agency -
E243-05EPAMailroom
Research Triangle Park, NC 27711
Dave Fashimpaur
BP Refining Technology
150 W. Warrenville Rd. (MC 605-1W)
Naperville, IL 60563 USA
Christina T. Wisdom
Vice President & General Counsel
Texas Chemical Council
1402 Nueces Street
Austin, Texas 78701-1586
Julie Woodard
EH&S Regulatory Affairs Manager - Texas
The Dow Chemical Company
2301 N. Brazosport Blvd., B-101
Freeport, TX 77541
Jerry Duke
LDAR Environmental Team Leader
BP - Texas City
2501 Palmer Highway, Suite 200
Texas City, Texas 77590
Mark Leidig
Field Operations Support Division
TCEQ/MC-174
PO Box 13087
Austin, TX 78711-3087
Michael P. Walls
Director, Regulatory and Technical Affairs
American Chemistry Council
1300 Wilson Boulevard
Arlington, VA 22209
Battelle
Amy Dindal
Brian Boczek
Zachary Willenberg
Battelle
505 King Ave.
Columbus, OH 43201
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 8 of 46
Version 1.0
Date: 9/18/2008
SECTION A
PROJECT MANAGEMENT
Al VERIFICATION TEST ORGANIZATION
The verification test will be conducted under the auspices of the U.S. Environmental
Protection Agency (EPA) through its Environmental Technology Verification (ETV) Program. It
will be performed by Battelle, which is managing the ETV Advanced Monitoring Systems
(AMS) Center through a cooperative agreement with EPA. The scope of the AMS Center covers
verification of monitoring technologies for contaminants and natural species in air, water, and
soil.
The day to day operations of this verification test will be coordinated and supervised by
Battelle, with the participation of the vendors who will be having the performance of their
technologies for leak detection and repair (LDAR) verified. Testing will be conducted at a
laboratory (BP laboratory in Naperville, IL), a petrochemical plant in the Gulf Coast region of
Texas and a chemical plant also in the Gulf Coast region of Texas. Staff from these host
facilities will support this test by providing infrastructure, coordination and overseeing the
testing at their respective site. Each LDAR vendor will provide two units of their respective
technologies, operate the technology through portions of the test (unless they give written
consent and training for host facility staff to operate it), and repair or maintain their technology
during the test.
The organization chart in Figure 1 identifies the responsibilities of the organizations and
individuals associated with the verification test. Roles and responsibilities are defined further
below. Quality Assurance (QA) oversight will be provided by the Battelle Quality Manager and
also by the EPA AMS Center Quality Manager, at her discretion.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 9 of 46
Version 1.0
Date: 9/18/2008
Battelle
Management
Zachary Willenberg
Battelle AMS Center
Quality Manager
Amy Dindal
AMS Center
Manager/
Verification
Testing Leader
Brian Boczek
Battelle
Verification
Test Coordinator
Battelle and
Host Site
Technical Staff
Douglas Grosse
EPA AMS Center
Project Officer
Lauren Drees
EPA AMS Center
Quality Manager
(acting)
LDAR
Technology
Vendor
Representatives
Figure 1. Organization Chart for the Verification Test
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 10 of 46
Version 1.0
Date: 9/18/2008
Al.l Battelle
Mr. Brian Boczek is the AMS Center's Verification Test Coordinator for this test. In this
role, Mr. Boczek will have overall responsibility for ensuring that the technical, schedule, and
cost goals established for the verification test are met. Specifically, Mr. Boczek will:
Prepare the test/QA plan, verification reports, and verification statements.
Revise the test/QA plan, verification reports, and verification statements in response
to reviewers' comments.
Establish a budget for the verification test and manage staff to ensure the budget is
not exceeded.
Coordinate and communicate with host sites and subcontractor.
Assemble a team of qualified technical staff to conduct the verification test.
Direct the team in performing the verification test in accordance with this test/QA
plan.
Hold a kick-off meeting approximately one week prior to the start of the verification
test to review the critical logistical, technical, and administrative aspects of the
verification test. Responsibility for each aspect of the verification test will be
confirmed.
Ensure that all quality procedures specified in this test/QA plan and in the AMS
Center Quality Management Plan1 (QMP) are followed.
Serve as the primary point of contact for vendor representatives.
Ensure that confidentiality of sensitive vendor information is maintained.
Work with host sites to ensure protection of site confidential business information.
Assist vendors as needed during verification testing.
Become familiar with the operation and maintenance of the technologies through
instruction by the vendors, if needed.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 11 of 46
Version 1.0
Date: 9/18/2008
Respond to any issues raised in assessment reports, audits, or from test staff
observations, and institute corrective action as necessary.
Where applicable to the host site, ensure that all contractors are approved with the site
prior to entering the site. This may require completion of certain courses and passing
a drug test.
Ensure that all guests to host site (contractors, technical staff, etc.) comply with site
health and safety requirements.
Coordinate distribution of the final test/QA plan, verification reports, and verification
statements.
Ms. Amy Dindal will serve as Verification Testing Leader and is also Battelle's Manager
for the AMS Center. Ms. Dindal will:
Support Mr. Boczek in preparing the test/QA plan and organizing the testing.
Review the final test/QA plan.
Attend the verification test kick-off meeting.
Review the draft and final verification reports and verification statements.
Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test.
Ensure that confidentiality of sensitive vendor information is maintained.
Support Mr. Boczek in responding to any issues raised in assessment reports and
audits.
Maintain communication with EPA's technical and quality managers.
Issue a stop work order if Battelle or EPA QA staff discover adverse findings that
will compromise test results.
Battelle Technical Staff will support Mr. Boczek in planning and conducting the
verification test. The responsibilities of the technical staff will be to:
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 12 of 46
Version 1.0
Date: 9/18/2008
Assist in planning for the test and making arrangements for the receipt of the
technologies.
Perform statistical calculations specified in this test/QA plan on the technology data
as needed.
Provide results of statistical calculations and associated discussion for the verification
reports as needed.
Support Mr. Boczek in responding to any issues raised in assessment reports and
audits related to statistics and data reduction as needed.
Mr. Zachary Willenberg is Battelle's Quality Manager for the AMS Center.
Mr. Willenberg will:
Review the draft and final test/QA plan.
Attend the verification test kick-off meeting.
Conduct a technical systems audit at least once during the verification test, or
designate other QA staff to conduct the audit.
Audit at least 10% of the verification data or designate other QA staff to conduct the
data audit.
Prepare and distribute an assessment report for each audit.
Verify implementation of any necessary corrective action.
Request that Battelle's AMS Center Manager issue a stop work order if audits
indicate that data quality is being compromised.
Provide a summary of the QA/QC activities and results for the verification reports.
Review the draft and final verification reports and verification statements.
A1.2 Technology Vendors
The responsibilities of the technology vendors are as follows:
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 13 of 46
Version 1.0
Date: 9/18/2008
Ian.
Review and provide comments on the draft test/QA ph.__.
Accept (by signature of a company representative) the final test/QA plan prior to test
initiation.
Provide two units of their technology for evaluation during the verification test.
Provide all other equipment/supplies/reagents/consumables needed to operate their
technology for the duration of the verification test.
Provide an operator for the duration of the test, or supply training on the use of the
technology and written consent and instructions for test staff to carry out verification
testing, including written instructions for routine operation of their technology.
Provide maintenance and repair support for their technology, on-site if necessary,
throughout the duration of the verification test.
Review and provide comments on the draft verification report and statement for their
respective technology.
A1.3 EPA
EPA's responsibilities in the AMS Center are based on the requirements stated in the
"Environmental Technology Verification Program Quality Management Plan" (EPA QMP).2 The
roles of specific EPA staff are as follows:
Ms. Lauren Drees is EPA's acting AMS Center Quality Manager. For the verification
test, Ms. Drees or her designee will:
Review the draft test/QA plan.
Perform at her option one external technical systems audit during the verification test.
Notify the EPA AMS Center Project Officer of the need for a stop work order if the
external audit indicates that data quality is being compromised.
Prepare and distribute an assessment report summarizing results of the external audit.
Review draft verification reports and verification statements.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 14 of 46
Version 1.0
Date: 9/18/2008
Mr. Douglas Grosse is EPA's Project Officer for the AMS Center. Mr. Grosse will:
Review the draft test/QA plan.
Approve the final test/QA plan.
Review the draft verification reports and verification statements.
Oversee the EPA review process for the test/QA plan, verification reports, and
verification statements.
Coordinate the submission of verification reports and verification statements for final
EPA approval.
A1.4 BP Naperville, IL Test Facility
This test will be conducted in collaboration with the BP Naperville, IL Test Facility, who
will provide in-kind support for this test. The responsibilities of personnel from the BP
Naperville, IL Test Facility include the following:
Coordinate use of the test site for the purposes of ETV testing, assuring access to the
test site for Battelle, EPA, and vendor representatives.
Support the test by providing facilities and needed utilities (e.g., laboratory space, test
apparatus, electricity, and compressed gases) for the technologies and reference
methods during testing.
Following BP safety requirements, conduct a Process Safety Management Review.
Assist Battelle and vendor staff in the operation and testing of the technologies at the
test site.
Perform testing activities and data acquisition, as needed, as specified in this test/QA
plan.
Review the draft verification reports and verification statements.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 15 of 46
Version 1.0
Date: 9/18/2008
A1.5 Petrochemical Plant and Chemical Plant
This test will be conducted in collaboration with the petrochemical plant and chemical
plant, both located in the Gulf Coast region of Texas, who will provide in kind support for this
test. The responsibilities of personnel from these two sites include the following:
Coordinate use of the test site for the purposes of ETV testing, assuring access to the
test site for Battelle, EPA, subcontractor, and vendor representatives.
Support the test by providing facilities and needed utilities (e.g., work space,
electricity) for the technologies and reference methods during testing.
Identify components or portions of the plant appropriate for use as field testing
venues for the LDAR technologies.
Assist Battelle and vendor staff in the operation and testing of the technologies at the
test site.
Perform testing activities and data acquisition, as needed, as specified in this test/QA
plan.
Collect and report basic meteorological data (e.g., wind speed, wind direction,
temperature, and relative humidity) at the test site during the field period.
Review the draft verification reports and verification statements.
Provide contractors with necessary health and safety training prior to field testing, as
specifically required by the sites.
A1.6 Sage Environmental Consulting Richardson, TX
Quantification of the leakage rates of leaks found during testing will be established by
enclosing (or bagging) the leaking components and analyzing the captured samples. The
reference method sample collection and analysis will be subcontracted to Sage Environmental
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 16 of 46
Version 1.0
Date: 9/18/2008
Consulting from Richardson, Texas. This firm has experience in performing these analyses for
similar testing conducted in the past. They will follow the EPA reference methods with the
variations stated in Section B4. The responsibilities of personnel performing the reference
analysis include the following:
Coordinate with Battelle, host site, and vendor personnel in carrying out the field
testing procedures.
Perform the on-site reference bagging procedure and analytical determination of
leaks.
Provide all supplies necessary to carry out the reference method collection and
analytical determination.
Conduct all QC efforts as specified in this test/QA plan.
Calculate the reference results in terms of leak rate and provide a data package to
Battelle that includes all sampling data sheets, analysis records, calibration data, and
QA/QC information, and that presents the reference sample analysis results.
A2 BACKGROUND
The ETV Program conducts third-party performance testing of commercially available
technologies that detect or monitor natural species or contaminants in air, water, and soil. The
purpose of ETV is to provide objective and quality assured performance data on environmental
technologies, so that users, developers, regulators, and consultants can make informed decisions
about purchasing and applying these technologies. Stakeholder committees of buyers and users
of such technologies guided the development of this test on LDAR technologies.
The purpose of this test/QA plan is to specify procedures for a verification test applicable
to commercial LDAR technologies. The purpose of the verification test is to evaluate the
performance of participating technologies in both field and laboratory environments. In
performing the verification test, Battelle will follow the technical and QA/QC procedures
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 17 of 46
Version 1.0
Date: 9/18/2008
specified in this test/QA plan and will comply with the data quality requirements in the AMS
Center QMP.1
A3 VERIFICATION TEST DESCRIPTION AND SCHEDULE
A3.1 Summary of Technology Category
Currently in the United States and in other industrialized countries, fugitive emission
monitoring (FEM) programs are based on U. S. EPA Method 213 which involves the use of a
portable hydrocarbon analyzer to monitor for leaks at industrial component interfaces such as
flanges, couplings, and valves. Current fugitive emission monitoring methods such as organic
vapor analyzers, bubblers, and ultrasonic leak detection equipment, have traditionally been used
in accordance with Method 21. Method 21 requires sampling of each of the numerous
components of a plant that may leak. While thorough, this procedure is costly because it requires
a large amount of operator effort. In addition, since most of the fugitive emissions come from a
few large leaks, the practice of individually checking each potential source is not very efficient.
The actual number of components to be tested in a refinery or chemical plant can be quite large,
making Method 21 monitoring both time intensive and expensive.
A class of technology, generally referred to as optical imagers or leak detection and
repair (LDAR) devices, offers operators the ability to monitor components from a distance and
instantaneously identify leaking components within the line of sight of the optical imager. The
remote sensing and instantaneous detection capabilities of optical imaging technologies allow an
operator to scan areas containing many potential leaks, thus eliminating the need to visit and
individually measure all potential leak sites. Significant leaks are identified immediately,
allowing quicker repair, and ensuring efficient use of resources.
Typically, remote monitoring using these technologies can be broken down into two
approaches: active and passive. Active imagers use a powered radiation source to illuminate the
region of interest, while passive imagers rely on differential absorption and scene differences.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 18 of 46
Version 1.0
Date: 9/18/2008
The majority of the passive imagers used in the petrochemical industry rely on detecting infrared
(IR) radiation changes due to the molecular IR transitions of many hydrocarbon species.
Different molecules have IR spectral features at different wavelengths, and imaging different
wavelengths requires different imaging devices or detectors. This evaluation will be testing
passive infrared imaging technologies.
A potential difficulty with using this type of technology in the field is the differing
environments in which it might be used. Since the passive imagers rely on the physical
characteristics of the environment and the molecules being imaged to create an image viewed by
the operator, the environmental characteristics may confound the measurement. For example, if
there is not sufficient thermal emission or absorption by the leaking gas, the imager may not be
able to detect a leak against the ambient thermal background.
The passive technologies that will be evaluated in this verification test will be chosen on
specific criteria as potential alternatives to Method 21. These criteria are:
The technology must be commercially available.
The technology must be portable, i.e., hand-held or able to be carried through the
plant by one person on foot.
The technology must rapidly relay a real-time response. This can be an image on a
screen showing the leak plume or an electronic reading of the relative intensity or
concentration of the leak.
The technology must have wavelength response that allows detection of a variety of
chemicals of importance in the petroleum and petrochemical industries.
The technology must have readily-accessible data collection capability.
The technology must be intrinsically safe in a potentially explosive atmosphere (i.e., a
hydrocarbon plant).
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 19 of 46
Version 1.0
Date: 9/18/2008
A3.2 Verification Test Schedule
Table 1 shows the planned schedule of testing and data analysis/reporting activities to be
conducted in this verification test. As shown in Table 1, preparation to test the technologies will
begin summer and fall of 2008. The preparation includes training on operation of the
technologies, fulfilling any health and safety documentation for the laboratory and field testing
sites, and fulfilling any training requirements for approval to work at the test sites (i.e., safety
training and drug testing). The field and laboratory testing activities are planned to be conducted
from summer/fall 2008 through winter/spring 2009. The period of testing is estimated to cover
six months because of coordination with the laboratory site and with multiple field sites, many
safety requirements, the host site facilities' schedules, and desire to test in both winter and
summer. The laboratory testing will be conducted first and is expected to last approximately one
week. The field testing will follow the laboratory testing and will be scheduled at one site during
the winter and the other site during the summer. Battelle, the subcontractor, and the vendors will
be performing the testing. Following testing, a separate ETV verification report will be drafted
for each LDAR technology. The reports will be reviewed by the technology vendor and
subsequently by peer reviewers selected from the stakeholders, the funding organizations, and
the EPA. The final reports will be submitted to EPA for final signature, and these documents
will be made publicly available on both the EPA/ETV and the Battelle AMS Center websites.
A3.3 Test Sites
Laboratory testing will be conducted at the BP Naperville, IL Research Complex. Field
testing will be conducted at two industrial plants to include one petrochemical plant and one
chemical plant, both located in the Gulf Coast region of Texas. Sample bagging at the field sites
for the reference sample collection will be conducted by a subcontracted laboratory who
typically perform this function at the site. Reference analysis of the bagged samples will be
performed by a sub-contracted analytical laboratory. In performing this verification test at the
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 20 of 46
Version 1.0
Date:9/18/2008
various sites, Battelle will follow the procedures specified in the test/QA plan and will comply
with quality requirements in the AMS Center QMP.1
Table 1. Planned Verification Test Schedule
Completed by
Month/Year
August/Sept.
2008
Oct. 2008 to
Feb. 2009
May 2009
June 2009
July 2009
September 2009
Testing Activities
Technology training by vendor or
coordination with vendor
representative
Coordinate for technologies and
testing supplies to be delivered to
testing sites
Conduct Process Safety
Management Review
Coordinate for all testing personnel
to be approved with the site. This
may require completion of certain
courses and passing a drug test.
Perform testing at laboratory site
(approximately 1 week)
Perform testing at field sites
(approximately 1 week per site)
Calibration of technologies
regularly and as needed
Data Analysis and Reporting
Begin preparation of ETV report template
Compile data from all technologies at all sites
Compile testing environment conditions
Collect and analyze data from reference samples
Analyze and finalize all data
Complete common sections of reports
Prepare draft reports
Internal review of draft reports
Vendor review of draft reports
Revision of draft reports
Peer review of draft reports
Revision of draft reports
Submission of final reports for EPA approval
A3.4 Health and Safety
Battelle will conduct all verification testing and reference measurements following the
safety and health protocols in place for the host facilities. This includes maintaining a safe work
environment and a current awareness of handling potentially toxic chemicals. Exposure to
potentially toxic chemicals will be minimized, personal protective equipment will be worn as
needed, and safe laboratory practices will be followed.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 21 of 46
Version 1.0
Date:9/18/2008
A4 QUALITY OBJECTIVES
In performing the verification test, Battelle will follow the technical and QA procedures
specified in this test/QA plan and will comply with the data quality requirements in the AMS
Center QMP.1 This verification test is designed to evaluate the performance of the LDAR
technologies for detecting gaseous leaks from processing plant piping systems and from
laboratory manifested scenarios. Calibrations of LDAR technologies will follow manufacturer
specified procedures and acceptance criteria. The verification of technology performance will
include a comparison of the technology results to known leak rates produced using calibrated
flow meters and gas cylinders in the laboratory, and to EPA reference method results in the field.
In addition, environmental factors and testing conditions will be documented. The validity of the
execution of the reference method analytical procedures will be checked by a Performance
Evaluation (PE) audit. The Battelle AMS Center Quality Manager or designee will carry out
QA/QC oversight and auditing. This will include a Technical Systems Audit (TSA) and a data
quality audit. The planned audit procedures are described in Section Cl. The EPA (acting)
AMS Center Quality Manager also may conduct an independent TSA, at her discretion.
Data quality objectives indicate the minimum data quality required to meet the LDAR
technology verification objectives. Data quality objectives for this verification test include those
related to the reference method performance, those related to the LDAR technology
performance, and those related to documenting verification testing staff observations. Data
quality objectives for the reference method (see Section B4) are presented in terms of data
quality indicator (DQI) criteria for the critical measurements associated with the reference
method and are listed in Table 5 (Section B5). In the field, the reference method data quality
relies, in part, on proper sample collection and preparation, proper application of the reference
method, and proper maintenance of reference method instrumentation. In the laboratory, the data
quality relies on the known leak rate as calculated by the flow from certified gas cylinders and
calibrated flow controllers.
Battelle will rely on the vendor's data quality objectives for each LDAR technology in
order to insure that the technology is performing properly during testing. The technology data
quality relies on proper operation and maintenance of the LDAR technologies. The results from
these technologies are expected to be qualitative and will be reported as either detect or non-
detect for the test conditions in both the laboratory and the field. A result is reported as detected
when the operator and two confirming individuals all agree that they observe the leak.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 22 of 46
Version 1.0
Date:9/18/2008
A5 SPECIAL TRAINING/CERTIFICATION
Operation of each LDAR technology may be carried out in one of two possible ways.
First, a previously trained vendor representative may operate the technology during testing. In
this scenario, the vendor will verify that the operator is sufficiently trained to successfully utilize
the technology. The second option is that a vendor representative will train a Battelle technical
staff member to operate the technology. This training will be documented and verified by both
the trainer and the trainee, and the vendor will be required to attest that the operator is adequately
trained to carry out the testing. Documentation of training related to technology testing, data
analysis, and reporting is maintained with all testing documentation for the specified period of
time in the AMS Center QMP.1 The Battelle AMS Center Quality Manager may verify the
presence of the appropriate training records prior to the start of testing.
A6 DOCUMENTATION AND RECORDS
The records for this verification test will be contained in the test/QA plan, chain-of-
custody (COC) forms, laboratory record books (LRBs), data collection forms, electronic files
(both raw data and spreadsheets), and the final verification report and assessment reports. All of
these records will be maintained in the Verification Test Coordinator's office or at the field site
during the test and will be transferred to permanent storage at Battelle's Records Management
Office at the conclusion of the verification test, except for assessment reports, which are
permanently stored with the Battelle AMS Center Quality Manager. All Battelle LRBs are stored
indefinitely, either by the Verification Test Coordinator or Battelle's Records Management
Office. EPA will be notified before disposal of any files. Table 2 has further details regarding
the data recording practices and responsibilities. Finally, duplicates will be made of all records
and data collected at the field sites and will be left on premises in the possession of the testing
site.
All written records must be in ink. Any corrections to notebook entries, or changes in
recorded data, must be made with a single line through the original entry. The correction is then
to be entered, initialed, and dated by the person making the correction. In all cases, strict
confidentiality of data from each vendor's technology will be maintained. Separate files
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 23 of 46
Version 1.0
Date:9/18/2008
(including manual records, printouts, and/or electronic data files) will be kept for each vendor's
technology.
In some cases, recorded images might have less resolution than the originally viewed
image of the technology. All images will be evaluated on the direct image from the technology
and not evaluated on a recorded image.
Table 2. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test
events, technology
maintenance, down
time, etc.
Technology
calibration
information
Technology
readings
Sample collection
for reference
method analysis
procedures,
calibrations, QA,
etc.
Reference method
results
Where Recorded
ETV LRBs or data
recording forms
ETV LRBs, data
recording forms, or
electronically
Recorded
electronically by
the technology and
downloaded to an
independent
computer, or hard
copy data printed
by the technology
and taped into the
ETV LRB, or hand
entered into ETV
LRBs or data
recording forms
LRBs, chain-of-
custody, or other
data recording
forms
Electronically or
manually into ETV
LRBs or data
recording forms
How Often
Recorded
Start/end of test
procedure, and at
each change of a
test parameter or
change of
technology status
At technology
calibration or
recalibration
Recorded
continuously for
electronic data,
printed after each
measurement for
hard copy print-
outs, or recorded
manually with each
reading
Throughout
sampling and
analysis processes
Every sample or
QC analysis
By Whom
Battelle or
technology
operator; host
facility
Battelle or
technology
operator
Battelle or
technology
operator
Subcontractors,
Battelle, and others
assisting in
reference sample
collection and
sample analysis
Subcontractors,
Battelle, or other
reference sample
analysis technician
Disposition of
Data
Used to organize
and check test
results; manually
incorporated in
data spreadsheets
as necessary
Incorporated in
verification report
as necessary
Converted to or
manually entered
into spreadsheet
for statistical
analysis and
comparisons
Shared with the
testing site to
facilitate necessary
repairs
Retained as
documentation of
sample collection
or reference
method
performance
Transferred to
spreadsheets for
calculation of
results, and
statistical analysis
and comparisons
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 24 of 46
Version 1.0
Date:9/18/2008
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
LDAR technologies will be tested in a laboratory under controlled conditions and in the
field at petroleum and/or chemical plants. The laboratory tests will allow comparison of the
technology results to known scenarios under a specified set of conditions that eliminate other
confounding variables (humidity, ambient light, exposure to the elements, etc). Field tests will
allow for performance evaluation under "real world" conditions. Overall, the performance of the
LDAR technologies will be verified based on the following factors:
Detection of different chemical gas species relative to a portable monitoring device
(acceptable under Method 21)
Method detection limits for each detectable species
Inter-unit comparability
Influence of confounding factors (wind, background, etc.)
Operational factors (ease of use, sampling time, sampling costs)
The responses to these parameters will be collected as either detect or non-detect from the
technologies; however, a quantitative method will confirm leak flow rates. Test compounds that
a technology proves capable of detecting in the laboratory portion will be tested in the field. The
laboratory test will have known leaks from gas cylinders with calibrated flow meters to calculate
the leak emission rate. The flow rates of the field leaks will be determined by a reference
method, called the bagging method or EPA Protocol for Equipment Leak Emissions Estimates4
This method involves completely enclosing the leak with non-permeable material, collecting the
leak with a flow of clean air or nitrogen entering the bag, and mass measurement of the bagged
leak from an analytical method. More details on the bagging method are provided in Section B4.
The evaluations will be performed according to the vendor's recommended procedures as
described in the user's instructions or manual, or during training provided to the operator. Care
will be taken to use and record the use of the most appropriate lens and zoom features for the
technology. Similarly, calibration and maintenance of the technologies will be performed as
specified by the vendor. The technologies will be evaluated on chemicals observable according
to the vendor stated abilities. Results from the technologies being verified will be recorded
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 25 of 46
Version 1.0
Date:9/18/2008
manually by the operator on appropriate data sheets or captured in an electronic data system and
then transferred manually or electronically for further data workup. The results from each
technology will be reported individually and evaluated from the direct reading and not from the
recorded image. There could be a resolution difference between the original viewing and the
recorded image that will not be tested in this evaluation. No direct comparison will be made
between technologies, but each technology will undergo the same testing so it is convenient for
end users to evaluate the ETV testing results.
Bl.l Test Procedures
The following sections describe the test procedures that will be used to evaluate LDAR
technologies in the laboratory and the field scenarios.
Bl. 1.1 Laboratory Test Procedures
The laboratory LDAR technology evaluation will be conducted at the Naperville BP plant
near Chicago, Illinois. The testing group will consist of lab host site staff setting up the
experiments, the technology operator, and two confirming individuals to verify the result.
The laboratory testing will be conducted with the specific testing conditions that are
presented in Table 3. The experimental factors to be altered for each chemical tested are the
background materials, wind speed, and stand-off distance. These experimental factors were
chosen, because the passive imagers rely on physical characteristics of the monitoring
environment. The background will demonstrate the ability of the technology to visualize the
leak with a background similar to the leaking component (curved metal gas cylinder) and with a
background that is different than the leaking component, but more uniform in nature (cement
board). The wind speed variations and the stand-off distances will elucidate these effects on the
method detection limit, and in turn on real-world applicability.
A list of chemicals to be used in laboratory testing is shown in Table 4. From one to all
of these chemicals will be used in the laboratory testing of each LDAR technology; however, this
list might be limited by the vendor-stated capabilities of detecting the chemicals and the testing
constraints of time, labor, funding, etc. This list of chemicals encompasses different groups of
chemicals that absorb at different wavelengths. It is an abbreviated but representative list of
chemicals of interest that were identified by the stakeholder committee, which is made up of
experts in this field from governmental agencies and private companies. Many of the chemicals
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 26 of 46
Version 1.0
Date:9/18/2008
have a general absorbance in the 3 - 5 jim range and a fingerprint absorbance specific to the
compound in the 7 - 14 jim range. Thus, these chemicals would be applicable for a mid or long
range imaging technology. The list also includes some chemicals that would be found at the
field sites so as to challenge the technologies in controlled and real world environments with
some similar chemicals.
Table 3. Laboratory Testing Matrix for Each Chemical of Interest
Test
Condition
1
2
3
4
5
6
7
8
9
10
11
12
Experimental Factors
Component
Valve
Valve
Valve
Valve
Valve
Valve
Valve
Valve
Valve
Valve
Valve
Valve
Background
Curved metal gas cylinder
Curved metal gas cylinder
Curved metal gas cylinder
Curved metal gas cylinder
Curved metal gas cylinder
Curved metal gas cylinder
Cement board
Cement board
Cement board
Cement board
Cement board
Cement board
Wind Speed
0 m/sec
0 m/sec
2.5 m/sec
2.5 m/sec
5 m/sec
5 m/sec
0 m/sec
0 m/sec
2.5 m/sec
2.5 m/sec
5 m/sec
5 m/sec
Stand-off Distance
10ft
30ft
10ft
30ft
10ft
30ft
10ft
30ft
10ft
30ft
10ft
30ft
Table 4. Chemicals of Interest in Testing
Chemical Group
Acetate
Acid
Alcohol
Alkane
Aromatic
Aromatic
Chlorinated
Chlorinated
Inorganic
Olefin
Olefin
Chemical
Acetic acid
Acrylic acid
Methanol
Pentane
Styrene
Benzene
Methylene chloride
Propylene dichloride
Hydrochloric acid
1,3 -Butadiene
Ethylene
Lab Testing
X
X
X
X
X
X
X
X
X
X
X
Field Testing
X
X
X
X
X
X
X
X
X
The general testing procedure for the laboratory experiments will begin with a specific
testing condition set up. Then, the operator will view this condition at a leak rate high enough to
be identified by the operator and two confirming individuals. The method detection limit will be
determined and recorded for the leak condition. Detection limits will be determined for each of
the twelve testing conditions with each chemical tested. See section B1.1.2 for the procedure for
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 27 of 46
Version 1.0
Date:9/18/2008
assessing the detection limit. At each detection limit, a portable monitoring device (acceptable
under Method 21) will "sniff the leak to determine if it would detect a leak at that rate and
under the tested conditions. In addition, two units of the technology will be operated by the
same staff following the same procedure to assess inter-unit reproducibility. All results will be
recorded along with the corresponding environmental conditions. Specifically, the temperature of
the chemical plume and the ambient temperature will be recorded and reported for each testing
condition. Any operational factors will be noted by the operator related to the use of the
technology.
Bl. 1.2 Field Test Procedures
The evaluation of LDAR technologies will be performed at two field sites: a
petrochemical and a chemical plant. The chemicals tested in the laboratory will be possible
leaking chemicals at the field sites. The testing group will consist of the technology operator,
two confirming individuals to verify the result, and the bagging team to collect the reference
samples. Additionally, representatives from the host testing sites will be permitted to operate
their own LDAR equipment simultaneously during the field testing. This group will move
through the plant screening for possible leaking components using a portable monitoring device
(acceptable under Method 21) as specified in the bagging reference method. Once a leak is
detected, leak characteristics and environmental factors will be recorded (i.e. type of component,
background, date and time, temperature, etc). Where space permits, the operator will then take
readings from two units of the same LDAR technology at three stand-off distances (10, 30, and
greater than 30 feet, if possible) adjusting and noting the lens and zoom features, as needed.
Meteorological parameters and operational factors related to the use of the technology will be
noted by the operator to further characterize the testing environment. Meteorological data will
be retrieved from the nearest meteorological data station. Every reading will be verified by two
confirming individuals and recorded by the operator as detect or non-detect. The operator and
the two confirming individuals must agree on each result. If they do not agree, the result will be
considered a non-detect. Once all the technologies being evaluated have scanned the leak, the
bagging team will commence collecting the reference samples using the procedure described in
Section B4. This procedure will conclude with a final screening from an accepted portable
monitoring device to verify that the leak has not changed from the beginning to the end of testing
the component. If a leak changes more than 20% between the pre- and post- screening with the
portable monitoring device, the leak will not be considered consistent enough to report in the
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 28 of 46
Version 1.0
Date:9/18/2008
results. All field conditions will be reported with the technology results. This entire process will
be repeated between 20 and 40 times or as scheduling permits at each field site. Collecting a
large sample size at each field site, randomizing the testing order of the technologies, and
collecting a broad range of sample types (different leak rates, components, time of day, etc) at
the field sites will mask the variability of the field conditions. To reduce the possible variability
from day to day conditions of the field environment, this sampling process for each leak will be
completed on the same day it begins. Finally, on a daily basis, the testing group will report to the
testing site the leaking components detected by the technology to assist the site in making
necessary repairs.
Bl. 1.3 Testing Parameters for Laboratory and Field Testing
The following sections describe in more detail the evaluation of the testing parameters at
the laboratory and/or field sites.
Bl. 1.3.1 Detection of Chemical Gas Species Relative to a Portable Monitoring Device
The detection of a single chemical gas leak in either the lab or field environments will be
determined by the operator as well as two confirming individuals and reported qualitatively as
either detected or non-detected. All three individuals must agree on the results. If they do not
agree, the result will be reported as a non-detect. The leak rate will be known from certified gas
cylinders and calibrated flow meters in the laboratory, or determined through the bagging
method in the field environment. Whether a leak is detected or not will be reported along with
the conditions of the leak and the environment. A portable monitoring device (acceptable under
Method 21) will "sniff the leaks as part of the field reference method. In the lab, these
qualitative data will be reported with the technology results. This will reflect whether or not a
technology would have detected a leak relative to a device acceptable under Method 21. This
will be reported as percent agreement by chemical that would have been detected by Method 21.
Bl. 1.3.2 Method Detection Limit
Method detection limits will only be determined in the laboratory portion of this test.
Once it has been determined that a LDAR technology can detect a specific chemical, the method
detection limit will be determined. To do this, the leak rate will be set at a nominally detectable
level that is either specified by the vendor's LOD or is based on previous literature concerning
the technology's capabilities. The criteria for finding the leak are as follows:
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 29 of 46
Version 1.0
Date:9/18/2008
The operator and two confirming individuals all must identify the leak.
They will view the leak through the intended view (eye piece or monitor) for the
technology as specified by the vendor.
They will have five seconds to identify the origin of the leak or be able to track the
cloud back to the leaking component.
For the laboratory portion of the test, if all three individuals identify the leak, then the
leak rate will be reduced at the discretion of the testing staff. Once a leak rate that is not
identifiable by all three people has been reached, the rate will be increased to the level where all
three can again identify the leak using the LDAR technology. This rate is the method detection
limit for the technology under the tested conditions. The detection limit and testing conditions
will be reported. This process will be completed for every testing trial listed in Table 3 for each
chemical gas species tested.
Bl. 1.3.3 Inter-unit Reproducibility
Each technology will be tested with duplicate units to evaluate the inter-unit
reproducibility. The same operator will use both units and all results will be verified by the
confirming individuals. In the lab, Inter-unit reproducibility will be determined by calculating a
linear regression comparing the MDLs of the two units for each chemical of interest. The field
results will determine inter-unit reproducibility by calculating percent agreement between the detect
and non-detect results from both units for measurements at each field site.
Bl. 1.3.4 Confounding Factor Effects
Confounding factors will be assessed in the lab and field environments. In the lab, the
conditions will be controlled and will follow the test conditions presented in Table 3 for each
chemical. These include two different backgrounds, three wind speeds and two stand-off
distances. In the field, in addition to background, wind speed, and distance, other factors that
will be noted include: component type, relative wind direction with respect to the LDAR
technology, humidity, temperature, location, and meteorological conditions (among other
components or stand-alone).
B1.1.3.5 Operational Factors
Operational factors such as maintenance needs, calibration frequency, data output,
consumables used, ease of use, repair requirements, and sample throughput will be evaluated
based on operator observations. A laboratory record book (LRB) or data sheets will be used to
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 30 of 46
Version 1.0
Date:9/18/2008
document observations. Examples of information to be recorded include the daily status of
diagnostic indicators for the technology, use or replacement of any consumables, the effort or
cost associated with maintenance or repair, vendor effort (e.g., time on site) for repair or
maintenance, the duration and causes of any technology down time or data acquisition failure,
operator observations about technology startup, ease of use, clarity of the vendor's instruction
manual, user-friendliness of any needed software, overall convenience of the technologies and
accessories/consumables, or the number of samples that could be processed per hour or per day.
These observations will be summarized to aid in describing the technology performance in the
verification report on each technology.
B1.2 Statistical Analysis
The statistical methods and calculations used for evaluating quantitative performance
parameters are described in the following sections.
B1.2.1 Method Detection Limit
The method detection limit will be assessed in the lab through the procedure stated in
section B1.1.3.2 and reported as such. The detection limits will be reported in tabular form
relative to the conditions specified in Table 3. An overall detection limit variation will be
calculated as the standard deviation of the detection limits determined under all the conditions
for each chemical of interest. The equation for standard deviation is as follows:
S =
(i)
where Sx is the standard deviation of all detection limits determined for chemical x, n is the
number of replicate samples, Ck is the leak rate measured for the kth sample, and C is the
average leak rate of the replicate samples.
B 1.2.2 Linear Regression
Inter-unit reproducibility for the laboratory results will be assessed by linear regression with
the MDLs for unit # 1 along the x-axis and the corresponding MDLs for unit # 2 along the y-axis.
Linearity will be expressed in terms of the slope, intercept, and the coefficient of determination (r2).
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 31 of 46
Version 1.0
Date:9/18/2008
Bl.2.3 Percent Agreement
The field results will assess inter-unit reproducibility by calculating the percent agreement
between the two units. Percent agreement will be calculated using the following:
% Agreement = x 1 00 (2)
where A is the number of tests that both units agree and T is the total number of tests.
Percent agreement will also be used to assess the agreement between the technology and
the portable monitoring device (acceptable under Method 21) in the laboratory for each chemical
of interest. The inverse of the percent agreement will be the percentage of the results that the
technology would detect a leak when Method 21 would not.
B1.3 Reporting
The data obtained in the verification test will be compiled separately for each vendor's
technology, and the data evaluations will be applied to each technology's data set without
reference to any other. At no time will data from different vendor's technology be inter-
compared or ranked. Following completion of the data evaluations, a draft verification report
and verification statement will be prepared for each vendor's technology, stating the verification
test procedures and documenting the performance observed. For example, descriptions of the
data acquisition procedures, use of vendor supplied proprietary software, consumables used,
repairs and maintenance needed, and the nature of any problems will be presented in the draft
report. Each report will briefly describe the ETV Program, the AMS Center, and the procedures
used in verification testing. The results of the verification test will be stated quantitatively,
without comparison to any other technology tested, or comment on the acceptability of the
technology's performance. Each draft verification report will be submitted for review by the
respective technology vendor and by EPA and other peer reviewers. Comments on the draft
report will be addressed in revisions of the report. The peer review comments and responses will
be tabulated to document the peer review process. The reporting and review process will be
conducted according to the requirements of the AMS Center QMP.1
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 32 of 46
Version 1.0
Date:9/18/2008
B2 SAMPLING REQUIREMENTS
B2.1 Sample Collection, Storage and Shipment
Samples in the forms of leaks in both lab and field environments will be detected by the
LDAR technologies in real time. The reference method samples for each leak will be collected
and handled in either bags or canisters as specified in the reference method. The reference
samples will be collected and analyzed in duplicate to obtain an average mass emission from the
leak. They will be analyzed on-site or sent to an analytical laboratory. Reference sample
shipments will be via a traceable overnight delivery service to the Battelle sample custodian or
an appropriate analytical lab. The samples will be stored at room temperature. The holding time
will be determined by the analytical chemist developing the GC method following EPA Method
185. It will depend on the collection container (Teflon bag or canister) and the chemical
collected.
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Reference sample custody will be documented throughout collection, transport, shipping
(if necessary), and analysis using standard chain-of-custody (COC) forms provided by Battelle or
supplied by others providing samples for testing, as appropriate. Samples transferred within
Battelle may be documented in bound sample login LRBs. Each COC form will summarize the
samples collected. The COC forms will track sample release from the sampling location to
Battelle. Each COC form will be signed by the person relinquishing the samples once that person
has verified that the COC form is accurate. The original sample COC forms will accompany the
samples; the shipper will keep a copy. Upon receipt, COC forms will be signed by the person
receiving the samples once that person has verified that all samples identified on the COC forms
are present. Any discrepancies will be noted on the form and the sample receiver will
immediately contact the Verification Test Coordinator to report missing, broken, or otherwise
compromised samples. Copies of all COC forms will be delivered to the Verification Test
Coordinator, and maintained with the test records. Samples being transferred within Battelle may
be documented in a sample login LRB which will be used to note the current location of all
samples housed at Battelle.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 3 3 of 46
Version 1.0
Date:9/18/2008
B4 REFERENCE METHOD
The laboratory will not use a specified reference method for determining the leak rate of
the test conditions. Certified gas cylinders will be used with a calibrated flow meter. Once gas
flow is equilibrated through the system, this will constitute a known leak.
The reference method for all field samples will be the combination of a reference
collection method and an analytical method for the determination of the reference samples. The
U.S. EPA Protocolfor Equipment Leak Emission Estimates4 Section 4 describes the bagging
method used to collect the reference samples. It incorporates an initial screening of the leaking
component with a portable monitoring device. Then the leaking component is completely
enclosed with a non-permeable material and a known clean air or nitrogen flow is supplied to the
enclosure to equilibrate the mass flow. The material may be Mylar, Tedlar, Teflon,
aluminum foil, or aluminum Mylar with a thickness ranging from 1.5 mm to 15 mm. The
contents of the enclosure or "bag" are then sampled in duplicate into a Teflon bag or a canister
for analysis by a gas chromatographic (GC) method. Finally, the component is re-screened with
the portable monitoring device. There are two basic variations to this approach that have been
widely utilized to measure the mass emissions from equipment leaks: the vacuum method and
the blow-through method. The main difference procedurally, is that the vacuum method uses a
vacuum pump to pull the air through the enclosure and the blow-through method uses a carrier
gas such as nitrogen (or another inert gas) blown into the enclosure. Either method may be used
depending on the expected sample concentrations and resources available for the test. The
vacuum method is preferred when monitoring larger leaks and the blow through method is
preferred when monitoring smaller leaks. The bagging protocol will have two variations from
the stated EPA procedure.
No background samples will be taken when using the vacuum method. They would
have a negligible effect on the results from higher leaking components.
No analytical tests will be performed on any liquid leak materials collected. Vapor
leak detection is the objective of the performance evaluation; therefore, only the
vapor will be analyzed through the reference method procedure.
The analytical method will determine the concentration of the collected sample and will
follow EPA Method 18 Measurement of Gaseous Organic Compound Emissions by Gas
Chromatography.5 This is a general analytical method that is specific to leak-emission type
samples. It is a GC method that may be equipped with a flame ionization detector or an electron
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 34 of 46
Version 1.0
Date:9/18/2008
capture detector. Because of the varying numbers and kinds of possible compounds to be
detected at differing concentrations, this method allows the technical analyst to research the
correct analytical column and temperature program to obtain the necessary detection limit and
separation of the compounds of interest. The QA/QC requirements for the documentation and
performance of the analytical method are described as data quality indicators (DQI) in Section
B5.
B5 QUALITY CONTROL CRITERIA FOR REFERENCE METHOD
MEASUREMENT DATA
Table 5 presents the DQIs and criteria for the reference collection and analytical method
critical measurements. The reference method measurement quality will be assured by adherence
to these DQI criteria.
Prior to start of the reference sampling, the portable monitoring device will be calibrated
according to the manufacturer's specified procedure. Initial calibration will consist of an un-
spiked gas standard and then four or more additional concentrations of gas standard. A
calibration check sample will also be sampled at a minimum of 5 % of the samples to verify the
calibration of the portable monitoring device over time. On each day of testing the device will
be calibrated and verified that the overall sensitivity has changed < 10 %.
The bagging procedure will be verified by bagging an artificial leak at a known rate in
the middle of the analytical calibration curve called a Leak Rate Check. This is to be performed
at the beginning and end of the testing period and two times per week of testing. This procedure
will be followed as specified in the EP A Protocol for Equipment Leak Emissions Estimates* with
any flow meters or measurement equipment being calibrated following the manufacturer's
specified procedure. 80% to 120% recovery will be the acceptable range. If this criterion is not
met, the procedure and supplies will be investigated and corrected before sampling continues.
The reference method GC analytical measurement quality will be monitored by
comparing the analytical instrument response with calibration standards traceable to certified gas
cylinder standards. The calibration curve will consist of an un-spiked standard and four or more
additional concentrations of gas standards. The gas standards should be within 2% of the
certification. If not, the source of standard contamination or changes in the instrument
parameters will be investigated and the standards re-analyzed. Steps will be taken to maintain
the quality of data collected during this verification test. This will include analyzing specific
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 35 of 46
Version 1.0
Date:9/18/2008
quality control samples (QCS) at a regular frequency. QCSs will include negative controls and
positive controls. Negative control samples are blank samples that will be used to help ensure
that no sources of contamination are introduced in the analysis procedures. The positive control
samples will indicate to the operator whether or not the GC is functioning properly and be at a
mid-level calibration concentration. QCSs producing results that do not meet the anticipated
criteria will be reanalyzed and corrective action taken if needed to ensure that test sample results
are not affected. Positive controls will be accepted if there is < 10% change. Corrective actions
for positive controls may include reanalyzing samples to verify that the GC is operating properly,
conducting maintenance, or recalibrating. For negative controls, the source of contamination
should be investigated, corrected, and the samples re-analyzed. Positive and negative controls
will be analyzed at a frequency of approximately 10 % based on the total number of test samples
for the LDAR technologies.
Table 5. DQIs and Criteria of Critical Measurements for Reference Method
DQI
Confirmation of
Detected Leaks
Bias and Accuracy of
Sample Screening
Measurements- using
Portable Monitoring
Device
Bias and Accuracy of
enclosure equilibration
gas- zero for bagging
component
Method of
Assessment
Portable monitoring
device measurement
before and after
bagging of
component
Initial Calibration-
various levels of
calibration gas.
Starting with un-
spiked gas standard
then 4 or more
additional
concentrations of gas
standard.
Calibration check
Sample- one
concentration of
calibration gas
standard
Collect equilibration
gas in bag and
analyze for
contamination
Frequency
All field test
samples
Perform initial
calibration at
the start and
end of every
verification
testing day or if
overall device
sensitivity
changes > 10%.
Minimum 5%
of all samples
tested.
Before using
cylinder as
equilibration
gas
Minimum
Acceptance Criteria
Pre and post screening
results within 20%
Each calibration gas
standard concentration
within 10%
Check standard is
within < 10% change
in response from initial
calibration after
adjustment of overall
monitoring device
sensitivity
Concentration of
organic compounds
should be < 0. 1 ppm of
chemical of interest
Corrective Action
Data considered suspect
and reanalyzed or reported
with qualifiers if reanalysis
is not possible
Investigate sources of
standard contamination or
changes in monitoring
device, reanalyze standards.
Adjust overall monitoring
device sensitivity; check
new calibration check
samples; repeat initial
calibration
Reanalyze affected sampled
components since last
successful check if
calibration check change is
> 10%
A fresh cylinder or a
different equilibration gas
should be used
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 36 of 46
Version 1.0
Date:9/18/2008
TableS. (Continued)
Bias and Accuracy of
Bagging Procedure-
Leak Rate Check for
Bag Sampling
Completeness
Representativeness
Bias and Accuracy of
GC Analytical Method
Create an artificial
leak of a known gas
to clarify the
magnitude of any
bias in the
combination of
sampling/test method
and define the
variance in emissions
estimate due to the
sampling
Amount of valid data
obtained
Performance Test
and Environmental
Samples
Initial Calibration-
various levels of
certified calibration
gas. Starting with
un-spiked gas
standard then 4 or
more additional
concentrations of gas
standard
Positive Control
Sample/Calibration
check Sample- one
concentration of
calibration gas
standard
Negative Control
Sample
At the
beginning and
end of the
testing period
and 2 times per
week during
testing
Overall number
of data points
collected for
test
Overall for test
Perform initial
calibration at
the start and
end of every
analytical
sequence or if
overall
instrument
sensitivity
changes > 10%
Minimum 10%
of all samples
tested
Minimum of
10% of all
samples tested
80% to 120% is the
accepted recovery
range
90% of overall data
points collected should
be valid
Samples which span
the responses obtained
in the initial calibration
curve
Standards are within
2% of certification
Check standard is
within < 10% change
in response from initial
calibration after
adjustment of Overall
Instrument Sensitivity
Reanalyze affected
samples since last
successful check if
calibration check
change is > 10%
Must remain lower
than the lowest
calibration standard
Investigate and correct
before sampling continues
If feasible, analyze
additional samples to meet
the acceptance criteria
Dilute sample and
reanalyze
Investigate sources of
standard contamination or
changes in instrument
parameters, reanalyze
standards
Adjust Overall Instrument
Sensitivity; check new
calibration check samples;
repeat initial calibration
Investigate source of
contamination or changes
in instrument parameters,
reanalyze sample.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 37 of 46
Version 1.0
Date:9/18/2008
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The test condition and reference equipment used in this test (i.e., instrumentation used for
reference measurements, temperature, flow rates, etc.) will be tested, inspected, and maintained
as per the manufacturer's recommendations so as to meet the performance requirements
established in this document. When technical staff operate and maintain technologies undergoing
testing, those activities will follow directions provided by the technology vendor.
B7 INSTRUMENT CALIBRATION AND FREQUENCY
Prior to start of the reference sampling, the portable monitoring device will be calibrated
according to the manufacturer's specified procedure. Initial calibration will consist of an un-
spiked gas standard and then four or more additional concentrations of gas standard. A
calibration check sample will also be sampled at a minimum of 5 % of the samples to verify the
calibration of the portable monitoring device over time. On each day of testing the device will
be calibrated and verified that the overall sensitivity has changed < 10 %.
The reference method GC analytical measurement quality will be monitored by
comparing the analytical instrument response with calibration standards traceable to certified gas
cylinder standards. The calibration curve will consist of an un-spiked standard and four or more
additional concentrations of gas standards. The gas standards should be within 2% of the
certification. The positive control samples will indicate to the operator whether or not the GC is
functioning properly and be at a mid-level calibration concentration. Positive controls will be
accepted if there is < 10% change. Positive controls will be analyzed at a frequency of
approximately 10 % based on the total number of test samples for the LDAR technologies.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
All materials, supplies, and consumables will be ordered by the Verification Test
Coordinator or designee. Where possible, Battelle will rely on sources of materials and
consumables that have been used previously as part of ETV verification testing without
problems. Battelle will also rely on previous experience or recommendations from EPA
advisors, host facilities, or the LDAR vendor.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 3 8 of 46
Version 1.0
Date:9/18/2008
B9 NON-DIRECT MEASUREMENTS
No non-direct measurements will be used during this verification test.
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by
Battelle during the verification test. Table 2 summarizes the types of data to be recorded. All
maintenance activities, repairs, calibrations, and operator observations relevant to the technology
operation will be documented by technical staff in LRBs or on data sheets. Results from the
reference methods, including raw data, analyses, and final results, will be compiled by Battelle,
and duplicates of the data will be left on premises of the testing sites.
Records received by or generated by any technical staff during the verification test will
be reviewed by a Battelle staff member within two weeks of generation or receipt, before the
records are used to calculate, evaluate, or report verification results. If a Battelle staff member
generated the record, this review will be performed by a Battelle technical staff member involved
in the verification test, but not the staff member who originally generated the record. The review
will be documented by the person performing the review by adding his/her initials and date to the
hard copy of the record being reviewed. In addition, any calculations performed by technical
staff will be spot-checked by Battelle QA and/or technical staff to ensure that calculations are
performed correctly. Calculations to be checked include any statistical calculations described in
this test/QA plan. The data obtained from this verification test will be compiled and reported
independently for each technology. Results for technologies from different vendors will not be
compared with each other. Finally, representatives from the testing sites will have the
opportunity to review all images and data prior to public distribution or publication.
Among the QA activities conducted by Battelle QA staff will be an audit of data quality.
This audit will consist of a review by the Battelle AMS Center Quality Manager (or his designee)
of at least 10% of the test data. During the course of any such audit, the Battelle AMS Center
Quality Manager will inform the technical staff of any findings and any need for immediate
corrective action. If serious data quality problems exist, the Battelle AMS Center Quality
Manager will request that Battelle's AMS Center Manager issue a stop work order. Once the
assessment report has been prepared, the Verification Test Coordinator will ensure that a
response is provided for each adverse finding or potential problem, and will implement any
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 3 9 of 46
Version 1.0
Date:9/18/2008
necessary follow-up corrective action. The Battelle AMS Center Quality Manager will ensure
that follow-up corrective action has been taken.
Data obtained during the verification test will be maintained confidentially at Battelle,
and used only for purposes of the LDAR evaluation. Data reporting in the final report will
consist of tabular results of the calculations in Section B. Any images from the LDAR devices
that are presented will only be included after permission from the host site has been granted, and
it has been determined that no proprietary information is contained in the image. All data and
images, to the extent possible, will be host site anonymous.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 40 of 46
Version 1.0
Date:9/18/2008
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in this verification test to anticipate and resolve potential
problems before the quality of performance is compromised. One of the major objectives of this
test/QA plan is to establish mechanisms necessary to ensure this. The procedures described in
this test/QA plan, which is peer reviewed by a panel of outside experts, implemented by the
technical staff and monitored by the Verification Test Coordinator, will give information on data
quality on a day-to-day basis. The responsibility for interpreting the results of these checks and
resolving any potential problems resides with the Verification Test Coordinator. Technical staff
have the responsibility to identify problems that could affect data quality or the ability to use the
data. Any problems that are identified will be reported to the Verification Test Coordinator, who
will work with the Battelle AMS Center Quality Manager to resolve any issues. Action will be
taken to control the problem, identify a solution to the problem, and minimize losses and correct
data, where possible. Independent of any EPA QA activities, Battelle will be responsible for
ensuring that the following audits are conducted as part of this verification test.
Cl.l Performance Evaluation Audits
A Performance Evaluation (PE) audit will be conducted to establish the traceability of the
GC analytical reference method measurements made in this verification test. The reference
method PE audit will be performed using an independent, certified cylinder calibration gas
mixture. This PE audit sample will be analyzed in the same manner as all other samples and the
analytical results for the PE audit samples will be compared to the nominal concentration. The
target criterion for this PE audit is agreement of the analytical result within 25% of the nominal
concentration. If the PE audit result does not meet the target criterion, the PE audit will be
repeated. If the outlying results persist, the source of error will be investigated and corrective
action taken as necessary until successful PE audit results are obtained. This audit will be
performed once prior to the start of the test at each field site, and will be the responsibility of the
Verification Test Coordinator or designee.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 41 of 46
Version 1.0
Date:9/18/2008
C1.2 Technical Systems Audits
The Battelle AMS Center Quality Manager will perform a technical systems audit (TSA)
at least once during this verification test. The purpose of this audit is to ensure that the
verification test is being performed in accordance with the AMS Center QMP,1 this test/QA plan,
using the EPA designated reference method, and any Standard Operating Procedures (SOPs)
used by Battelle. In the TSA, the Battelle AMS Center Quality Manager or a designee may
review the reference method used, compare actual test procedures to those specified or
referenced in this plan, and review data acquisition and handling procedures. The Battelle
Quality Manager will tour the test sites, observe and review the test procedures, and review
record books. He will also check calibration certifications for test measurement devices. A TSA
report will be prepared, including a statement of findings and the actions taken to address any
adverse findings. The EPA AMS Center Quality Manager will receive a copy of Battelle's TSA
report. At EPA's discretion, EPA QA staff may also conduct an independent on-site TSA during
the verification test. The TSA findings will be communicated to technical staff at the time of the
audit and documented in a TSA report.
C1.3 Data Quality Audits
The Battelle AMS Center Quality Manager will audit at least 10% of the verification data
acquired in the verification test. The Battelle AMS Center Quality Manager will trace the data
from initial acquisition, through reduction and statistical comparisons, to final reporting. All
calculations performed on the data undergoing the audit will be checked.
C1.4 QA/QC Reporting
Each assessment and audit will be documented in accordance with Sections 3.3.4 and
3.3.5 of the AMS Center QMP.1 The results of the technical systems audit will be submitted to
EPA. Assessment reports will include the following:
Identification of any adverse findings or potential problems
Response to adverse findings or potential problems
Recommendations for resolving problems
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 42 of 46
Version 1.0
Date:9/18/2008
Confirmation that gases have been implemented and are effective
Citation of any noteworthy practices that may be of use to others.
C2 REPORTS TO MANAGEMENT
The Battelle AMS Center Quality Manager, during the course of any assessment or audit,
will identify to the technical staff performing experimental activities any immediate corrective
action that should be taken. If serious quality problems exist, the Battelle AMS Center Quality
Manager is authorized to request that Battelle's AMS Center Manager issue a stop work order.
Once the assessment report has been prepared, the Verification Test Coordinator will ensure that
a response is provided for each adverse finding or potential problem and will implement any
necessary follow-up corrective action. The Battelle AMS Center Quality Manager will ensure
that follow-up corrective action has been taken. The test/QA plan and final report are reviewed
by EPA (acting) AMS Center Quality Manager and EPA AMS Center Project Officer. Upon
final review and approval, both documents will then be posted on the ETV website
(www.epa.gov/etv).
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 43 of 46
Version 1.0
Date:9/18/2008
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are stated in Section BIO of
this test/QA plan. In general, the data review requirements specify that the data generated during
this test will be reviewed by a Battelle technical staff member within two weeks of data
generation. The reviewer will be familiar with the technical aspects of the verification test, but
will not be the person who generated the data. This process will serve both as the data review
and the data verification, and will ensure that data have been recorded, transmitted, and
processed properly. Furthermore, this process will ensure that the LDAR technology data and the
reference method data are collected under appropriate testing conditions and that the reference
method data meet the reference method specifications.
The data validation requirements for this test involve a data quality assessment relative to
the DQIs and audit acceptance criteria specified for this test. The DQIs listed in Section B5 will
be used to validate the data quality. The QA audits described within Section C of this document,
including the performance evaluation audit and data quality audit, are designed to validate the
data quality.
D2 VALIDATION AND VERIFICATION METHODS
As part of the normal data and report review process the TCEQ and the US EPA will
have the opportunity to review the draft final report and provide comments. Data verification is
conducted as part of the data review, as described in Section BIO for this test/QA plan. A visual
inspection of handwritten data will be conducted to ensure that all entries were properly recorded
or transcribed, and that any erroneous entries were properly noted (i.e., single line through the
entry with an error code and the initials of the recorder and date of entry). Electronic data from
the technologies and other instruments used during the test will be inspected to ensure proper
transfer from the data logging system. Data manually incorporated into spreadsheets for use in
calculations will be checked against handwritten data to ensure that transcription errors have not
occurred. All calculations used to transform the data will be reviewed to ensure the accuracy and
the appropriateness of the calculations. Calculations performed manually will be reviewed and
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 44 of 46
Version 1.0
Date:9/18/2008
repeated using a handheld calculator or commercial software (e.g., Excel). Calculations
performed using standard commercial office software (e.g., Excel) will be reviewed by
inspecting the equations used in calculations and verifying selected calculations by handheld
calculator. Calculations performed using specialized commercial software (i.e., for analytical
instrumentation) will be reviewed by inspection and, when feasible, verified by handheld
calculator, or standard commercial office software.
To ensure that the data generated from this test meet the goals of the test, a number of
data validation procedures will be performed. Section C of this test/QA plan provides a
description of the validation safeguards employed for this verification test. Data validation and
verification efforts include the completion of QC activities and the performance of ISA and PE
audits as described in Section C. The data from this test will be evaluated relative to the
measurement DQIs described in Section A4 and B5, and the PE audit acceptance criteria given in
Section Cl.l of this test/QA plan. Data failing to meet these criteria will be flagged in the data
set and not used for evaluation of the technologies, unless these deviations are accompanied by
descriptions of their potential impacts on the data quality.
A data quality audit will be conducted by the Battelle AMS Center Quality Manager to
ensure that data review, verification, and validation procedures were completed, and to assure the
overall data quality.
D3 RECONCILIATION WITH USER REQUIREMENTS
The purpose of a verification test performed following this test/QA plan is to evaluate the
performance of commercial technologies which detect gaseous leaks from industrial
components. This test evaluates the LDAR technology capability only and is not a verification of
whether or not the measurement quantitatively measures fugitive emissions. This evaluation will
include comparisons of the results from the technologies to results from standard reference
techniques. To meet the requirements of the user community, the data obtained in such a
verification test will include thorough documentation of the technology's performance during the
verification test. The data review, verification, and validation procedures described above will
assure that verification test data meet these requirements, are accurately presented in the
verification reports generated from the test, and that data not meeting these requirements are
appropriately flagged and discussed in the verification reports. Additionally, all data generated
using the reference method, which are used to evaluate technology results during the verification
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 45 of 46
Version 1.0
Date:9/18/2008
test, should meet the Q A requirements of any applicable standard operating procedures or
instrumentation instruction manuals.
This test/QA plan and any resulting ETV verification report(s) generated following
procedures described in this test/QA plan will be subjected to review by participating technology
vendors, ETV AMS Center staff, test collaborators, EPA, and external expert peer reviewers.
These reviews will assure that this test/QA plan, verification test(s) of LDAR technologies, and
the resulting report(s) meet the needs of potential users and regulators. The final report(s) will be
submitted to EPA in 508 compliant Adobe Portable Document Format (pdf) and subsequently
posted on the ETV website.
-------
Verification of Leak Detection and Repair Technologies Test/QA Plan
Page 46 of 46
Version 1.0
Date:9/18/2008
SECTION E
REFERENCES
El REFERENCES
1. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version
6.0, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus,
Ohio, November 2005.
2. Environmental Technology Verification Program Quality Management Plan, EPA/600/R-
03/021, U.S. Environmental Protection Agency, Cincinnati, Ohio, December 2002.
3. EPA Method 21 - Detection of Volatile Organic Compound Leaks, EPA-600/2-18-110;
U.S. EPA, September 1981.
4. EPA Protocol for Equipment Leak Emissions Estimates, EPA-453/R-95-017; U.S. EPA:
Research Triangle Park, NC, November 1995.
5. EPA Method 18 - Measurement of Gaseous Organic Compound Emissions by Gas
Chromatography, 40 CFR, Part 60, Appendix A; U.S. EPA, 1994.
------- |