Baltelle
The Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Test/QA Plan for Verification of
Radio Frequency Identification (RFID)
for Tracking Hazardous Waste
Shipments across International Borders
-------
Verification of
Radio Frequency Identification (RFID) for Tracking Hazardous
Wastes Shipments across International Borders
March 23, 2009
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Test/QA Plan for Verification of RFID Tracking
Version 1.0
3/23/2009
Page 3 of 3 8
TABLE OF CONTENTS
Section Page
SECTION A PROJECT MANAGEMENT 7
Al VERIFICATION TEST ORGANIZATION 7
A2 BACKGROUND 14
A3 VERIFICATION TEST DESCRIPTION AND SCHEDULE 14
A4 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA 17
A5 SPECIAL TRAINING/CERTIFICATION 18
A6 DOCUMENTATION AND RECORDS 18
SECTION B MEASUREMENT AND DATA ACQUISITION 20
Bl EXPERIMENTAL DESIGN 20
B2 SAMPLING REQUIREMENTS 29
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS 29
B4 Field Test REFERENCE METHOD 29
B5 Reference Method QUALITY CONTROL 30
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE .... 30
B7 Technology CALIBRATION 30
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES 30
B9 NON-DIRECT MEASUREMENTS 31
BIO DAT A MANAGEMENT 31
SECTION C ASSESSMENT AND OVERSIGHT 33
Cl ASSESSMENTS AND RESPONSE ACTIONS 33
C2 REPORTS TO MANAGEMENT 34
SECTION D DATA VALIDATION AND USABILITY 36
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS 36
D2 VALIDATION AND VERIFICATION METHODS 36
D3 RECONCILIATION WITH USER REQUIREMENTS 37
SECTION E REFERENCES 38
El REFERENCES 38
-------
Test/QA Plan for Verification of RFID Tracking
Version 1.0
3/23/2009
Page 4 of 3 8
List of Figures Page
Figure 1. Organization Chart for the Verification Test 8
Figure 2. Trucking Route Map (read points labeled) 25
List of Tables
Table 1. Data Collected During Verification Test 18
Table 2. Summary of Testing Route Read Points 22
TableS. Summary of Data Recording Process 32
-------
Test/QA Plan for Verification of RFID Tracking
Version 1.0
3/23/2009
Page 5 of 3 8
ETV Advanced Monitoring Systems Center
Test/QA Plan for Verification of
Radio Frequency Identification (RFID) for Tracking Hazardous Wastes
Shipments across International Borders
March 23, 2009
APPROVAL:
Name
Company
Date
-------
DISTRIBUTION LIST
John McKernan
Douglas Grosse
Michelle Henderson
U.S. Environmental Protection Agency
26 W. Martin Luther King Dr.
Mail Code: 208A
Cincinnati, OH 45268
Deborah Kopsick
USEPA Headquarters
Ariel Rios Building
1200 Pennsylvania Avenue, N. W.
Mail Code: 2254A
Washington, DC 20460
Debra Tellez
Carlos Rincon
USEPA U.S.-Mexico Border Program
Outreach
4050 Rio Bravo Suite # 100
Mail Code: 6PD
El Paso, TX 79902
Israel Anderson
USEPA REGION 6
1445 Ross Avenue Suite 1200
Mail Code: 6ENXP
Dallas, TX 75202-2733
Jose Mario Sanchez - Soledad
City of Juarez, Mexico
Stephen Niemeyer
Texas Commission on Environmental
Quality
Border Affairs Manager
P.O. Box 13087
Austin, TX 78711-3087
Karl Krueger
AVANTE International Technology, Inc.
70 Washington Road
Princeton Junction, NJ 08550
Test/QA Plan for Verification of RFID Tracking
Version 1.0
3/23/2009
Page 6 of 3 8
John Samony
Private Pallet Security Systems, LLC
4320 Winfield Road, Suite 200
Warrenville, IL 60555
Barry Thatcher
BorderWriting
MSC3EPO Box 30001
Las Cruces, NM 88001
Battelle
Amy Dindal
Zack Willenberg
Brian Boczek
Ryan James
505 King Ave.
Columbus, OH 43201
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 7 of 3 8
SECTION A
PROJECT MANAGEMENT
Al VERIFICATION TEST ORGANIZATION
The verification test will be conducted under the auspices of the U.S. Environmental
Protection Agency (EPA) through the Environmental Technology Verification (ETV) Program.
It will be performed by Battelle, which is managing the ETV Advanced Monitoring Systems
(AMS) Center through a cooperative agreement with EPA. The scope of the AMS Center covers
verification of monitoring, sampling, characterization, and detection technologies.
The day to day operations of this verification test will be coordinated and supervised by
Battelle, with the participation of the vendors who will be having the performance of their
technologies for radio frequency identification (RFID) for tracking hazardous waste shipments
across international borders evaluated. Testing will be conducted in the Santa Teresa, New
Mexico-Jeronimo, Mexico border crossing area. Each vendor will provide their respective
technology and will set it up and either operate their technology themselves during the
verification test or train verification staff to operate the technology during the verification test.
Either way, verification staff will be on-hand during testing to ensure that the test/QA plan is
followed.
The organization chart in Figure 1 identifies the responsibilities of the organizations and
individuals associated with the verification test. Roles and responsibilities are defined further
below. Quality Assurance (QA) oversight will be provided by the Battelle Quality Manager and
also by the EPA AMS Center Quality Manager, at her discretion.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 8 of 3 8
Battelle
Management
Amy Dindal
Battelle AMS
Center
Manager/Test
Leader
Zachary Willenberg
Battelle AMS Center
Quality Manager
EPA Region VI and
EPA/OECA HQ
Dr. Ryan James
Battelle
Verification
Test Coordinator
Battelle Technical
Staff
U.S. Technical Staff
(Border Writing)
Dr. John McKernan
EPA AMS Center
Project Officer
Michelle Henderson
EPA AMS Center
Quality Manager
RFID System Vendor
Representatives
Mexican Technical
Staff
(Border Writing)
Figure 1. Organization Chart for the Verification Test
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 9 of 3 8
Al.l Battelle
Dr. Ryan James is the AMS Center's Verification Test Coordinator for this test. In this
role, Dr. James will have overall responsibility for ensuring that the technical, schedule, and cost
goals established for the verification test are met. Specifically, Dr. James will:
Prepare the draft test/QA plan, verification reports, and verification statements.
Establish a budget for the verification test and manage staff to ensure the budget is
not exceeded.
Revise the draft test/QA plan, verification reports, and verification statements in
response to reviewers' comments.
Assemble a team of qualified technical staff to conduct the verification test.
Direct the team in performing the verification test in accordance with this test/QA
plan.
Hold a kick-off meeting, which will be conducted in person or by teleconference
approximately one week prior to the start of the verification test to review the critical
logistical, technical, and administrative aspects of the verification test. Responsibility
for each aspect of the verification test will be confirmed.
Ensure that all quality procedures specified in this test/QA plan and in the AMS
Center Quality Management Plan1 (QMP) are followed.
Serve as the primary point of contact for vendor representatives.
Ensure that confidentiality of sensitive vendor information is maintained.
Become familiar with the operation and maintenance of the technologies through
instruction by the vendors, if needed.
Respond to any issues raised in assessment reports, audits, or from verification staff
observations, and institute corrective action as necessary.
Coordinate distribution of the final test/QA plan, verification reports, and verification
statements.
Establish subcontract with Border Writing to assist Battelle in coordinating the
verification test near the border; especially the tasks requiring coordination with the
Mexican government, Mexican maquila, and the Mexican trucking companies.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 10 of 38
Subcontract language will require Border Writing to comply with all aspects of this
TQAP, and therefore, the AMS Center QMP.
Ms. Amy Dindal will serve as Verification Testing Leader and is also Battelle's Manager
for the AMS Center. Ms. Dindal will:
Support Dr. James in preparing the test/QA plan and organizing the testing.
Review the draft and final test/QA plan.
Attend the verification test kick-off meeting.
Review the draft and final test/QA plan.
Review the draft and final verification reports and verification statements.
Attend the verification test kick-off meeting.
Ensure that necessary Battelle resources are committed to the verification test.
Ensure that confidentiality of sensitive vendor information is maintained.
Support Dr. James in responding to any issues raised in assessment reports and audits.
Maintain communication with EPA's project officer.
Issue a stop work order if Battelle or EPA QA staff discovers adverse findings that
will compromise test results.
Battelle Technical Staff will support Dr. James in planning and conducting the
verification test. The responsibilities of the technical staff will be to:
Assist in planning for the test including coordination with vendors.
Attend the verification test kick-off meeting.
Conduct reference testing.
Support Dr. James in the preparation of test/QA plan and reports, as necessary
Support Dr. James in responding to any issues raised in assessment reports and audits.
Mr. Zachary Willenberg is Battelle's Quality Manager for the AMS Center.
Mr. Willenberg or a qualified designee will:
Review the draft and final test/QA plan.
Attend the verification test kick-off meeting.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 11 of 38
Conduct a technical systems audit at least once during the verification test.
Audit at least 10% of the verification data.
Prepare and distribute an assessment report for each audit.
Verify implementation of any necessary corrective action.
Request that Battelle's AMS Center Manager issue a stop work order if audits
indicate that data quality is being compromised.
Provide a summary of the QA/QC activities and results for the verification reports.
Review the draft and final verification reports and verification statements.
A1.2 Technology Vendors
The responsibilities of the technology vendors are as follows:
Review and provide comments on the draft test/QA plan.
Accept (by signature of a company representative) the final test/QA plan prior to test
initiation.
Provide an adequate amount of equipment (e.g., tags, readers, software, etc.) for the
verification test.
Provide all other equipment/supplies/reagents/consumables needed to operate their
technology for the duration of the verification test.
Provide training on the use of the technology, as necessary. If training is provided,
provide written consent and instructions for verification staff to carry out verification
testing, including written instructions for routine operation of their technology.
Provide maintenance and repair support for their technology, on-site if necessary,
throughout the duration of the verification test.
Review and provide comments on the draft verification report and statement for their
respective technology.
A1.3 EPA
EPA's responsibilities in the AMS Center are based on the requirements stated in the
r\
ETV QMP. The roles of specific EPA staff are as follows:
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 12 of 38
Ms. Michelle Henderson is EPA's AMS Center Quality Manager. For the verification
test, Ms. Henderson or her designee will:
Review the draft and review and approve the final test/QA plan.
Have the option of performing one external technical systems audit during the
verification test.
Notify the EPA AMS Center Project Officer of the need for a stop work order if the
external audit indicates that data quality is being compromised.
Prepare and distribute an assessment report summarizing results of the external audit.
Review draft verification reports and verification statements.
Dr. John McKernan is EPA's Project Officer for the AMS Center. Dr. McKernan will:
Review the draft test/QA plan.
Review final version of test/QA plan to ensure all applicable comments have been
addressed and then approve the final test/QA plan.
Issue a stop work order if adverse findings are discovered that would compromise
data quality.
Review the draft and final verification reports and verification statements.
Oversee the EPA review process for the test/QA plan, verification reports, and
verification statements.
Coordinate the submission of verification reports and verification statements for final
EPA approval.
U.S. EPA Region 6 personnel and EPA's Office of Enforcement and Compliance
Assurance will support the collaboration of U.S. and Mexican localities through existing network
of colleagues/acquaintances and provide peer reviewers for the test/QA plan and reports.
A1.4 Maquilas - Foreign corporation-owned manufacturing plants located in Mexican
border cities
At least one maquila (e.g., TYCO Electronics) will participate in the verification testing,
subject to schedule and availability. These facilities manufacture various products and are
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 13 of 38
foreign corporation-owned, but Mexican managed and staffed. They are often ISO 9000
certified, and utilize and produce a variety of hazardous materials. Multiple maquila facilities are
located in the Bermudez Industrial Park area. Each has its own loading dock, storage area,
material handling equipment, and loading personnel that can support the testing. The
responsibilities of the maquilas include the following:
Provide location for readers to be placed during verification activities in their
loading dock area.
Provide 110 volt power for use with the readers.
A1.5 Trucking Companies
Service de Transporte International y Local (STIL), a company that provides international
HAZMAT shipping will collaborate in the verification testing. The responsibilities of the trucking
company will include the following:
Load test packages onto trucks as described in the test/QA plan.
Drive truck following route described in test/QA plan, stopping to allow for
scanning at the specified locations.
A1.6 Border Writing
Border Writing is a small business located in Las Cruces, NM with experience with RFID
technologies that will provide verification test staff to record observations and conduct reference
measurements at all of the field locations. These staff will be familiar with the area and able to
travel back and forth between the U.S. and Mexico. In addition, Border Writing will coordinate
collaboration with the New Mexico border authority, the Mexican border authority, the Mexican
maquilas, and the trucking companies. All of Border Writing's activities will be guided by the
test/QA plan and AMS Center QMP. Border Writing staff will either ride along with the truck or
follow the truck in a separate vehicle. They will transport the collision tags so they can be added
to the truck at the maquila. Border Writing staff will also be responsible for documenting their
experience in setting up and operating the vendor technologies in order to capture the user-
friendliness of each technology.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 14 of 38
A2 BACKGROUND
The ETV Program's AMS Center conducts third-party performance testing of
commercially available technologies important to protecting the environment from
contamination. The purpose of ETV is to provide objective and quality assured performance
data on environmental technologies, so that users, developers, regulators, and consultants can
make informed decisions about purchasing and applying these technologies. Stakeholder
committees of buyers and users of such technologies recommend technology categories, and
technologies within those categories, as priorities for testing.
The purpose of this test/QA plan is to specify procedures for a verification test applicable
to commercial RFID technologies. The purpose of the verification test is to evaluate the
performance of the participating technologies in a field environment. In performing the
verification test, Battelle and its subcontractor will follow the technical and QA/QC procedures
specified in this test/QA plan and will comply with the data quality requirements in the AMS
Center QMP.1
A3 VERIFICATION TEST DESCRIPTION AND SCHEDULE
A3.1 U.S./Mexican Hazardous Material Transport
There are thousands of shipments of hazardous wastes that enter the United States each
year. These shipments must be properly handled and recycled or disposed according to
appropriate regulatory standards. The Resource Conservation and Recovery Act (RCRA)
requires cradle-to-grave tracking of hazardous wastes. At the present time, there is no automated
method to track these shipments from the generator facility to the receiving facility and a particular
gap exists in EPA's ability to track shipments of hazardous wastes originating in the Mexico
border zone and crossing into the U.S.
Therefore, the U.S.-Mexico border area presents a unique opportunity for testing this type
of technology, largely because of the presence of the maquila industry. Maquilas are foreign-
owned manufacturing plants that are located in Mexican cities that border the U.S. There are
more than 4,000 of these manufacturing plants along the border, and many of them produce
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 15 of 38
hazardous waste as a function of their manufacturing processes, including solvents, solder, plastic,
and metal waste - wastes that are historically not tracked as required by RCRA. A maquila is
the assembly portion of the manufacturing process, where raw materials, including hazardous
materials, are imported into Mexico for assembly. The finished products and wastes are then
shipped to the U.S. or other countries.
The La Paz Agreement is between the United States and Mexico for the Cooperation for the
Protection and Improvement of the Environment. Under this agreement, all hazardous wastes
generated by raw materials shipped into Mexico for use in maquilas must be shipped back to their
country of origin. Mexico does not classify the returned material as hazardous, but as a returned
product, and therefore does not submit a Notice of Intent (NOI) to the United States for the
export of hazardous waste for this material. The current process makes it difficult to develop an
accurate accounting of hazardous waste entering the United States from the maquilas and
does not provide for timely identification of shipments that do not reach their designated
receiving facilities. The lack of tracking of these wastes creates the possibility for waste to be
illegally abandoned. An enhanced tracking system that provides accurate, timely data to
regulatory officials would be beneficial in preventing this from occurring.
A3.2 RFID Systems
A networked radio-frequency identification (RFID) system may provide the technology
to address some of these concerns. RFID is an emerging, commercial-ready commodity tracking
technology that is being tested and implemented in a large number of applications worldwide. A
typical RFID system consists of four main components: tags, an encoder, readers, and central
processing unit. An RFID tag consists of a microchip and a flexible antenna sealed in a plastic-
coated inlay, which can be applied to or incorporated into a product for the purpose of
identification. The encoder writes information to the tag that is acquired by a reader.
The RFID system operates by transmitting data using radio waves for communication
between a tag and a reader, and ultimately to a database. Most tags contain at least two parts.
One part is an integrated circuit for storing and processing information, modulating and
demodulating an RF signal. The second part is the antenna that receives and transmits the
signal. The information being read is provided in the form of a unique Electronic Product
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 16 of 38
Code (EPC), which links to information in a database, and may be accessed on-site or remotely.
RFID tags come in three general varieties: passive, active, or semi-passive.
Passive RFID tags have no internal power supply. A small electrical current is induced in
the antenna by the incoming radio frequency signal. These electromagnetic waves received from
the reader induce a current within the integrated circuit in the tag to power up and transmit an
amplitude-modulated response back to the reader. Active RFID tags have their own internal power
source (i.e., battery), which energizes the integrated circuits and broadcast a signal to a reader.
Active tags have an increased read range that may increase read accuracy due to their ability to
conduct a "session" with a reader. Semi-passive tags are similar to active tags in that they have a
battery; however, it only powers the microchip and does not broadcast a signal. The RF energy
sent by the reader is transmitted back to the reader, similar to the passive tag. This verification test
will include systems that use active RFID tags that are read using readers that are mounted inside
the truck and tracked using external tracking systems (cell phone tower or global positioning
system based) as well as readers that are mounted outside the truck.
A3.4 Test Description
The verification test will be conducted in the El Paso, Texas/Ciudad Juarez, Mexico trade
area, which is among the largest border trade zones in the world. For safety purposes, non-
hazardous substances will be used in the field testing. In all other regards, the field testing will
reflect actual conditions of shipping hazardous waste across the U.S.-Mexico border. The
verification test will simulate shipments of hazardous waste contained in plastic and metal
drums and corrugated boxes through routine land transportation routes and across international
ports of entry in the El Paso/Ciudad Juarez trade area. RFID tags will be attached to various
types of packages and loaded onto trucks at the U.S. loading dock at the New Mexico Border
Authority. Semi-trailer trucks from local HAZMAT trucking companies will carry containers
in a loose configuration as well as tightly packaged pallets of waste containers along a typical
cross-border shipping route. Read points for the RFID technologies will be located at the New
Mexico Border Authority (NMBA) loading dock (simulating the U.S. factory), the Mexican port
of entry, a maquila, and at the State of New Mexico truck inspection station near the U.S. port of
entry. The RFID technologies will be evaluated based on their read accuracy on the various
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 17 of 38
types of packaging and, if the system requires external readers, with readers placed at various
distances from the truck. The verification test will be conducted during March of 2009 with
report writing occurring during April and May of 2009. Exact test dates will be determined based
on the schedules of the technology vendor participants and the supporting organizations.
A4 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA
Because of the nature of RFID technologies, no traditional reference samples or
performance evaluation audit samples (often used during a chemical monitoring verification test)
will be analyzed during this verification test. The function of RFID tags will be confirmed by a
Rohde and Schwarz FSH6 Spectrum Analyzer, an instrument that measures the frequency and
power of a radio signal, at the time of tagging the packages. In addition, the spectrum analyzer
will be used to measure the power of the signal at each NMBA and New Mexico truck inspection
facility reader. Table 1 shows all of the measurements to be collected throughout the verification
test. In addition to the frequency and power measurements, these measurements include hourly
outdoor meteorological conditions that will be obtained from the U.S. National Weather Service,
"in-trailer" meteorological and shock conditions that will be recorded by the ShocklogRD 298
equipped with a humidity, pressure, and temperature sensor. Both these instruments will be
factory calibrated and not require additional calibration during the test. Truck speeds at the
NMBA, the Mexico POE, and the maquila will not be dictated to the driver. The speeds at those
locations will be documented according to the speedometer of the truck (speeds based on safe
speeds in those scenarios). However, the truck speeds at the entrance to the New Mexico truck
inspection facility will be dictated to the driver and measured by a factory-calibrated Stalker
Sport Doppler radar gun. The meteorological data will be collected as part of the test, but
there are not specific meteorological criteria that will be required for testing to be performed, or
conversely, no meteorological criteria would require that testing be discontinued.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 18 of 38
Table 1. Data Collected During Verification Test
Measurement
RFID Frequency
Effective Radiated
Power (RF Energy)
Trailer Conditions
Environmental
Conditions
Semi-trailer
Truck
Velocity
Data Source
Rohde and Schwarz FSH6
Spectrum Analyzer
Rohde and Schwarz FSH6
Spectrum Analyzer
Shocklog RD298
Recorder with HPT
Sensor
U.S. National Weather
Service, AIRNow reports
Stalker Sport Doppler
Radar gun
Key Measurement
Frequency range from
100KHz to 6.0 GHz;
resolution: IHz
Maximum power: 20
dBm
Event-based shock,
vibration
measurement;
temperature, relative
humidity, and
pressure measured
once per minute
Hourly temperature,
relative humidity (RH),
pressure, wind speed
and direction ; daily
averages of particulates
Truck velocity during
tag reading
Units
Frequency
megahertz (MHz)
Power ratio of
Decibel referenced
to milliwatt (dBm)
Centimeter/second
(cm/s),
gravitational force
(g), °C, %RH, and
inches mercury
Degrees Celsius
(°C), parts per
million, %RH, bar,
miles per hour
(mph)
mph
Calibration
Factory certified
calibration
Factory certified
calibration
Factory certified
calibration
Data from third
party sources
Factory certified
calibration
A5 SPECIAL TRAINING/CERTIFICATION
If the vendors request that verification staff operate and maintain their technology during
the verification test, the vendors will train the verification staff prior to the start of testing.
Battelle will document this training with a consent form, signed by the vendor, which states
which specific verification staff have been trained to use their sensor. All truck drivers will be
licensed in both the U.S. and Mexico and have experience in crossing the U.S./Mexican border
with hazardous waste shipments.
A6 DOCUMENTATION AND RECORDS
The records for this verification test will include the test/QA plan, laboratory record books
(LRB), field data sheets, electronic files (both raw data and spreadsheets), and the final verification
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 19 of 38
report. LRBs and field data sheets will be maintained by verification staff at the field site during the
test, upon completion of the field test, will be transferred to the Verification Test Coordinator, then
following completion of the reporting process will be transferred to permanent storage at Battelle's
Records Management Office. All Battelle LRBs are stored indefinitely, either by the Verification
Test Coordinator or Battelle's Records Management Office. Results collected during the field
verification test will be transferred electronically to the Verification Test Coordinator on a daily basis
by the verification staff working at the field site. Section BIO further details the data recording
practices and responsibilities.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 20 of 3 8
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
The verification test will allow for performance evaluation under "real world" conditions.
Overall, the performance of the RFID technologies will be verified based on the following
parameters:
Accuracy -proper identification of the tagged containers at various locations, truck
speeds, package type, packaging configurations conditions, etc. Specifically, proper
identification is defined as the retrieval of all information available about the tagged
item according to the vendor's standard procedures. For example, if the vendor has
database fields available for a container number, container volume, and container
description; a proper identification would include the retrieval of all of those pieces of
information.
Precision - repeatability of the ratio of correct to incorrect RFID results
Interference of other RFID signals (collision test) - ability to discriminate the
hazardous waste tags from other commercially-available tags
Influence of confounding factors (container type, packing configuration and
placement of tags/containers, environmental conditions, internal trailer conditions,
and semi-trailer truck velocity)
Operational factors (ease of use, read time, technology cost, user-friendliness of
vendor software, troubleshooting/downtime, etc.)
The responses to these parameters will be collected as either qualitative or quantitative
observations. The quantitative information will consist of the data from tag readings under each
condition, as well as the RFID frequency and power measurements. Qualitative information will
include those observations relating to operational factors.
The evaluations will be performed according to the vendor's recommended procedures as
described in the user's instructions or manual, or during training provided by the vendor.
Calibration and maintenance of the technologies will be performed as specified by the vendor.
The results from the technologies will be recorded manually by the verification staff on
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 21 of 38
appropriate field data sheets or captured in an electronic data system and then transferred
manually or electronically for further data workup. The results from each technology will be
reported individually, but each technology will undergo the same testing so it is convenient for
end users to evaluate the ETV testing results.
Bl.l Setup of RFID Technologies and Field Testing Procedures
In the 1-2 days leading up to the start of the verification test, each participating vendor
will be responsible for setting up their RFID technologies according to their recommended
configuration for optimal performance. For some technologies, this might include the use of
readers that are handheld or mounted to a stand outside of the truck. Others may require a reader
mounted on the inside of the truck. Regardless of the technology, the requirement will be that
read events need to occur at the NMBA loading dock, arrival/departure area of the NMBA
loading dock, as well as the exit drive leading from the NMBA loading dock and at the maquila
in Mexico. In addition, one read event will occur at the Mexican port of entry (POE) and read
events at 15 + 3 and 25 + 3 miles per hour will be conducted on the entrance lane to the New
Mexico truck inspection facility. The vendors will need to set up the technologies in a way that
will allow for the completion of the scope described in the test/QA plan.
Table 2 provides a summary of the testing route read points. The route is designed to
simulate the import of raw materials to the Mexican maquila facilities and the return of the
hazardous waste generated by these materials. If the participating RFID technologies consist of
readers that are handheld or mounted to a fixed location outside of the truck, vendors can
configure the system (e.g., reader angles/heights) for optimal performance.
Each container will be tagged with a vendor's RFID tag and loaded onto the trailer. Two
readers attached to a portable stand will be used at read point (RP)1 A, the loading dock of the
NMBA. At this location, the truck will be stopped and several reads will be collected at
increasing distances from the truck. The two separate readers will be situated on a stand at 20° and
45° with respect to the plane of the truck with the stand placed four feet from the side of the parked truck
even with the front bumper (similar to the stand being alongside a road). Then, for the various distance
reads, the stand with the readers will be moved straight forward from that point five, 15, and 30 feet
(determined by a tape measure). RP IB is the entrance/exit lane of the NMBA loading dock. The
same stand with the two readers will be moved to the exit lane prior to the truck exiting the
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 22 of 3 8
Table 2. Summary of Testing Route Read Points
Location
Read
Points
Description
l.NMBA
Loading Dock
RP1A
RP1B
Arrival/departure area: if readers required, two separate readers will be situated on a
stand at 20° and 45° with respect to the plane of the truck with the pole placed four
feet from the side of the parked truck even with the front bumper (similar to the stand
being alongside a road) and then for the various distance reads, the readers will be
moved straight forward from that point five, 15, and 30 feet; while the truck will be
stopped for these reads, they are mimicking the scenario of a truck approaching a
reader 30,15, and five feet away on the side of the road;..
Readers (20° and 45° with respect to the road) for a slow-moving read at
the exit of the NMBA loading dock
2. Jeronimo
Mexico POE
RP2
Readers (20° and 45° with respect to the road) placed at POE.
3. Maquila
facility at Cd.
Juarez
RP3
Readers (20° and 45° with respect to the road) placed at entrance/exit lane
of the maquila for a slow moving read upon entrance and exit of the
maquila facility. Collision tags will be added at maquila.
4. NM Truck
Inspection
Facility
RP4
Readers (20° and 45° with respect to the road) placed at entrance. Truck will
make one pass at 15 mph and another pass at 25 mph. Collision tags will be
removed after first pass at 15 mph. Truck will drive around block and return
two times to pass at 15 and 25 mph
5. NMBA
Loading Dock
RP5A.
RP5B
Readers (20° and 45° with respect to the road) for a slow-moving read at the
entrance of the NMBA loading dock.
Repeat of five, 15, and 30 feet reads as performed at Location 1.
Unexpected Route Deviaton
At one point during each round-trip, the truck will deviate from the planned route by
at least one mile. Following the deviation, the truck will turn around and return to the
planned route at the same location it left.
facility. There, a slow-moving read upon exit of the storage facility will take place. No specific
speed will be dictated for this read event, just whatever speed the driver would normal drive in
that situation of exiting the NMBA.
RP2 will be at the Mexican POE. Two readers (20° and 45° with respect to the road) will
be situated next to the road for a slow moving read upon entrance into Mexico. Again, no speed
will be dictated for this read point, but the driver will document the speed the truck is traveling
when it passes the read point. RP3 will be at the maquila. This read event will be performed in a
similar way as RP1B at the exit of the NMBA storage facility. There will be a read upon the
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 23 of 3 8
entrance of the maquila facility. Then, after having the door of the trailer opened and the
collision tags added to the load and any load shifting documented, the truck will turn around and
there will be another read event as the truck exits the maquila facility and returns to the U.S.
following the same route. RP4 will be near the U.S. POE at the New Mexico truck inspection
facility. Two readers (20° and 45° with respect to the road) will be placed on a stand alongside
of the entrance to the truck inspection facility. The trucks will enter the facility and pass the
readers once at 15 + 3 mph. The truck will then stop and the collision tags will be removed from
the trailer. Then, the truck will drive around the block and pass the reader once again at 15 + 3
mph and then make another pass at 25 + 3 mph. RP5 will be back at the NMBA where the truck
started. RP5A will be at the entrance lane to the storage facility (this location is the same as
RP1B). A slow moving read event will take place upon return to the NMBA. RP5B will be
conducted in the same location and in a similar way as was RP1 A. The read events will be
conducted at five, 15, and 30 feet from the front of the truck. When possible, the vendors may
be able to utilize the same readers at several of the read point locations. At this time, the round
trip will be completed and all the read points will have been collected. The trucks used during
the test will either be the same truck across the duration of the verification test or at least of the
same size and construction. This process will be performed twice with loosely packed containers
and twice with tightly palletized containers for a total of four round trips for each vendor.
Both technologies being tested have a feature that alerts the user if the shipments leave
the route that is planned and entered into the vendor's system. In order to verify the performance
of that feather, route at some point during each round trip the truck will deviate from the planned
route by at least one mile, but not more than 3 miles. Following the deviation, the truck will
return to the planned route at the same point it left to deviate from the route. The vendors will
not be informed of where the deviation in route will occur.
B1.2 HAZMAT Containers and Packing Configuration
According to current shipping practices, most of the hazardous materials entering Mexico
from the United States are contained in either polyethylene (poly) or steel 55-gallon drums, and
much of the hazardous waste returning from the maquila and entering the United States is solid and
packed in one cubic yard corrugated boxes or drummed liquids. Accordingly, testing will be
completed using three different containers: 55-gallon poly drums, 55-gallon stainless steel drums,
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 24 of 3 8
and one cubic yard corrugated box. In the interest of safety, no actual hazardous waste will be
transported during this verification test, but containers that are typically used for hazardous
material transport will be used to best simulate actual conditions. The 55-gallon drums will be
filled with tap water and the corrugated boxes will contain non-hazardous cloth rags. The
verification of each technology will include four total round trips that will begin at the NMBA
loading dock.
Two round-trip shipments will include tightly packed, palletized poly and steel drums
and corrugated boxes and the other two round-trip shipments will include poly and steel drums
and corrugated boxes loosely packed on the floor of the trailer. It is anticipated that there will be
at least four of each type of container included with each truck run (for a total of 12 containers)
and that each vendor will be tested using the same HAZMAT containers. Upon loading of the
trucks, the vendors will attach one RFID tag to each container. Therefore, there will be a total of
four tags for each container type. The tags will be attached such that one container of each type
will have a tag that faces towards the front of the truck, one container will have a tag that faces
towards the rear of the truck, one container will have a tag that faces towards the passenger side
of the truck, and one container will have a tag that faces towards the driver side of the truck.
This configuration is meant to provide performance information about whether or not tag
placement is critical to the results. The packing configuration will be checked at the mid-point
of the route (at the maquila) and any shifting of the containers or damage to tags will be
documented. If any tags are damaged, we will consult with the vendor and consider replacing
them before progressing with the test.
B1.3 Trucking Route
As depicted on the map in Figure 1, the trucking route will occur between the NMBA
loading dock near Santa Teresa, New Mexico and the maquila(s) in Ciudad Juarez Mexico and
back again. It will also include passage through the Santa Teresa, New Mexico and Jeronimo,
Mexico International POE. The selected route replicates the typical HAZMAT routes across
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 25 of 3 8
Figure 2. Trucking Route Map (read points labeled)
the U.S.-Mexico border zone: it mixes urban and highway driving; it starts at the NMBA near the
Santa Teresa POE, comes to the halfway point at the maquila in a heavily industrialized section of
Cd. Juarez, and then returns following the same route. The length of this route is approximately
40 miles round-trip so we anticipate that it will take about one hour.
B1.4 Other Measurements
Bl.4.1 Spectrum Analyzer Measurements
As described in Section A4, the frequency and power of each tag placed on packages will
be verified by the spectrum analyzer (at short distances) upon being applied to the packages. In
addition, the power level of the tag signals will be read at each NMBA and New Mexico truck
inspection facility reader placement. The spectrum analyzer antennae will be placed next to the
readers prior to each read event at those locations.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 26 of 3 8
Bl.4.2 Collision Test
To evaluate the ability for readers to discriminate the tags used for tracking the
simulated hazardous waste containers from other commercially available tags (not from the
vendors being tested), a collision test will be completed along the truck route. This will be done
by placing a wooden block (or other available support) containing commercially obtained tags
(not from the vendor being verified) on the semi-trailer truck after completing the initial tag
reading at the maquila facility. The door of the trailer will be open and the tags will be placed
adjacent to the location at which the other tagged containers are located. At the NMBA read
point, the vendor tags will be read upon the initial 15 mph pass. After this reading, the block will
be removed from the truck, the doors closed up again, and the truck will make passes at 15 and
25 mph without the collision tags. This collision test will be performed with each round trip.
Therefore, eight reads will be made with the collision tags within the truck, all at the NMBA
read point at a truck speed of 15 mph.
Bl.4.3 Weather and Particulate Data
The temperature, precipitation, dew point, barometric pressure, humidity, wind speed, fine
particulate matter (PM^.s), and ozone will be recorded at the time the tags are read by the RFID
system. The temperature, precipitation, dew point, barometric pressure, humidity, and wind
speed will be recorded from the U.S. National Weather Service website for the El Paso/Juarez
region (www.srh.noaa.gov/elp/). The particulate and ozone readings will be based on the hourly
reading of the air quality monitors in the El Paso region as recorded on AIRNow, a U.S. cross-
agency government air quality control Web site (www.airnow.gov). The weather and particulate
data will be correlated with the read accuracy to assess their possible influence on the RFID
performance, should there exist sufficient variance in weather conditions or particle
concentrations during field testing.
Bl.4.4 Internal Trailer Conditions
The operating and storage temperature, vibration, and shock will be recorded for each test
using appropriate electronic monitoring devices such as Shocklog. This data will be
correlated with the read rate/accuracy data, humidity, temperature, barometric pressure, and shock
events to assess their possible influence on the RFID performance.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 27 of 3 8
Bl.4.5 Semi-Trailer Truck Velocity
Tag readings will be performed at four locations while the truck is traveling at a low
velocity upon entrance and exit of the NMBA loading dock, the maquila, and at the Mexico
POE. Truck speeds at these locations will not be dictated to the driver. The driver will
document the speed (according to the speedometer of the truck) at which it was practical to drive
in those scenarios. In addition, the New Mexico truck inspection facility read points will include
reads at 15 + 3 mph and 25 + 3 mph. These speeds will be dictated to the driver and confirmed
through measurement by a Stalker Sport Doppler radar gun.
Bl.4.6 Operational Factors
Operational factors such as maintenance needs, ease of use, data output, and software
requirements will be documented by the verification staff. LRBs or field data sheets will be used
to document observations. Examples of information to be recorded include the expertise required
for equipment setup and operation, the format of data exported by the technology and method by
which to obtain that data, ease and durability of tag, quality of tag attachment, difficulty of set-
up, required repair or maintenance, the duration and causes of any technology down time or data
acquisition failure, user-friendliness of any needed software, and the overall convenience of the
technologies and accessories/consumables. These observations will be summarized qualitatively
to aid in describing the technology performance in the verification report on each technology.
B1.5 Statistical Analysis
The statistical methods and calculations used for evaluating quantitative performance
parameters are described in the following sections.
Bl.5.1 Accuracy
Accuracy is a measure of the agreement between a measured value and the "true" value. A
primary objective for this verification test is to determine the accuracy of RFID system
performance with reading tags under critical variables and test conditions. For this verification,
accuracy will be determined as a percentage according to the following formula:
A = (l-E/N)x 100 (1)
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 28 of 3 8
where A is the percent accuracy of the RFID system reader, E is the total number of tags
that were not properly recognized by the reader, and N is the total number of tagged HAZMAT
containers. Tag reader accuracy will be determined for each read point and packaging type for
each vendor's system. The highest percent accuracy possible is 100%.
Result of the collision test will be reported as false positive and false negative results.
For example, the presence of the non-vendor RFID tags cause the vendor's tags not be read
successfully, that will reported as a false negative result. Alternatively, if the presence of the
collision tags cause additional readings that are not linked to a vendor tag, that will be reported
as a false positive result. The accuracy of the RFID systems will be reviewed in the context of
all the possible confounding factors (e.g., packaging type, direction of tag, trailer environmental
conditions, etc.). If possible, correlations will be considered through statistical approaches such
as linear regressions, analysis of variance, etc.
Bl.5.2 Precision
Precision is a measure of the agreement among repeated measurements. Each tag will be
read at least two or three times at each read point (depending on the read frequency of the vendor
systems), each read point creating a set of replicate results. At the read points where the truck is
moving, there will likely be fewer results than for the read points when the truck is stopped.
Each of these results can either be correct or incorrect identification of the tag. In order to
evaluate precision for this verification testing program, these number of correct identifications
within each read set will be reported. For example, throughout one truck trip, a single tag will be
read at seven different instances (RP1 A, RP1B, RP2, RP3, RP4, RP5A, and RP5B). Therefore,
if there would happen to be the same ratio of correct to incorrect results for each result set, that
would indicated a high level of precision. However, if the ratio of correct to incorrect results
fluctuated between sets of results, that would indicate poor repeatability. The ratio of correct to
incorrect results will be reported for each set of results.
B1.6 Reporting
The data obtained in the verification test will be compiled separately for each vendor's
technology, and the data evaluations will be applied to each vendor's technology without
reference to any other. Following completion of the data evaluations, a draft verification report
will be prepared for each vendor's technology, stating the verification test procedures and
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 29 of 3 8
documenting the results. For example, descriptions of the data acquisition procedures, use of
vendor-supplied proprietary software, consumables used, repairs and maintenance needed, and
the nature of any problems will be presented in the draft report. Each report will briefly describe
the ETV Program, the AMS Center, and the procedures used in verification testing. Each draft
verification report will be submitted for review by the respective technology vendor and by EPA
and other peer reviewers. Battelle will compile all the comments and revise the reports as
necessary. The reporting and review process will be conducted according to the requirements of
the AMS Center QMP.1
B2 SAMPLING REQUIREMENTS
B2.1 Sample Collection, Storage and Shipment
Because of the nature of RFID measurements, there will be no physical reference samples
collected during the verification testing.
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Because of the nature of RFID measurements, there will be no physical reference samples
collected during the verification testing.
B4 FIELD TEST REFERENCE METHOD
Because of the nature of RFID measurements, there will be no physical reference samples
for which standard measurement methods can serve as reference methods. However, as
described in previous sections, various measurements (e.g., the spectrum analyzer, the
meteorological conditions recorded by the NWS, "in-trailer" conditions measured by the
Shocklog, Stalker Sport Doppler radar gun, etc.) will be made to support the verification
test.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 30 of 38
B5 REFERENCE METHOD QUALITY CONTROL
Because of the nature of RFID measurements, there will no physical reference samples
collected during the verification testing. However, the instruments used for the various
measurements mentioned in Section B4 will be certified accurate by the manufacturer. In
addition to the factory calibration, the spectrum analyzer will be run through a daily preliminary
electronic diagnostic sequence before use in the verification testing. Completion of the diagnostic
sequence will be documented in the field data sheets.
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The RFID technologies will be setup and maintained by the vendors or at the direction of
the vendors according to their specifications. If the verification staff encounters problems with
the operation of the technology, the vendor representative will be contacted immediately.
B7 TECHNOLOGY CALIBRATION
As described throughout this test/QA plan, the spectrum analyzer will be used to confirm
the frequency and power of the technologies being verified. The RFID technologies being
verified will be supplied ready for use by the vendor and will not require calibration.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
All materials, supplies, and consumables will be ordered by the Verification Test
Coordinator or designee. Where possible, Battelle will rely on sources of materials and
consumables that have been used previously as part of ETV verification testing without
problems. Battelle will also rely on previous experience or recommendations from EPA
advisors, host facilities, or the RFID vendors.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 31 of 38
B9 NON-DIRECT MEASUREMENTS
As described previously, the NWS meteorological data will be used during this
verification test. Similarly, the AIRNow atmospheric particulate data will also be used. Both of
these data sources are highly regarded and are routinely used in government projects.
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by
Battelle during the verification test. Table 3 summarizes the types of data to be recorded. All
maintenance activities, repairs, and operator observations relevant to the technology operation
will be documented by technical staff in LRBs or on field data sheets.
Records received by or generated by any technical staff during the verification test will
be reviewed by a Battelle staff member within two weeks of generation or receipt, before the
records are used to calculate, evaluate, or report verification results. If a Battelle staff member
generated the record, this review will be performed by a Battelle technical staff member involved
in the verification test, but not the staff member who originally generated the record. The review
will be documented by the person performing the review by adding his/her initials and date to the
hard copy of the record being reviewed. In addition, any calculations performed by technical
staff will be spot-checked by Battelle QA and/or technical staff to ensure that calculations are
performed correctly. Calculations to be checked include any statistical calculations described in
this test/QA plan. The data obtained from this verification test will be compiled and reported
independently for each technology.
Among the QA activities conducted by Battelle QA staff will be an audit of data quality.
This audit will consist of a review by the Battelle AMS Center Quality Manager (or his designee)
of at least 10% of the test data. During the course of any such audit, the Battelle AMS Center
Quality Manager will inform the technical staff of any findings and any need for immediate
corrective action. If serious data quality problems exist, the Battelle AMS Center Quality
Manager will request that Battelle's AMS Center Manager issue a stop work order. Once the
assessment report has been prepared, the Verification Test Coordinator will ensure that a
response is provided for each adverse finding or potential problem, and will implement any
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 32 of 38
necessary follow-up corrective action. The Battelle AMS Center Quality Manager will ensure
that follow-up corrective action has been taken.
Table 3. Summary of Data Recording Process
Data to Be
Recorded
Tag ID and location
(what package and
location on package)
Spectrum analyzer
results
Read results
"In trailer" shock and
environmental
condition
Environmental
conditions
Environmental
conditions
Speed of truck
Operational factors
Where Recorded
Field data sheet
Field data sheet
Vendor software
Shocklog RD298
Recorder
U.S. National Weather
Service (NWS) web
site-El Paso area,
www. srh.noaa. sov/e
IE/
www.airnow.gov
Field data sheet
Field data sheet or
LRB
How Often
Recorded
Tag attachment
Tag attachment
After all U.S. read
points
During read events
Continuously during
transport
Daily
Daily
During read events
Throughout
verification testing
By Whom
Field verification
staff
Field verification
staff
Automated
Automated
Field verification
staff
Field verification
staff
Field verification
staff
Field verification
staff
Disposition of
Data
Verification test
binder
Verification test
binder
Files provided to
verification staff on
thumb drive or disk
Files provided to
verification staff on
thumb drive or disk
Verification test
binder
Verification test
binder
Verification test
binder
Verification test
binder or LRB
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 33 of 38
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in this verification test to anticipate and resolve potential
problems before the quality of performance is compromised. One of the major objectives of this
test/QA plan is to establish mechanisms necessary to ensure this. The procedures described in
this test/QA plan, which is peer reviewed by a panel of outside experts, implemented by the
technical staff and monitored by the Verification Test Coordinator, will give information on data
quality on a day-to-day basis. The responsibility for interpreting the results of these checks and
resolving any potential problems resides with the Verification Test Coordinator. Technical staff
have the responsibility to identify problems that could affect data quality or the ability to use the
data. Any problems that are identified will be reported to the Verification Test Coordinator, who
will work with the Battelle AMS Center Quality Manager to resolve any issues. Action will be
taken to control the problem, identify a solution to the problem, and minimize losses and correct
data, where possible. Independent of any EPA QA activities, Battelle will be responsible for
ensuring that the following audits are conducted as part of this verification test.
Cl.l Performance Evaluation Audits
Because of the nature of RFID measurements, a performance evaluation audit, as is
usually performed to confirm the accuracy of the reference method, will not be applicable during
this verification test.
C1.2 Technical Systems Audits
The Battelle AMS Center Quality Manager will perform a technical systems audit (TSA)
at least once during this verification test. The purpose of this audit is to ensure that the
verification test is being performed in accordance with the AMS Center QMP,1 and this test/QA
plan. In the TSA, the Battelle AMS Center Quality Manager or a designee may compare actual
test procedures to those specified or referenced in this plan, and review data acquisition and
handling procedures. The Battelle Quality Manager will tour the test sites, observe and review
the test procedures, and review LRBs/field data sheets. He will also check calibration
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 34 of 38
certifications for test measurement devices. A ISA report will be prepared, including a
statement of findings and the actions taken to address any adverse findings. The EPA AMS
Center Quality Manager will receive a copy of Battelle's ISA report. At EPA's discretion, EPA
QA staff may also conduct an independent on-site ISA during the verification test. The ISA
findings will be communicated to technical staff at the time of the audit and documented in a
ISA report.
C1.3 Data Quality Audits
The Battelle AMS Center Quality Manager will audit at least 10% of the verification data
acquired in the verification test. The Battelle AMS Center Quality Manager will trace the data
from initial acquisition, through reduction and statistical comparisons, to final reporting. All
calculations performed on the data undergoing the audit will be checked.
C1.4 QA/QC Reporting
Each assessment and audit will be documented in accordance with Sections 3.3.4 and
3.3.5 of the AMS Center QMP.1 The results of the TSA will be submitted to EPA. Assessment
reports will include the following:
Identification of any adverse findings or potential problems
Response to adverse findings or potential problems
Recommendations for resolving problems
Citation of any noteworthy practices that may be of use to others.
C2 REPORTS TO MANAGEMENT
The Battelle AMS Center Quality Manager, during the course of any assessment or audit,
will identify to the technical staff performing experimental activities any immediate corrective
action that should be taken. If serious quality problems exist, the Battelle AMS Center Quality
Manager is authorized to request that Battelle's AMS Center Manager issue a stop work order.
Once the assessment report has been prepared, the Verification Test Coordinator will ensure that
a response is provided for each adverse finding or potential problem and will implement any
necessary follow-up corrective action. The Battelle AMS Center Quality Manager will ensure
that follow-up corrective action has been taken. The test/QA plan and final report are reviewed
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 35 of 38
by EPA AMS Center Quality Manager and EPA AMS Center Project Officer. Upon final review
and approval, both documents will then be posted on the ETV website (www.epa.gov/etv).
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 36 of 38
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are stated in Section BIO of
this test/QA plan. In general, the data review requirements specify that the data generated during
this test will be reviewed by a Battelle technical staff member within three weeks of data
generation. The reviewer will be familiar with the technical aspects of the verification test, but
will not be the person who generated the data. This process will serve both as the data review
and the data verification, and will ensure that data have been recorded, transmitted, and
processed properly. Furthermore, this process will ensure that the RFID technology data are
collected under appropriate testing conditions.
D2 VALIDATION AND VERIFICATION METHODS
As part of the normal data and report review process the US EPA and ETV peer
reviewers will have the opportunity to review the draft final report and provide comments. Data
verification is conducted as part of the data review, as described in Section BIO for this test/QA
plan. A visual inspection of handwritten data will be conducted to ensure that all entries were
properly recorded or transcribed and that any erroneous entries were properly noted (i.e., single
line through the entry with an error code and the initials of the recorder and date of entry).
Electronic data from the technologies and other instruments used during the test will be inspected
to ensure proper transfer from the data logging system. Data manually incorporated into
spreadsheets for use in calculations will be checked against handwritten data to ensure that
transcription errors have not occurred. All calculations used to transform the data will be
reviewed to ensure the accuracy and the appropriateness of the calculations. Calculations
performed manually will be reviewed and repeated using a handheld calculator or commercial
software (e.g., Excel). Calculations performed using standard commercial office software (e.g.,
Excel) will be reviewed by inspecting the equations used in calculations and verifying selected
calculations by handheld calculator. Calculations performed using specialized commercial
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 37 of 38
software will be reviewed by inspection and, when feasible, verified by handheld calculator, or
standard commercial office software.
To ensure that the data generated from this test meet the goals of the test, a number of
data validation procedures will be performed. Section C of this test/QA plan provides a
description of the validation safeguards employed for this verification test. Data validation and
verification efforts include the completion of QC activities and the performance of a TSA audit
as described in Section C. A data quality audit will be conducted by the Battelle AMS Center
Quality Manager to ensure that data review, verification, and validation procedures were
completed, and to assure the overall data quality.
D3 RECONCILIATION WITH USER REQUIREMENTS
The purpose of a verification test performed following this test/QA plan is to evaluate the
performance of commercial RFID technologies for tracking hazardous waste shipments across
international borders. The data obtained in such a verification test will include thorough
documentation of the technology's performance during the verification test. The data review,
verification, and validation procedures described above will assure that verification test data
meet these requirements, are accurately presented in the verification reports generated from the
test, and that data not meeting these requirements are appropriately flagged and discussed in the
verification reports. Additionally, all data generated using the reference method, which are used
to evaluate technology results during the verification test, should meet the QA requirements of
any applicable standard operating procedures or instrumentation instruction manuals.
This test/QA plan and any resulting ETV verification report(s) generated following
procedures described in this test/QA plan will be subjected to review by participating technology
vendors, ETV AMS Center staff, test collaborators, EPA, and external expert peer reviewers.
These reviews will assure that this test/QA plan, verification test(s) of RFID technologies, and
the resulting report(s) meet the needs of potential users and regulators. The final report(s) will be
submitted to EPA in 508 compliant Adobe Portable Document Format (pdf) and subsequently
posted on the ETV website.
-------
Test/QA Plan for Verification of Radio Frequency
Identification (RFID) Tracking
Version 1.0
3/23/2009
Page 38 of 38
SECTION E
REFERENCES
El REFERENCES
1. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version 7.0,
U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio,
November, 2008.
2. Quality Management Plan for the Environmental Technology Verification Program, Version
3.0, U.S. EPA, Cincinnati, Ohio, January, 2008.
------- |