September 2009
Environmental Technology
Verification Report
AVANTE International Technology,
Inc.
PanaSec Mobile Personnel and Asset
Visibility System
Prepared by
Battelle
Batt6ll6
The Business of Innovation
Under a cooperative agreement with
CmtA U.S. Environmental Protection Agency
ETV ETV EtV
-------
September 2009
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
AVANTE International Technology, Inc.
PanaSec Mobile Personnel and Asset Visibility
System
by
Ryan James, Brian Boczek, Zachary Willenberg, Amy Dindal, Battelle
Deborah Kopsick, Carlos Rincon, Michelle Henderson, and John McKernan, U.S. EPA
-------
Notice
The U.S. Environmental Protection Agency, through its Office of Research and Development,
funded and managed, or partially funded and collaborated in, the research described herein. It
has been subjected to the Agency's peer and administrative review and has been approved for
publication. Any opinions expressed in this report are those of the author (s) and do not
necessarily reflect the views of the Agency, therefore, no official endorsement should be inferred.
Any mention of trade names or commercial products does not constitute endorsement or
recommendation for use.
11
-------
Foreword
The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the
Nation's land, air, and water resources. Under a mandate of national environmental laws, the
Agency strives to formulate and implement actions leading to a compatible balance between
human activities and the ability of natural systems to support and nurture life. To meet this
mandate, EPA's research program is providing data and technical support for solving
environmental problems today and building a science knowledge base necessary to manage our
ecological resources wisely, understand how pollutants affect our health, and prevent or reduce
environmental risks in the future.
The National Risk Management Research Laboratory (NRMRL) is the Agency's center for
investigation of technological and management approaches for preventing and reducing risks
from pollution that threaten human health and the environment. The focus of the Laboratory's
research program is on methods and their cost-effectiveness for prevention and control of
pollution to air, land, water, and subsurface resources; protection of water quality in public water
systems; remediation of contaminated sites, sediments and groundwater; prevention and control
of indoor air pollution; and restoration of ecosystems. NRMRL collaborates with both public and
private sector partners to foster technologies that reduce the cost of compliance and to anticipate
emerging problems. NRMRL's research provides solutions to environmental problems by:
developing and promoting technologies that protect and improve the environment; advancing
scientific and engineering information to support regulatory and policy decisions; and providing
the technical support and information transfer to ensure implementation of environmental
regulations and strategies at the national, state, and community levels.
This publication has been produced as part of the Laboratory's strategic long-term research plan.
It is published and made available by EPA's Office of Research and Development to assist the
user community and to link researchers with their clients.
Sally Gutierrez, Director
National Risk Management Research Laboratory
111
-------
Acknowledgments
The authors wish to acknowledge the contribution of the New Mexico Border Authority for
hosting this verification at the Santa Teresa Port of Entry (POE) and, in particular, the efforts of
Mr. Marco Herrara in providing his export expertise in support of this verification test. We thank
Dr. Barry Thatcher of BorderWriting, for supporting this verification test, but specifically for his
coordination with the Mexican Customs officials along with all other local participants. Also, we
acknowledge the cooperation of the New Mexico Department of Public Safety in providing read
locations and Servicio de Transporte Internacional y Local in providing a truck and driver and
Ivan Calzada of the Texas Transportation Institute. Finally, we thank Ms. Debra Tellez, Ms.
Katrina Varner, and Mr. Israel Anderson of the U.S. EPA, Mr. Stephen Niemeyer of the Texas
Commission on Environmental Quality, and Mr. Jose Mario Sanchez Soledad, Comision de
Ecologia y Protection Civil, Gobierno Municipal De Juarez for their review of the test/QA plan
and/or this verification report.
iv
-------
Contents
Page
Foreword iii
Acknowledgments iv
List of Abbreviations vii
Chapter 1 Background 1
Chapter 2 Technology Description 2
Chapter 3 Test Design and Procedures 4
3.1 Test Overview 4
3.2 Experimental Design 5
3.2.1 RFID Tags and HAZMAT Waste Containers 5
3.2.2 Waste Container Configuration in the Semi-Trailer 5
3.2.3 Meteorological and Shock Data 6
3.2.4 Collision Test 7
3.2.5 Truck Routes and Descriptions of Round Trips 7
3.2.6 Route Deviation 11
3.3 Qualitative Evaluati on Parameters 12
Chapter 4 Quality Assurance/Quality Control 13
4.1 Audits 13
4.1.1 Technical Systems Audit 13
4.1.2 Data Quality Audit 13
4.2 QA/QC Reporting 13
4.3 Data Review 14
Chapter 5 Statistical Methods 15
5.1 Accuracy 15
5.2 Precision 15
5.3 Influence of Possible Confounding Factors 16
Chapter 6 Test Results 17
6.1 Accuracy 17
6.1.1 External Reader Accuracy 17
6.1.2 Relayer Uplinked Accuracy 19
6.2 Precision 21
6.3 Interference of Other RFID Signals 21
6.4 Influence of Confounding Factors 22
6.5 Operational Factors 23
Chapter 7 Performance Summary 25
Chapter 8 References 28
v
-------
Figures
Figure 2-1. AVANTE's RI II) Tags 2
Figure 2-2. AVANTE's External Readers 3
Figure 2-3. AVANTE's Uplinked System (Relayer) 3
Figure 3-1. RFID tag affixed to poly drum 5
Figure 3-2. Tightly-packed configuration 6
Figure 3-3. Loosely-packed configuration 6
Figure 3-4. Collision Tags 7
Figure 3-5. Stationary Tag Read with External Readers 8
Figure 3-6. U.S. route used during testing (border area enlargement) 8
Figure 3-7. U.S. route used during testing 9
Figure 3-8. Mexico route used during testing (border area enlargement) 10
Figure 3-9. Mexico route used during testing 11
Figure 6-1. Screenshot from AVANTE's Software 24
Tables
Table 3-1. Summary of Round Trips 11
Table 6-1. Accuracy - External Readers 18
Table 6-2. Accuracy - In-truck Reader and Uplinked Communication 20
Table 6-3. Data Completeness Due to Continuity of Data Unlink 21
Table 6-4. Overall Accuracy ± Standard Deviation of Each RT 21
Table 6-5. Collision Test Results 21
Table 6-6. Accuracy Results by Container Type (in percent) 22
Table 7-1. Accuracy and Precision Summary for the PAVS 25
vi
-------
List of Abbreviations
AMS
Advanced Monitoring Systems
AVANTE
AVANTE International Technology, Inc
EPA
United States Environmental Protection Agency
ETV
Environmental Technology Verification
GPS
Global positioning system
GPRS
General packet radio service
GHz
Gigahertz
HAZMAT
Hazardous materials
LRB
Laboratory record book
MHz
Megahertz
mph
Miles per hour
MX
Mexico
NMBA
New Mexico Border Authority
NMDPS
New Mexico Department of Public Safety
NRMRL
National Risk Management Research Laboratory
PAVS
PanaSec Mobile Personnel and Asset Visibility System
poly
Polyethylene
POE
Port of Entry
QA
Quality assurance
QMP
Quality Management Plan
RFID
Radio-frequency identification
RT
Round Trip
SATCOM
Satellite communication
SD
Standard deviation
STIL
Servicio de Transporte Internacional y Local
TQAP
Test Quality Assurance Plan
U.S.
United States
VDC
volts direct current
vii
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Risk Management Research Laboratory (NRMRL) and its verification
organization partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under
ETV. ETV AMS Center verifies the performance of technologies for monitoring, sampling, and
characterizing contaminants and natural species in a variety of matrices including air, water, and
soil. The AMS Center evaluated the performance of the PanaSec Mobile Personnel and Asset
Visibility System by AVANTE International Technology, Inc. (AVANTE), a web-based real-time
locating and reporting system, in tracking hazardous materials (HAZMAT) being returned to the
U.S. from Mexico under terms of the La Paz Agreement. Under this agreement, all HAZMAT
waste generated by raw materials shipped into Mexico for use in foreign-owned factories (called
maquilas) must be shipped back to their country of origin. Mexico does not classify the returned
material as hazardous, but as a returned product, and therefore does not submit a Notice of Intent
to the United States for the export of such HAZMAT waste. The current process makes it
difficult to develop an accurate accounting of HAZMAT waste entering the United States from
the maquilas and does not provide for timely identification of shipments that do not reach their
designated receiving facilities. The lack of tracking of these wastes creates the possibility for
waste to be illegally abandoned. An enhanced tracking system that provides accurate, timely data
to regulatory officials would be beneficial in preventing this from occurring. This verification
test evaluated the performance of such tracking technologies.
1
-------
Chapter 2
Technology Description
This verification report provides results for the verification testing of AVANTE's PanaSec
Mobile Personnel and Asset Visibility System (hereafter referred to as PAVS). Following is a
description of two configurations of AV ANTE's PAVS that were used during this verification
test based on information provided by the vendor.
PAVS is a web-based real-time locating and reporting system that provides real-time tracking of
HAZMAT in transit or in storage as well as the whereabouts of the responsible personnel. This
HAZMAT transportation tracking system and solution incorporates patented radio-frequency
identification (RFID) technologies coupled with global positioning system (GPS)-general packet
radio service (GPRS) cellular data communications to assist in HAZMAT transportation
security. The AVANTE data center server stores all transit and condition data to facilitate real-
time tracking of the HAZM AT.
Each container of HAZMAT is individually tagged with an active ZONERIM tag. Figure 2-1
shows these active RFID tags that get attached to containers with pressure-sensitive adhesive or
other mechanical fasteners. The identity of each
container is linked to the active tag identification.
During verification testing, AVANTE's PAVS used two
different methods for RFID tag reading and
communication: (1) external roadside readers and (2) an
uplinked system using a vehicle mounted and powered
RELAYER™.
External Readers. The roadside reading configuration
of the PAVS consisted of pole-mounted antennae
coupled with an external tag data storage device. These
are shown in Figure 2-2 and are referred to in this report
as external readers. The external readers are located in
a fixed position (e.g., on the side of a road or at the
entrance to a facility); this location is referred to as a
read location. As the RFID tags come within range of
the external readers (e.g., as the truck drives past), the
external readers collect the information transmitted by
the RFID tags and records this information onto a
secure digital memory card loaded into the external
reader. The information collected by the external reader
is collected by removing the memory card and
downloading the information to a computer.
0 A vante
Tag #:0040000056
H 00000M)0:#
Tag #:0040000047
SI00000t-00:#8el
9}UOAy
0W0000M)0:#
dlUDAy
Figure 2-1. A VANTE's RFID Tags
2
-------
Figure 2-2. AVANTE's External
Readers
Uplmked System. The ZONER™-RELAYER™
configuration (referred to as "Uplinked System")
is capable of collecting information from RFID
tags while the shipment is either stationary or in
transit. It functions by equipping each carrier
with a monitoring reader/transponder unit called
the RELAYER™-CTCR (Relayer) (shown in
Figure 2-3) that communicates with all of the
active RFID tags inside the container or trailer.
Each Relayer is equipped with GPS and GPRS
and/or SATCOM capabilities. The Relayer
transmits all transit and condition data to the
AVANTE data center servers. This information
may then be viewed using a world-wide web
application providing near real-time viewing.
The uplinked system also allows for the use of personnel badges that identify him/her along with
their proximity to the Relayer and the "panic button" feature.
Prior to the start of the verification test,
AVANTE was responsible for setting up the
PAVS technology using both the external readers
and the uplinked system according to their
recommended configuration for optimal
performance. This included configuring the
system to ensure that read events occur at each of
the testing route read points. Both the external
readers and Relayer are powered by 12 VDC and
backup battery. The Relayer system was
configured to download the RFID data to the
AVANTE data center server via cellular
telephone communication link.
Figure 2-3. AVANTE's Uplinked
System (Relayer)
3
-------
Chapter 3
Test Design and Procedures
3.1 Test Overview
This verification test was conducted according to procedures specified in the Test/QA Plan for
Verification of Radio Frequency Identification (RFID) for Tracking Hazardous Waste Shipments
across International Borders^ (TQAP) and adhered to the quality system defined in the ETV
AMS Center Quality Management Plan (QMP).(2) Battelle conducted this verification test with
support from the New Mexico Border Authority (NMBA), New Mexico Department of Public
Safety (NMDPS), Texas Transportation Institute, U.S. EPA Region 6 El Paso Border Office,
U.S. EPA Office of Enforcement and Compliance Assurance, BorderWriting (a New Mexico
company that coordinated participation of the local collaborators prior to the ETV test), and
Servicio de Transporte Internacional y Local (STIL).
This verification test simulated shipments of HAZMAT waste contained in polyethylene (poly)
drums, metal drums, and corrugated boxes through routine land transportation routes and across
international ports of entry in the El Paso/Ciudad Juarez trade area. Originally, this ETV test
was planned with the expectation that all of the trucking routes would include border crossings.
However, due to concern of local authorities related to the violence in Ciudad Juarez during the
test, there were some difficulties in obtaining permission to cross the border into Mexico (MX),
so two of the trucking routes did not cross into MX and two routes included crossing the border
into MX. RFID tags were attached to various containers and loaded onto a truck at the U.S.
loading dock at the NMBA facility using a standard 53-foot semi-truck and trailer provided by
STIL, a local trucking company. Throughout the testing, the containers were arranged in the
trailer in either a tight-packed or loose-packed orientation. The PAVS included the RFID tags
(attached to HAZMAT waste containers) and readers (roadside external or an in-truck reader
(referred to as the Relayer) that resided in the cab of the truck and uplinked data to a central
server in near real-time. The truck then left the NMBA loading dock, drove a prescribed route
either solely in the U.S. or across the U.S. - MX Border. RFID tag reads were recorded
electronically throughout each truck route.
This verification test was conducted from March 24-26, 2009 at the NMBA Santa Teresa facility
and other field locations throughout the El Paso/Ciudad Juarez trade area. The PAVS was
verified by evaluating the following parameters:
• Accuracy - proper identification of the tagged containers at various locations, at various
truck speeds, on corrugated boxes or steel and poly 55-gallon drums, and in tightly
packed and loosely packed loading orientations. Specifically, proper identification is
defined as the retrieval of all information available about the tagged item according to the
vendor's standard procedures.
4
-------
• Precision - standard deviation (SD) of percent accuracy RFID tag read results.
• Interference of other RFID signals (collision test) - ability to discriminate the tags on the
HAZMAT waste containers from other commercially-available RFID tags.
• Influence of confounding factors - container type, packing configuration and placement
of tags/containers, environmental conditions, and internal trailer conditions.
• Operational factors - ease of use, technology cost, user-friendliness of vendor software,
troubleshooting/downtime, etc.
3.2 Experimental Design
3.2.1 RFID Tags and HAZMAT Waste Containers
At the beginning of each day of testing, verification staff verified the function of 12 AVANTE
RFID tags to be used as part of this verfication testing by measuring the frequency and the
effective radiated power of the tag using a Rhode and Schwartz FSH6 spectrum analyzer with a
435 Megahertz (MHz) antenna. This was done by placing all the RFID tags involved in the test
into the trunk of an automobile thereby i solating the signals emitted by each RFID tag from the
spectrum analyzer. Next, a single RFID tag was removed from the trunk, the trunk was again
closed, and the RFID tag was taken into the NY1BA office building where the frequency and the
effective radiated power was measured using the spectrum analyzer. This process was repeated
for all 12 AVANTE RFID tags and the four RFID tags used for the collision test.
According to current shipping data, most of the HAZMAT entering MX from the U.S. are
contained in either poly or steel 55-gallon drums, and much of the HAZMAT waste returning
from the maquilas and entering the U.S. is solid and packaged in one-cubic yard corrugated
boxes or as drummed liquids. Therefore, when the functioni ng of the RFID tags had been
confirmed, four RFID tags were secured to
poly 55-gallon drums, four were secured to
the steel 55-gallon drums, and four were
affixed to corrugated boxes for a total of
twelve individual containers. One tag was
affixed to each individual container. Figure
3-1 is a photo of an AVANTE RFID tag
affixed to a poly drum. In the interest of
safety, no actual HAZMAT waste was
transported during the verification test. Each
poly and metal 55-gallon drum used in the
verification testing was filled with tap water
and each corrugated box was filled with
loosely folded cardboard. RFID tags were
secured to the top of each poly and metal
dnim and to the side of each corrugated box
using Velcro tape.
3.2.2 Waste Container Configuration in the Semi-Trailer
Each round trip (RT) conducted in the U.S. was performed using a tightly packed configuration
of the HAZMAT waste containers. Each RT into MX was performed using a loosely packed
configuration.
Figure 3-1. RFID tag affixed to poly drum
5
-------
Each packaging configuration consisted of 12 individual containers. Two metal drums were
placed on a pallet and the drums were shrink-wrapped to secure the drums during transport. The
process was repeated for the poly drums as well. Therefore, four pallets of drums were created;
two pallets of two metal drums each and two pallets of two poly drums each. The corrugated
boxes were not affixed to a pallet but placed directly on the floor of the trailer. The corrugated
boxes and the pallets of drums were then positioned inside the truck trailer in a tightly-packed
configuration. For the loosely-packed configuration, each pallet contained one of each kind of
drum. Figures 3-2 and 3-3 show diagrams of tightly packed and loosely packed container
configurations.
Front of Truck
o
T3
in
_y>
Q
>
Q
i i
••••
PdPd
Q)
T3
in
Q)
DO
C
0>
Rearof Truck
i—1 ^
| l Pallet 0 55-gallon Metal Drum
~ 1 Cubic Yard o 55-gallon Poly Drum
Corrugated Box
Figure 3-2. Tightly-packed
configuration
1
"rontof Trucl
c
1
1 1
1
»~
1
o
T3
QJ
T3
in
y>
d)
>
Q
~a?
_y)
o
tuo
c
o>
to
nj
j
H
1
Rearof Truck
| I Pallet
0 55-gallon Metal Drum
I 1 1 Cubic Yard O 55-gallon Poly Drum
1 1 Corrugated Box
Figure 3-3. Loosely-packed
configuration
Regardless of packing configuration, each corrugated box was positioned such that one RFID tag
faced each side of the trailer (i.e., one RFID tag faced the driver's side, the passenger's side, the
front, and the rear of the trailer).
3.2.3 Meteorological and Shock Data
After the palletized poly and metal 55-gallon drums and the corrugated boxes were loaded into
the trailer and positioned in the correct packaging configuration, a Shocklog™ RD 298 system
(Shocklog) was installed directly onto the floor of the trailer and a calibrated hot wire
anemometer (TSI Incorporated, VelociCalc 9555-P Multi-function Ventilation Meter), capable
of measuring temperature, barometric pressure, and relative humidity, was affixed to the
passenger's side of the trailer approximately two feet above the floor. At the end of each day of
verification testing, the electronic data generated and captured by the Shocklog and the
anemometer were transferred from the instrument to a computer by means of a portable drive.
6
-------
3.2.4 Collision Test
Figure 3-4. Collision Tags
A collision test was performed during each RT to evaluate the ability for the PAVS to
discriminate the AVANTE RFID tags from other commercially available active tags. Battelle
supplied four commercially available tags for collision testing; the collision tags were 433 MHz,
the same frequency at which the PAVS operated, active tags (Wavetrend' TG801). The
collision tags were affixed to a wooden block as shown in Figure 3-4. At the beginning of the
day of testing, the function of each collision tag was verified separately by measuring the
frequency and the effective radiated
power of the tags using the
spectrum analyzer.
A collision test was performed
during each verification test run at
the NMDPS truck inspection
facility. The truck conducted its
first pass of the 25 and 15 mph read
locations. The truck then began its
second pass of the 25 mph read
location. After passing the 25 mph
read location the second time, the
truck stopped, the trailer was
opened, and the collision tags were
added to one of the one-cubic yard
corrugated boxes. The trailer doors
were then closed and the truck
proceeded past the 15 mph read
location. After passing the 15 mph read location, the truck stopped and the collision tags were
removed.
3.2.5 Truck Routes and Descriptions of Round Trips
The TQAP1 was written with the expectation that the same trucking route would be used
throughout the verification test and that the route would include crossing over into MX.
However, there was some difficulty in obtaining permission to cross the border into MX, so two
of the truck RTs were performed within the U.S. and when the proper permission was obtained,
the other two RTs were performed crossing the border into MX. Also, the TQAP was written
with the assumption that the technologies to be evaluated would have external readers that would
be set up at various read points throughout the trucking route. While this was the case for
AVANTE, AVANTE's technology also had a feature that uplinked the RFID data, as well as
GPS coordinates, to a server via a cellular telephone communication link at approximately one
minute time intervals, reporting the presence and location of each RFID tags at each time
interval. Therefore, fixed read locations were used for the AVANTE external readers and the
uplinked data were reported every minute to provide additional tag identification and location
data throughout the RTs than specified in the TQAP. The two RTs performed in the U.S. and the
two RTs including the border crossing into MX are described below.
7
-------
U.S. Trucking Route. The U.S. routes (RT 1 and 2) were selected to mimic, as much as possible,
the read locations presented in the TQAP and to challenge the PAVS under similar test
conditions specified in the TQAP.
The HAZMAT waste containers
were loaded into the trailer and
placed in the tightly-packed
configuration for both of the U.S.
RTs. The truck began all of the RTs
(U.S. and MX) at the NMBA facility
Prior to embarking on each RT, an
external reader was situated on a
stand at 90° with respect to the road
with the stand placed four feet from
the side of the front bumper. This is
shown in Figure 3-5. At that time,
RFID tag reads were made at
distances of 5, 15, 30, 50, and 70 feet
from the front bumper of the
stationary truck by moving the
external reader straight forward from
the initial placement of the reader.
The external readers were kept at
each distance for approximately one
minute. Following the stationary
RFID reads, the reader was moved to
the exit of the NMBA and the truck
embarked on its RT by first driving
past the reader for a slow moving
read. During only the start of the
first U.S. RT, a second reader was
placed at 45° with respect to the
road, in addition to the reader at
each read point that was placed at
90°.
Upon exit from the NMBA, the
truck travelled to the NMDPS
facility. Figure 3-6 shows the path
of the truck between the NMBA
and the NMDPS facility. The total
distance between the two locations
was approximately 0.75 miles.
External readers were placed at two
locations at the facility. The first
reader was designated as a 25 mile
per hour (mph) read location and
the second reader was designated a
15 mph read location. The readers
used here were also placed at 90°
CjEQ'utaDsCs tosoxaafes) Up skjCd
r> IM'ulra3eOGE(jc33(Ito(^
Figure 3-6. U.S. route used during testing (border
area enlargement)
Figure 3-5. Stationary Tag Read with External
Readers
8
-------
with respect to the plane of the truck
As the truck approached both the 25
mph and 15 mph read points, the
speed at which the truck was
travelling was measured using a
Stalker Sport1Ni 24.15 Gigahertz
(GHz) Doppler radar gun. The truck
then exited the NMDPS facility and
doubled back to make a second pass.
After passing the 25 mph read
location, the truck was stopped, the
trailer doors opened, the contents of
the trailer inspected for any shifting
of the load, and collision tags were
added. The truck then passed the 15
mph read location. After passing the
15 mph read location, the truck was
stopped, the trailer doors opened, and
the collision tags were removed. As
was the case with the first pass of the
truck through the NMDPS facility,
the speed at which the truck was
travelling past the 25 mph and 15 mph read location was measured using the radar gun.
The truck then proceeded to the Sunland Park Racetrack and Casino parking lot (referred to as
the casino read location) as is shown in Figure 3-7. This location was selected because the route
to the casino (approximately 13 miles) provided highway and city driving and the parking lot at
the casino provided adequate space for the external reader to be set up as well as space for the
truck to turn around. From the NMDPS facility, the truck travelled northbound on Peter V.
Domenici Boulevard, east on Airport Road, southeast on McNutt Road (NM-273) and finally
north on Sunland Park Drive to the Sunland Park Racetrack and Casino parking lot. The external
reader was placed at the entrance of the parking lot and upon arrival, the truck passed the reader
for a slow moving read. Again, the reader was situated at 90° with respect to the road. The
speed of the truck passing this read location was not dictated to the driver, but was documented
on a data sheet (from a speedometer reading) by a passenger in the truck.
The truck left the Sunland Park Racetrack and Casino parking lot and proceeded northbound on
Sunland Park Drive. The truck next travelled northwest on Doniphan Road, west on Aircraft
Road and south on Peter V. Domenici Boulevard. During one of the U.S. Route runs, two read
locations were placed on Peter V. Domenici Boulevard to allow for tag reads at speeds of 55
mph and 40 mph using external readers. The truck then returned to the NMBA facility.
The final read location of the RT was at the entrance to the NMBA facility. The truck passed the
read location for a slow moving read. Following the tag read at the entrance to the NMBA
facility, tag reads were made at distances of 5, 15, 30, 50, and 70 feet from the truck and trailer
as they had been prior to the RT.
In addition to the external reader data collected during the U.S. RTs, the PAVS uplinked
communication system was also verified. The difference between this system and the external
readers is that instead of a fixed location that defines the read location (at which the RFID tags
9
GaM1iteDSCS[)DQS(3X3eaJS(B®0©CuCG^CEI3Si
^ Caa'iftm3te[)E10ex33(IteCQ®©[iI^
' sia ~-
IT,w:r.£ll
3 miles
M —
Figure 3-7. U.S. route used during testing
-------
are read to confirm their presence), the RFID tags are read and their presence and location (as
latitude/longitude coordinates) recorded to a central database approximately every minute,
regardless of location. Therefore, additional data were collected in addition to the fixed read
points defined by the location of the readers. However, because of a malfunction with that
system, these data were not collected during the first U.S. RT. and approximately half of the data
were missing from the second U.S. RT.
Mexico Trucking Route. The MX route was slightly different from what was planned in the
TQAP'!), but was selected to mimic, as closely as possible, the route in the TQAP and to
challenge the PAVS under similar test conditions specified in the TQAP, which included
crossing the border to evaluate an considerations regarding the technology in an actual border
crossing.
The 11 A/M AT waste containers
were loaded into the trailer and
placed in the loosely-packed
configuration for both of the MX
RTs. The external reader
locations used during the MX
route are identified in Figure 3-8
and Figure 3-9. These were only
used during RT3 as AVANTE
opted to not use readers and only
the uplinked technology during
RT4. As for each U.S. and MX
RT, the truck began its route at the
NMBA facility. Prior to
embarking on the first MX RT, tag
reads were made at distances of 5,
15, 30, 50, and 70 feet from the
truck. Following the stationary
RFID tag reads at various
distances, the reader was moved to
the exit of the NMBA and the truck
a slow moving read.
The truck then proceeded south through the Jeronimo, MX POE and into MX. A read location
was located at the south end of the Jeronimo POE and the truck passed the read location for a
slow moving read. After passing this external reader, the truck proceeded for approximately 12
miles southbound on the Samalayuca-El Oasis Flighway (Carratera Samalayuca-El Oasis).
The third read location (referred to as MX Turnaround) on the MX Route was located 0.6 miles
north of the intersection of the Samalayuca-El Oasis Highway and MX Highway 2. After
passing this read location, the truck performed a U-turn and proceeded northbound on the
Samalayuca-El Oasis Highway, passing back into the U.S. through the Santa Teresa POE and
through U.S. Customs.
After passing through U.S. Customs, the truck then proceeded to the NMDPS facility and
encountered the same read locations that had been included for the U.S RTs. Two read locations
GSKl IShbIS taefpefflsiB (JOS me Si
caaiitoasCs ttaaexasfflaa cu^3C«ig)qo^S)
embarked on RT3 by first driving past the external reader for
Figure 3-8. Mexico route used during testing
(border area enlargement)
10
-------
KEEft
7 I^l!ni0iHaa3
were located at the facility. In all, the truck passed a 25 mph read point and a 15 mph read point
and then doubled back to pass them again, with the collision tags added before the second pass
by the 15 mph read point. As was the case during the U.S. RTs, the speed at which the truck was
travelling past the 25 mph and 15 mph read location was measured using a radar gun. The truck
then proceeded back to the NMBA
read location. The RT was
completed after passing an external
reader upon re-entry to the NMBA
and then a final stationary read at
distances between 5 and 70 feet from
the front bumper of the truck. Each
of the four RTs are summarized in
Table 3-1.
KM litoBtS fcEcxsofflaa E$«>(?es£®
Figure 3-9. Mexico route used during testing
Table 3-1. Summary of Round Trips
Truck Route
Read Locations of Round Trips
Information about RT
RT 1 U.S.
Began at NMBA, NMDPS Facility (25 and 15
mph. collision test), Casino, completed at
NMBA
Tightly packed configuration uplinked data
not collected because of system malfunction,
stationary tag reads at various distances
performed at NMBA before and after RT
RT 2 U.S.
Began at NMBA, NMDPS, Facility (25 and 15
mph, collision test). Casino, 40 mph and 55
mph highway reads, completed at NMBA
Tightly packed configuration, some uplinked
data collected, stationary tag reads at various
distances performed at NMBA before and
after RT
RT3 MX
Began at NMBA, MX Port of Entry, MX
turnaround, NMDPS Facility (25 and 15 mph.
collision test), completed at NMBA
Loosely packed configuration, uplinked data
collected, stationary tag reads at various
distances performed at NMBA before and
after RT
RT 4 MX
Began at NMBA, MX Port of Entry, MX
turnaround, NMDPS Facility (25 and 15 mph
collision test), completed at NMBA
Loosely packed configuration, external
reader data not collected because AVANTE
opted not to have them used, uplinked data
collected, stationary tag reads at various
distances performed at NMBA before and
after RT
3.2.6 Route Deviation
The AVANTE uplinked system had an optional feature to provide indication when the RF ID tags
had traveled outside a pre-programmed route. To test this feature, during RT 4, the truck was
diverted by approximately one mile and then directed to return to the planned route. The test
evaluated whether or not the proper alert for such a route deviation was made.
11
-------
3.3 Qualitative Evaluation Parameters
Operational factors such as ease of use, technology cost, user-friendliness of vendor software,
and troubleshooting/downtime, etc. documented based on observations by Battelle, Border
Writing, and U.S. EPA staff.
12
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed in accordance with the TQAP for this verification test(1) and
the QMP for the AMS Center(2). As noted throughout Chapter 3, there were some deviations
from the TQAP, but the work was performed as described in the previous sections. None of the
deviations from the test/QA plan resulted in any adverse impacts on the quality of the data
produced by this verification test. All deviations were reviewed with the EPA ETV AMS Center
Project Officer and EPA ETV AMS Center Quality Manager. QA/QC procedures and results are
described in the following subchapters.
4.1 Audits
Two types of audits were performed during the verification test: a technical systems audit (TSA)
of the verification test procedures, and a data quality audit. Because of the nature of RFID
measurements, a performance evaluation audit, as is usually performed to confirm the accuracy
of the reference method, was not applicable for this verification test. Audit procedures for the
TSA and the data quality audit are described further below.
4.1.1 Technical Systems Audit
The Battelle AMS Center Quality Manager performed a TSA during the test to ensure that the
verification test was performed in accordance with the TQAP for this verification testn' and the
QMP for the AMS Center(2). The TSA noted no adverse findings. A TSA report was prepared,
and a copy was distributed to the EPA AMS Center Quality Manager. In addition, the EPA
AMS Center Quality Manager was present during the majority of the verification test and also
performed a separate TSA.
4.1.2 Data Quality Audit
At least 10% of the data acquired during the verification test were audited. The data were traced
from the initial acquisition, through reduction and statistical analysis, to final reporting to ensure
the integrity of the reported results. All calculations performed on the data undergoing the audit
were checked.
4.2 QA/QC Reporting
Each audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the
AMS Center.(2) Once the audit reports were prepared, the Battelle Verification Test Coordinator
ensured that a response was provided for each adverse finding or potential problem and
13
-------
implemented any necessary follow-up corrective action. The Battelle Quality Manager ensured
that follow-up corrective action was taken. The results of the TSA were submitted to the EPA.
4.3 Data Review
Records generated in the verification test received a one-over-one review before these records
were used to calculate, evaluate, or report verification results. Data were reviewed by a Battelle
technical staff member involved in the verification test. The person performing the review added
his/her initials and the date to a hard copy of the record being reviewed.
14
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the quantitative performance factors listed in Section 3.1
are presented in this chapter. Qualitative observations were also used to evaluate verification test
data.
5.1 Accuracy
A primary objective for this verification test was to determine the accuracy of PAVS
performance with reading tags under critical variables and test conditions. Accuracy is a
measure of the agreement between a measured value and the "true" value. For this verification,
accuracy was determined as a percentage according to the following formula:
where A is the percent accuracy of the RFID system reader, E is the total number of tags that
were not properly recognized by the reader, and N is the total number of tagged HAZMAT
containers. The accuracy of the system was determined for each read point and packaging type.
The highest percent accuracy possible is 100%.
5.2 Precision
Precision is a measure of agreement among repeated measurements. The precision of tag reads
completed the PAVS was determined by calculating the standard deviation of the accuracy at all
possible read locations (i.e., from the accuracy at each external read location when external
readers were used or the accuracy at each uplinked tag read event when the uplinked data system
was used). The standard deviation of the accuracy measurements was calculated using the
following formula:
where, Snj is the standard deviation of all accuracy measurements in verification test run n is
the total number of possible read events in verification test run A/:J is the percent accuracy of
the RFID system reader for read event k during verification test run and ~~K, is the overall
arithmetic mean percent accuracy of the RFID system during verification test run i.
A = (1 -E/N) x 100
(1)
k=l
15
-------
5.3 Influence of Possible Confounding Factors
The influence of the different container types on the accuracy of container identification was
evaluated by calculating the accuracy in each container during each RT. Then a paired t-test was
performed to determine if significant differences existed between the accuracy of the
identification of steel drums, poly drums, and cardboard boxes. Other possible confounding
factors included meteorological and environmental conditions. These data were reviewed
qualitatively in attempt to identify possible correlations where statistical approaches should be
considered.
16
-------
Chapter 6
Test Results
As mentioned previously, this verification test included both quantitative and qualitative
evaluations. The quantitative evaluation was conducted to assess the accuracy and precision of
PAVS, as well as by testing the influence of confounding factors and its ability to discriminate
the HAZMAT waste tags from other commercially-available active tags. The qualitative
evaluation was performed to document the operational aspects of PAVS when it was used during
verification testing. The following sections provide the results of the quantitative and qualitative
evaluations.
6.1 Accuracy
As described in Section 3.1, two configurations of the Mobile PAVS technologies were tested.
One configuration included roadside readers that read the RFID tags in the truck at fixed
locations and a second configuration that include a reader (referred to as the Relayer) that resided
in the cab of the truck and communicated (uplinked) the container identifications each minute to
a central database via a cellular phone connection. Both configurations of the PAVS were tested
simultaneously and the results are presented separately below.
6.1.1 External Reader Accuracy
Table 6-1 gives the accuracy results for the external reader configuration of the PAVS
technology. The left column of the table gives each read location as described throughout
Section 3.2 and the top row of the table gives the RT number, what route was followed, and
whether the load was packed in a tight or loose configuration. Because the truck contained 12
HAZMAT containers with vendor RFID tags, each read location had the possibility of 12 correct
tag identifications. Each tag was identified with a unique eight digit number (e.g., 40000020)
that was labeled on the outside of the tag. This number was also the piece of information that
was recorded by the RFID reader upon a successful read event. Therefore, the accuracy of each
read location is presented as a percentage of correctly read tags based on a total of 12 possible
correct tag reads at each read location. AVANTE opted not to include the external readers in the
second RT into MX (RT4). As was described in Section 3.2, during the stationary reads at
various distances prior to RT 1 and the initial exit from NMBA, the external readers were placed
at angles of 90° and 45° with respect to the road. At that time, AVANTE indicated a likely
interference between the two readers so use of readers at both angles was discontinued. Upon
review of the data, it did not appear that there was any interference between the two readers so
the available results for both reader angles are shown. Some of the read locations given in the
table were not used during every RT. The casino read location was only used during the U.S.
RTs, the MX POE and MX turnaround read location were only used during the MX RTs,
17
-------
Table 6-1. Accuracy - External Readers
Round Trip (RT) Number
RT 1
RT 2
RT 3
Truck Route
US
US
MX
Packaging Configuration:
Tight
Tight
Loose
Read Location
New Mexico Border Authority
(NMBA) Distance/Angle Reads:
5ft, 90°
100% (12/12)
83% (10/12)
83% (10/12)
5ft, 45°
100% (12/12)
(1)
(1)
15ft, 90°
100% (12/12)
83% (10/12)
92% (11/12)
15ft, 45°
100% (12/12)
(1)
(1)
30ft, 90°
92% (11/12)
83% (10/12)
83% (10/12)
30ft, 45°
92% (11/12)
(1)
(1)
50ft, 90°
92% (11/12)
83% (10/12)
83% (10/12)
50ft, 45°
92% (11/12)
(1)
(1)
70ft, 90°
92% (11/12)
83% (10/12)
92% (11/12)
70ft, 45°
92% (11/12)
(1)
(1)
NMBA Exit, 90°
100% (12/12)
100% (12/12)
100% (12/12)
NMBA Exit, 45°
100% (12/12)
(1)
(1)
Casino
100% (12/12)
100% (12/12)
(2)
MX Port of Entry
(2)
(2)
100% (12/12)
MX Turnaround
(2)
(2)
100% (12/12)
55 mph pass
(3)
100% (12/12)
(2)
40 mph pass
(3)
100% (12/12)
(2)
NMDPS Truck Inspection Facility
25 mph pass
92% (11/12)
75% (9/12)
17% (2/12)
15 mph pass
92% (11/12)
100% (12/12)
(4)
25 mph pass
100% (12/12)
100% (12/12)
33% (4/12)
15 mph pass (collision test)
100% (12/12)
75% (9/12)
(4)
NMBA Entrance
(3)
100% (12/12)
100% (12/12)
5ft, 90°
100% (12/12)
100% (12/12)
100% (12/12)
15ft, 90°
83% (10/12)
100% (12/12)
100% (12/12)
30ft, 90°
83% (10/12)
100% (12/12)
100% (12/12)
50ft, 90°
83% (10/12)
100% (12/12)
100% (12/12)
70ft, 90°
83% (10/12)
92% (11/12)
100% (12/12)
Overall External Reader Average
94% (248/264)
93% (211/228)
86% (166/192)
(1) 45° Read angle not used following RT 1
(2) Read location not applicable for this RT
(3) Read location not used because readers not in place during RT 1
(4) Data not collected for undetermined reason; reader was in place and seemingly recording, but no data stored.
and the 55 mph and 45 mph read locations were only used during the second U.S. RT. The
stationary reads at various distances were performed before and after each RT at the NMBA and
there were two read locations (25 and 15 mph) at the NMDPS facility. The truck passed the
NMDPS facility twice during each RT.
For all of the RTs, when a container was identified correctly, the identification number of its
RFID tag was read and documented by the external reader. If it was not identified, the tag
identification number was not read and documented. During RT 1, there were 22 possibilities
for the PAVS external reader configuration to make 12 correct reads of the tagged containers.
All 12 containers were identified in 10 instances, 11 out of 12 containers were identified in eight
instances, and 10 out of 12 containers were identified in four instances. Over the entire RT, 248
out of 264 containers were identified correctly for an overall accuracy of 94%.
During RT 2, there were 19 read locations to make 12 correct reads of the tagged containers. All
12 containers were identified in 11 instances, 11 out of 12 containers were identified in one
18
-------
instance, 10 out of 12 containers were identified in five instances, and 9 out of 12 containers
were identified in two instances. Over the entire RT, 211 out of 228 containers were identified
correctly for an overall accuracy of 93%. Prior to the RT, the stationary reads at various
distances from the truck produced 10 out of 12 correct identifications at each distance, but
following the RT, the same test produced 12 out of 12 correct identifications at all but the
furthest distance. Also, this was the only RT that included a 40 mph and 55 mph read location
and the results for both were 100% accuracy. Two 75% accuracy reads occurred at the NMDPS
facility, one at 25 mph and one at 15 mph. There was no discernable reason for the diminished
accuracy at these locations given that the read locations immediately before and after these read
locations produced 100% accurate results.
During RT 3, there were 16 possibilities to make 12 correct reads of the tagged containers. All
12 containers were identified in 9 instances, 11 out of 12 containers were identified in two
instances, and 10 out of 12 containers were identified in three instances, and two instances that
generated results that were less than 50%. There were two completely unique read locations
during this RT, the Mexican POE and the MX turnaround point. The MX POE read location was
located in a place that truck traffic waited in line to enter MX therefore, the external reader was
passed at a low rate of speed (~5 mph). The MX turnaround read location was approximately 12
miles south of the MX POE in a location just north of where the truck turned around to return to
the U.S. The truck was traveling at 30 mph when it passed the external reader. Both of these
read locations produced 100% accurate identifications. Upon the return to the U.S., the truck
passed the 25 mph and 15 mph read locations at the NMDPS facility. For a reason that was not
able to be determined, no data were collected at the 15 mph read location during this RT. The
results for the two 25 mph read locations during this RT were 17% and 33%, respectively, with
no discernible reason for the relatively low levels of accuracy given that the rest of the RT had
accuracies between 80% and 100%. Over the entire RT, 166 out of 192 containers were
identified correctly for an overall accuracy of 86%.
6.1.2 Relayer Uplinked Accuracy
This configuration of the Mobile PAVS technology included an RFID tag reader (referred to as
the Relayer) that was located in the cab of the truck. The RELAYER also served as the
communication link by uplinking the collected data via cellular to an AVANTE computer server
once per minute during each RT. Because the tags were being read every minute, regardless of
location, it eliminated the need to pass by an external reader at a single location. However,
because the data for both configurations were being collected simultaneously, the results are
presented in the same fashion, recording the accuracy of the container identification at the times
that the truck was at the various read locations. Performing reads at various distances from the
truck, an aspect of testing that included only the external readers, was not conducted for this
configuration of the Mobile PAVS technology.
Table 6-2 presents the results for the uplinked data in a similar fashion as in the previous section
for the external reader data. The uplinked data were collected for only RT 2, RT 3, and RT 4 as
data were not collected for RT 1. According to AVANTE, this gap in data was due to a gap in
cell phone service coverage preventing the transmittal and storage of the tag identification and
GPS tracking data. In addition to the missing data from RT1, uplinked data were only collected
for the first 40 minutes of RT 2. At that point in the RT, there was a gap in cell phone coverage
and the PAVS was unable to restart the communication when the truck travelled back in to an
area with cell phone service. Data collected for RT2 included the NMBA exit, three of the four
read locations at the NMDPS facility (data were not collected for the second 25 mph pass) and
19
-------
part of the way to the casino. The RT lasted from approximately 4:00 p.m. until 5:25 p.m., but
data were collected only until 4:45 p.m. Throughout this time period, there were 233 correct
identifications of containers out of 348 possibilities for an overall accuracy (while the technology
was functional) of 67%.
Table 6-3 gives the data completeness for RT 2-4. Data completeness is defined as the fraction
of the total duration of the RT that the continuous data uplink was occurring (less than two
minutes between data uplinks). RT 2 had a data completeness of 37%. Most of the missing data
occurred during the final 40 minutes of the RT, but there was also 11 minutes of missing data
surrounding the second 25 mph read location.
The accuracy of RT 3 ranged from 75% to 100% at the various read locations, but during the
time period that data were being collected properly, the overall accuracy was 1,594 correct
container identifications out of 1,786 possible identifications for an overall accuracy of 89%. In
addition, the data completeness for this RT was much higher than for RT2 as data were uplinked
at least once every two minutes for 86% of the RT. According to AVANTE, the missing data
were again due to a lack of cell phone coverage, but prior to RT 3, they were able to make a
change to the PAVS firmware to accommodate the restart of data transmission when the truck
returned into an area with adequate cell phone coverage. This restarting of data transmission was
observed following a time period of missing data during RT 3.
The accuracy at the various read locations during RT 4 was 100% with the exception of the
second 25 mph pass which did not identify two of the containers within the truck for 83%
accuracy at that location. However, during the time period that data were being collected
properly, the overall accuracy was 804 correct container identifications out of 850 possible
identifications for an overall accuracy of 95%. In addition, the data completeness for this RT
was 81%) as there were 17 minutes of missing data when the truck was in MX. The time period
of missing data included the MX turnaround so there was no data collected for that read location.
The reason for the missing data was again a lapse in cell phone service coverage which was
apparently regained by the end of RT 4 as data collection resumed.
Table 6-2. Accuracy - In-truck Reader and Uplinked Communication
Round Trip Number
RT 2
RT 3
RT 4
Truck Route
US
MX
MX
Packaging Configuration
Tight
Loose
Loose
Read Locations
NMBA exit
75% (9/12)
100% (12/12)
100% (12/12)
MX POE
(1)
100% (12/12)
100% (12/12)
NM Casino
(2)
(1)
(1)
MX Turnaround
(1)
83% (30/36)
(2)
NMDPS Truck Inspection
Facility
25 mph pass
58% (7/12)
83% (10/12)
100% (12/12)
15 mph pass
83% (10/12)
83% (10/12)
100% (12/12)
25 mph pass
(2)
92% (11/12)
83% (10/12)
15 mph pass (collision test)
100% (12/12)
92% (11/12)
100% (12/12)
55 mph pass
(2)
(1)
(1)
40 mph pass
(2)
(1)
(1)
NMBA entrance
(2)
75% (9/12)
100% (12/12)
Overall Road Driving
67% (233/348)
89% (1594/1786)
95% (804/850)J
(1) Read location not applicable for this RT
(2) Data not collected for undetermined reason, data collection halted completely after 40 minutes of the RT
(3) 25% of the misidentifications may have been due to a wet RFID tag (see Section 6.4)
20
-------
Table 6-3. Data Completeness Due to Continuity of Data Uplink
Round Trip Number
RT 2
RT 3
RT 4
Route
US
MX
MX
Elapsed Time (min) of route
81
162
91
Duration of continuous data
30
140
74
uplinks (min)
Data completeness
37%
86%
81%
6.2 Precision
The precision, or repeatability, of the RFID accuracy data were determined as described in
Section 5.2 and is given in Table 6-4 along with the overall accuracy for each RT.
Table 6-4. Overall Accuracy + Standard Deviation of Each RT
Truck Routes
External Reader + SD
Uplinked Data + SD
RT1-US
94% + 6%
(1)
RT2-US
93%+ 10%
67% + 32%
RT3-MX
86% + 25%
89%+ 15%
RT4-MX
(1)
95% + 7%
(1) Read location not applicable for this RT
For the external readers, the standard deviations were 10% or less for RT 1 and 2. The uplinked
data became more precise over the three RTs, with RT 2 generating a SD of 32% and RT 4
generating a SD of 7%.
6.3 Interference of Other RFID Signals
The collision test was performed as described in Section 3.2.4 by placing four RFID tags into the
trailer along with the containers that were tagged with the AVANTE RFID tags. Thereafter, the
truck passed the second 15 mph read location at the NMDPS facility. The data for that read
location are shown in Table 6-5. Overall, three out of the five available data sets resulted in
100%) accuracy, one had 92% accuracy, and one had 75% accuracy. While difficult to determine
conclusively with this small data set, these results were similar to the non-collision test results
presented in Tables 6-1 and 6-2 in that some non-collision test read locations resulted in
accuracy percentages of 75%, 92%, and 100%. These similarities, while not a quantitative
evaluation, suggest that the presence of the collision tags did not negatively impact the accuracy
results.
Table 6-5. Collision Test Results
15 mph pass -
Collision Test
RT 1 US
RT2 US
RT3 MX
RT 4 MX
External Readers
100% (12/12)
75% (9/12)
(1)
(2)
Uplinked Data
(1)
100% (12/12)
92% (11/12)
100% (12/12)
(1) Data not collected for undetermined reason; reader was in place and seemingly recording, but no data stored.
(2) Read location not applicable for this RT
21
-------
6.4 Influence of Confounding Factors
Container type and packaging configuration. The RFID tags were placed on three different
types of containers during the round trips. The accuracy results are presented in Table 6-6 by
container type. The container identification accuracy ranged from 59% to 100% with an average
of 85%) for the steel drums, 72% to 99% with an average of 89% for the poly drums, and 69% to
100%) with an average of 88% for the cardboard boxes. A paired t-test was performed on the
results by container type and no significant differences were determined at a 95% confidence
interval. As shown in Table 6-2 above, the overall accuracy of the uplinked data during RT 2
was considerably lower than RTs 3 and 4. This is consistent with the container-specific accuracy
Table 6-6. Accuracy Results by Container Type (in percent)
Configuration
Round Trip #
Steel
Drum
Poly
Drum
Cardboard
Box
RT1
100
93
89
External
RT2
96
91
91
Readers
RT3
83
86
91
RT4
(1)
(1)
(1)
RT1
(1)
(1)
(1)
Uplinked
RT2
59
72
69
RT3
85
94
89
RT4
85
99
100
Average
85
89
88
SD
14
9
10
(1) Read location not applicable for this RT
data shown here as the accuracies for RT 2 are lower for all three RTs. RT 2 was the only RT
for which uplinked data were collected when the containers were in a tightly-packed
configuration so the lower container identification accuracies could be attributed to packing
configuration. However, the uplinking data system stopped functioning 45 minutes into RT 2,
and upon review of the raw data, there was an increased number of missed container
identifications just before it stopped functioning, suggesting that the problem with the equipment
was more likely the contributing cause rather than the packing configuration. In addition, the
external readers did not exhibit the same diminished accuracy with the tight configuration.
Meteorological and other environmental conditions. Throughout the two days of testing, the
temperature inside the truck never exceeded 25.1 degrees Celsius (°C) and never dropped below
15.8 °C. The relative humidity ranged from 12% to 19%, and the barometric pressure ranged
from 25.4 inches of mercury (in Hg) to 26.1 in Hg. Upon a qualitative review of the accuracy
data, there was no suggestion that meteorological conditions impacted the results in any way so
no statistical analyses were performed. However, high winds caused sandstorm conditions
during the testing which caused difficulty in reading the memory cards used for data collection
on the external readers. Instead of using the memory card, the data were downloaded using an
ethernet port. Similar to the meteorological conditions, there was no suggestion of a correlation
between the Shocklog data collected during each RT and the accuracy results so no statistical
analyses were performed.
Other factors. Following RT 4, the verification staff noticed that one of the tags was very wet
(from water that had splashed out of the drum it was attached to) after the truck returned from the
22
-------
RT. Review of the tag specific data revealed that of the 46 incorrect container identifications
(out of 804 possible identifications) during RT 4, 25 of them had been due to the tag that was
found to be wet. Throughout RT 4, that tag had a 65% accuracy. One other tag (with no similar
explanation) had 79% accuracy while the others were between 96% and 100%.
6.5 Operational Factors
The verification staff found that AVANTE's PanaSec Mobile PAVS was easy to use. AVANTE
staff set up the external readers and assisted with the application of the RFID tags to the
HAZMAT waste containers. The Mobile PAVS was setup quickly by powering the system
through the dashboard 12v power outlet and positioning two small antennae inside the truck's
cab.
As described in Section 6.1, the PAVS technology had a decreased level of data completeness
due to blocks of missing data during the verification test. According to AVANTE, the reason for
these missing blocks were due to lack of cellular phone coverage. During RTs land 2 the loss of
cell phone service coverage took place in the U.S., during RT 3 the loss of cell phone service
coverage took place on both sides of the border near the MX POE on the way back in to the U.S.,
and during RT 4 the loss of coverage took place in MX on the way to the turnaround point.
During RTs 1 and 2, the transmission of tag identification data did not restart when the truck re-
entered an area with adequate cell phone service coverage. Prior to RT 3, AVANTE was able to
make a change to the PAVS firmware that allowed for this to take place during RTs 3 and 4.
This update did not accommodate simultaneous operation with the external reader configuration
so the external readers were not used for RT 4. In addition, following RT 1, AVANTE staff had
to repair a faulty GPS receiver within the Relayer.
AVANTE concluded that the RF conditions at the test location in close proximity to the United
States - Mexico Border were particularly adverse to the efficiency of communication between
the RFID tags and the vehicle-based Relayer component of our equipment. This adverse local RF
environment decreased the nominal read distance between Relayer and RFID tags from 500 ft to
less than 100 ft. AVANTE believe this RF interference caused diminished tag read accuracy
during the verification test. AVANTE has taken steps to resolve these problems such as adding
an amplifier to the Relayer to increase read distances, changing the orientation and type of the
external antenna, shifting the reception frequency of the antenna, and revising the software to
allow PAVS to store the tag identification and GPS tracking data and then transmit it when the
truck returns to an area with adequate cell phone coverage.
The software that AVANTE uses to handle the data collected by the uplinked data connection is
web-based and offers a number of optional utilities that were used during the verification test.
One feature that was tested was the ability for the software to provide an alert when the truck
deviated from its planned path by more than one mile. This was performed once during the
verification test and the alert was provided in real-time to verification staff that were watching
the computer monitor as well as by text message. In addition to the RFID tag identification data,
the uplinked data included the GPS coordinates of the Relayer at the time of uplink. Using
AVANTE's online software, the coordinates can be mapped in real-time and displayed as is
shown in Figure 6-1. The path of green shapes show the route of the truck from the NMBA into
MX. In addition to the GPS mapping, another utility was the use of a wallet sized "Driver's
Identification card" incorporating a "panic-button" that can be given to the driver of a truck.
This can be used as an actual panic button for the driver to alert those monitoring the truck, or
23
-------
those designated in advance to receive alert text messages on their mobile phones or via email, in
the event of a dangerous or threatening situation with a discreet one-second squeeze at a certain
point on the badge. This card can also be used to send an alert if the driver leaves the truck and
then send another report when the driver returns. While not a part of the formal testing,
AVANTE successfully demonstrated each of these utilities during the verification test. The user-
friendliness of the vendor software was very apparent as the interface was graphical and
intuitive. The time needed to train testing staff in the proper use of the system was minimal.
<<<>>> Acknowledged
1101 to 200 -to 3/26/2009 5:36:69
©
Google \
Figure 6-1. Screenshot from AVANTE's Software.
The uplinked configuration of the AVANTE PAVS can be purchased as a service at a current
price of $l/day/vehicle which includes the Relayer, one Driver's identification badge with Panic
Button, and two RFID tags for cargo or wall mount. The price is based on a two year contract,
with a $200 security deposit refundable at lease termination. The price includes the utilities
described above (i.e., route deviation, driver location alerts, panic button functionality, text
message alerts, etc.). Additional RFID tags are available for purchase at $25 - $32 depending on
quantity.
24
-------
Chapter 7
Performance Summary
Three RTs were performed using the external reader and uplinked configurations of the PAVS
system. The external readers were used during the first two RTs that were performed within the
U.S. and the first RT into MX. Prior to RT 3, AVANTE performed a firmware update to their
software to ensure the uplinked system worked properly. That update did not accommodate
simultaneous operation with the external reader configuration. The uplinked system was used
for all four RTs but had higher levels of data completeness during RTs 3 and 4. No uplinked
data were collected during RT 1 and data from 63% of RT 2 were not collected due to gaps in
cell phone coverage required for data transmission. Table 7-1 summarizes the accuracy,
precision, and data completeness of the PAVS system.
Table 7-1. Accuracy and Precision Summary for the PAVS
External Reader
Uplinked Data
Truck Route
(Identified Tags/Total Tags)+SD
(Identified Tags/Total Tags)+SD
RT1-US
94% (248/264) + 6%
(1)
RT2-US
93% (211/228) + 10%
67% (233/348) + 32%
RT3-MX
86% (166/192) + 25%
89% (1,594/1,786)+ 15%
RT4-MX
(1)
95% (804-850) + 7%
(1) Read location not applicable for this RT
The external readers generated container identification accuracies that were above 90% and had
SDs of 10%) or less for RTs 1 and 2. RT 3 resulted in a lower overall accuracy and a higher SD
with two read locations as part of RT 3 that resulted in less than 35% accurate results, thus
impacting the overall accuracy for that RT. All of the other read locations for that RT had
greater than 83% accuracy.
The uplinked configuration for RTs 2, 3, and 4 generated accuracies of 67%, 89%, and 95%,
respectively. Data were only collected for the first 40 minutes of RT 2 for data completeness of
37%) and the frequency of missed reads increased throughout that time period. The results from
RTs 3 and 4 show that data were collected during more than 80% of the time. According to
AVANTE, these gaps in data were due to gaps in cell phone service coverage preventing the
transmittal and storage of the tag identification and GPS tracking data.
Interference with other RFID signals. The collision test was performed as described in
Section 3.2.4 by placing four RFID tags into the trailer along with the containers that were
tagged with the AVANTE RFID tags. Three out of the five collision test data sets resulted in
100%) accuracy, one had 92% accuracy, and one had 75% accuracy. While difficult to determine
conclusively with this small data set, these results were similar to the non-collision test results
presented in Tables 6-1 and 6-2 in that some non-collision test read locations resulted in
accuracy percentages of 75%, 92%, and 100%. These similarities, while not a quantitative
25
-------
evaluation, suggest that the presence of the collision tags did not negatively impact the accuracy
results.
Influence of Confounding Factors. The influence of container type was quantitatively
considered as a possible factor in PAVS performance. A paired t-test was performed on the
container identification accuracies as a function of container type and no significant differences
were determined between the container types. In addition, there was no indication that
environmental conditions during the testing significantly impacted any of the results. However,
one tag was wetted by water splashed from a container during a RT. This tag was missed at a
higher rate compared to other tags. High winds caused sandstorm conditions during the testing
which caused difficulty in reading the memory cards used for data collection on the external
readers. Instead of using the memory card, the data were downloaded using an ethernet port.
Operational Factors. AVANTE staff set up the external readers and assisted with the
application of the RFID tags to the HAZMAT waste containers. The ease of use of the Mobile
PAVS was notable as setup could be quickly accomplished by powering the system through the
dashboard 12 volt power outlet and positioning of two small antenna inside the truck's cab.
As described in Section 6.1, the PAVS technology had a decreased level of data completeness
for the uplinking system due to blocks of missing data during the verification test. The reason
for these missing blocks were not fully known for RTs 1 and 2, and for RTs 3 and 4, the gaps in
data collection were due to lack of cellular phone coverage. During RTs land 2 the loss of cell
phone service coverage took place in the U.S., during RT 3 the loss of cell phone service
coverage took place on both sides of the border near the MX POE on the way back in to the U.S.,
and during RT 4 the loss of coverage took place in MX on the way to the turnaround point.
During RTs 1 and 2, the transmission of tag identification data did not restart when the truck re-
entered and area with adequate cell phone service coverage. Prior to RT 3, AVANTE was able
to make a change to the PAVS firmware that allowed for this to take place during RTs 3 and 4.
This update did not accommodate simultaneous operation with the external reader configuration
so the external readers were not used for RT 4. In addition, following RT 1, AVANTE staff had
to repair a faulty GPS receiver within the Relayer.
AVANTE concluded that the RF conditions at the test location in close proximity to the United
States - Mexico Border were particularly adverse to the efficiency of communication between
the RFID tags and the vehicle-based Relayer component of our equipment. This adverse local RF
environment decreased the nominal read distance between Relayer and RFID tags from 500 ft to
less than 100 ft. AVANTE believes this RF interference caused diminished tag read accuracy
during the verification test. AVANTE has taken steps to resolve these problems such as adding
an amplifier to the Relayer to increase read distances, changing the orientation and type of the
external antenna, shifting the reception frequency of the antenna, and revising the software to
allow PAVS to store the tag identification and GPS tracking data and than transmit it when the
truck returns to an area with adequate cell phone coverage.
26
-------
The uplinked configuration of the AVANT PAVS can be purchased as a service at a current
price of $l/day/vehicle which includes the Relayer, one Driver's identification badge with Panic
Button, and two RFID tags for cargo or wall mount. The price is based on a two year contract,
with a $200 security deposit refundable at lease termination. The price includes the utilities
described above (i.e., route deviation, driver location alerts, panic button functionality, text
message alerts, etc.). Additional RFID tags are available for purchase at $25 - $32 depending on
quantity.
27
-------
Chapter 8
References
1. Test/QA Plan for Verification of Radio Frequency Identification (RFID) for Tracking
Hazardous Waste Shipments across International Borders, Battelle, Columbus, Ohio, March
23, 2009.
2. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version 7.0,
U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio,
November, 2008
28
------- |