EPA/600/R-12/573
June 2012
Environmental Technology
Verification Report
PlCOMETRIX, LLC
T-RAY 4000® TIME-DOMAIN
TERAHERTZ SYSTEM
Prepared by
Battelle
Baireiie
Ine Business of Innovation
Under a cooperative agreement with
U.S. Environmental Protection Agency
-------
EPA/600/R-12/573
June 2012
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
PlCOMETRIX, LLC
T-RAY 4000® TIME-DOMAIN TERAHERTZ SYSTEM
by
Stephanie Buehler, Amy Dindal, and Rosanna Buhl, Battelle
John McKernan and Madeleine Nawar, U.S. EPA
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency, through its Office of Research and Development,
funded and managed, or partially funded and collaborated in, the research described herein. It
has been subjected to the Agency's peer and administrative review. Any opinions expressed in
this report are those of the author(s) and do not necessarily reflect the views of the Agency,
therefore, no official endorsement should be inferred. Any mention of trade names or
commercial products does not constitute endorsement or recommendation for use.
11
-------
Foreword
The EPA is charged by Congress with protecting the nation's air, water, and land resources.
Under a mandate of national environmental laws, the Agency strives to formulate and implement
actions leading to a compatible balance between human activities and the ability of natural
systems to support and nurture life. To meet this mandate, the EPA's Office of Research and
Development provides data and science support that can be used to solve environmental
problems and to build the scientific knowledge base needed to manage our ecological resources
wisely, to understand how pollutants affect our health, and to prevent or reduce environmental
risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http ://www. epa.gov/etv/centers/centerl .html.
in
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. We greatly appreciate the involvement
and support of Appleton, for allowing testing to be conducted in their plant. We thank, in
particular, the efforts of Mike Friese, Jason Morgan, and Dan Scholz of Appleton. We also thank
the EPA Office of Air and Radiation for their support of this verification test. Finally, we would
like to thank Paul Thomas, 3M; Mike Barlament, Kimberly-Clark; Temeka Taplin, Mele
Associates; and Madeline Nawar, U.S. EPA Office of Air and Radiation for their review of this
verification report. Quality assurance (QA) oversight was provided by Michelle Henderson and
Laurel Staley, U.S. EPA, and Rosanna Buhl, Battelle.
IV
-------
Contents
Notice ii
Foreword iii
Acknowledgments iv
List of Abbreviations ix
Chapter 1 Background 2
Chapter 2 Technology Description 3
Chapters Test Design and Procedures 7
3.1 Introduction 7
3.2 Test Design 7
3.3 Test Samples 14
3.5.1 Accuracy 16
3.5.2 Precision 16
3.5.3 Comparability 16
3.5.4 Operational Factors 16
Chapter 4 Quality Assurance/Quality Control 17
4.1 Quality Control 17
4.1.3 Data Quality Indicators 17
4.2 Equipment Testing, Inspection, and Maintenance 19
4.3 Audits 20
4.3.1 Technical Systems Audit 21
4.3.2 Data Quality Audit 22
Chapters Statistical Methods 24
5.1 Accuracy 24
5.2 Precision 24
5.3 Comparability 24
Chapter 6 Test Results 26
6.1 Accuracy 27
6.2 Precision 29
6.3 Comparability 31
6.4 Operational Factors 42
Chapter 7 Performance Summary 43
Chapters References 45
-------
Figures
Figure 2-1. Collinear THz Reflection Sensor (with Beam-Splitter) 5
Figure 2-2. Illustration of Photon Pulse's Origin and Detections 5
Figure 2-3. Picometrix, LLC, T-Ray 4000® Time-Domain THz System 6
Figure 3-1. Relative Position of Sample Paper Sheet, Nuclear (Beta) Gauge, and THz Sensor... 8
Figure 3-2. THz Sensor and Nuclear Gauge During Production Line Testing 9
Figure 3-3. Close-up of THz Sensor in the Production Line 9
Figure 3-4. THz sensor rotated 'off-sheet' for air space reading 11
Figure 3-5. Standard Basis Weight Determination using an Analytical Balance 13
Figure 3-6. THz sensor system in the Laboratory 13
Figure 3-7. Close-up of THz sensor system in the Laboratory 14
Figure 6-1. Laboratory sample 30
Figure 6-2. Zoom on multiple reflections 30
Figure 6-3. ACAl IB 16026 on-line results 32
Figure 6-4. AC Al IB 16027 on-line results 32
Figure 6-5. AC Al IB 16028 on-line results 32
Figure 6-6. ACA11B16035 on-line results 32
Figure 6-7. AC Al IB 1603 7 on-line results 32
Figure 6-8. ACA11B23035 on-line results 33
Figure 6-9. ACA11B23036 on-line results 33
Figure 6-10. ACA11B23042 on-line results 33
Figure 6-11. ACA11B23043 on-line results 33
Figure 6-12. ACA11B23044 on-line results 33
Figure 6-13. ACA11B16026 Percent Difference Trial Run 36
Figure 6-14. ACA11B16026 Percent Difference Histogram 36
vi
-------
Figure 6-15. ACA11B16027 Percent Differnece Trial Run Time 36
Figure 6-16. ACA11B16027 Percent Differnece Histogram 36
Figure 6-17. ACA11B16028 Percent Differnece Trial Run Time 36
Figure 6-18. ACAl IB 16028 Percent Differnece Histogram 36
Figure 6-19. ACA11B16035 Percent Difference Trial Run Time 37
Figure 6-20. ACA11B16035 Percent Difference Histogram 37
Figure 6-21. ACA11B16037 Percent Difference Trial Run Time 37
Figure 6-22. ACA11B16037 Percent Difference Histogram 37
Figure 6-23. ACA11B23035 Percent Difference Trial Run Time 38
Figure 6-24. ACA11B23035 Percent Difference Histogram 38
Figure 6-25. ACA11B23036 Percent Differnece Trial Run Time 38
Figure 6-26. ACA11B23036 Percent Differnece Histogram 38
Figure 6-27. ACA11B23042 Percent Differnece Trial Run Time 38
Figure 6-28. ACA11B23042 Percent Difference Histogram 38
Figure 6-29. ACA11B23043 Percent Difference Trial Run 39
Figure 6-30. ACA11B23043 Percent Difference Histogram 39
Figure 6-31. ACA11B23044 Percent Difference Trial Run 39
Figure 6-32. ACA11B23044 Percent Difference Histogram 39
Tables
Table 3-1. Environmental Conditions during Testing 15
Table 4-1. DQI and Criteria for Critical Supporting Measurements 18
Table 4-2. THz Mid-Day System Checks vs. Daily Initial Set-up Values 20
Table 4-3. Summary of THz Air Space Samples 20
Table 6-1. Accuracy of THz Basis Weight Values vs. Standard Laboratory Measurements 29
vii
-------
Table 6-2. Precision of Replicate THz Readings 29
Table 6-3. Comparison of Production Measurements Between THz and Nuclear Sensors 40
Table 6-4. Summary of THz Operational Factors 42
Table 7-1. Comparison of Online Production Mean Percent Difference Values for THz and
Nuclear Sensors Basis Weight 44
Vlll
-------
List of Abbreviations
ADQ Audit of Data Quality
AMS Advanced Monitoring Systems
DQI data quality indicator
EM electromagnetic
EPA U.S. Environmental Protection Agency
ERS External Reference Structure
ETV Environmental Technology Verification
gsm grams per square meter
LRB laboratory record book (the test logbook)
PE performance evaluation
ps picosecond = 10~12 second
QA quality assurance
QAPP quality assurance project plan
QC quality control
QMP quality management plan
RI Refractive Index
SOP Standard Operating Procedure
TAPPI Technical Association of the Pulp and Paper Industry
TD-THz Time-Domain Terahertz
THz terahertz
ToF Time-of-Flight
TSA Technical Systems Audit
|j,W microwatt = 10~6 watt
VTC Verification Test Coordinator
IX
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of the
ETV Program is to further environmental protection by accelerating the acceptance and use of
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-
quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized non-profit testing organizations (such as Battelle);
with stakeholder groups consisting of buyers, vendor organizations, and permitters; and with the
full participation of individual technology developers. The program evaluates the performance of
innovative technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing
peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality
assurance (QA) protocols to ensure that data of known and adequate quality are generated and
that the results are defensible.
The EPA's National Risk Management Research Laboratory (NRMRL) and its verification
organization partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under
ETV. The AMS Center recently evaluated the performance of the Picometrix, LLC, T-Ray
4000® Time-Domain Terahertz (TD-THz) System as an alternative to using sealed radioactive
source nuclear gauges in industrial applications such as paper manufacturing. This test was
conducted in conjunction with EPA's Alternative Technology Initiative, which aims to
encourage voluntary replacement of sealed nuclear devices with non-nuclear sources.
-------
Chapter 2
Technology Description
This report provides results for the verification testing of the Picometrix, LLC, T-Ray 4000® TD-
THz System. The following is a description of the system, based on information provided by the
vendor. The information provided below was not verified in this test.
TD-THz systems (shown in Figure 2-3) emit and detect a very narrow (<1 picosecond [ps])
electromagnetic (EM) pulse that forms photons in the THz frequency range. The THz frequency
range falls between microwaves (0.1 THz) and far infrared (IR) (10 THz). TD-THz systems
measure the electrical field strength of the EM photon pulse as a function of time. Most dielectric
materials are transparent in the region of study with TD-THz (0.05-3 THz). Plastics (regardless
of color), paper, textiles, dry wood, packaging materials, rubbers, foams, non-polar liquids (such
as oils), paints (including low observable "radar absorbing") and other coatings are all
transparent to THz wavelength photons. Polar liquids (such as water and alcohols) are strongly
absorbing over the THz frequency region. The EM photon pulse is also non-ionizing and thus
safer than sealed radioactive source techniques.
The THz pulse is low energy (less than 1 microwatt [|-iW]) and can be focused, reflected, and
treated essentially in the same manner as any pulsed photon (light) source. After this photon
pulse has interacted with matter (transmission, reflection, and scatter), the changes in the pulse
lead to two primary methods of analysis, spectroscopic and Time-of-Flight (ToF). Spectroscopic
methods of investigation are possible with THz. The transformation of the TD-THz data using a
Fourier function to better understand the time and frequency domains of the data allows time and
spectroscopic analysis. The second common method of analysis is to directly study the TD data
by measuring changes in the ToF of the photon pulse as it interacts with matter.
Analysis of ToF for the THz pulses can be used to determine the basis weight (mass per unit
area) of manufactured products. A material's ToF value is found in the following manner: when
photons transmit through a material, the transit time of the photon will be increased due to the
increased refractive index (RI) of the material compared with the RI of photons in air or vacuum
(~1). The ratio of the velocity of photons in a vacuum to the velocity of photons in the material
of interest defines the RI for that material. Because the velocity of the EM is less in a material,
the amount of time required for the EM to transmit through the material will be longer. The
difference in time between the EM pulses transmitting through the material, compared with the
same transmission through air, although extremely small can be precisely measured with THz
instrumentation. This difference is the ToF delay. This ToF delay (typically in ps) is calibrated
against basis weight values for the sample material determined using laboratory measurements.
The THz method to measure a material's basis weight is to measure the RI that causes the
increase of the ToF of the EM pulse as it transmits through the material of interest. This ToF
3
-------
value, which can be translated to an RI, is calibrated against accepted values of the material's RI
and basis weight. The THz method is a time-based measurement, as opposed to the amplitude
attenuation-based measurement method of nuclear gauges.
The measurement of basis weight is the most common use for nuclear gauges in industry. The
Appleton Paper Company, Appleton WI (Appleton), routinely uses nuclear gauge technology
and Technical Association of the Pulp and Paper Industry (TAPPI) laboratory measurements for
its business and maintains an appropriate safety license to operate a nuclear gauge. The
recognized TAPPI procedure for basis weight is to measure the basis weight of a specific area of
a sheet product (e.g., paper) under controlled laboratory conditions using an analytical balance.
The specific area cut from the sheet product varies depending on the product. Example basis
weight units of measure are pounds per square yard or grams per square meter. For the paper
industry, a common unit is pounds per ream, where a ream represents 3,300 square feet of paper.
A wide range of material basis weight values (5 grams per square meter [gsm] to greater than
100,000 gsm) can be measured with this single source THz instrument. The THz system directly
measures the ToF increase due to the pulse passing through the material under test. Formally this
increase in ToF is the volume of material in the beam path times the RI of the material at the
THz frequency minus 1. Finally, the amplitude of the transmitted THz pulse may be configured
to provide simultaneous complementary information related to the chemical and physical
properties of the sample, including moisture content. In this verification test, a distinction was
made between a measurement of thickness (sometimes called caliper thickness) or the physical
dimension of the material under test, and the basis weight, which is the mass per unit area. A
nuclear gauge measures the amount of matter between the source and detector, which is most
directly converted to basis weight. In circumstances where the material has a uniform density,
the nuclear gauge measurement can also be correlated to physical thickness.
Most THz measurements are made in reflection, as this geometry simplifies the system
configuration and reduces cost. Often, a fixed metal plate is installed behind the sample. The
THz pulse, reflected off a rear metal plate, will have transmitted through the sample twice. This
measurement mode is equivalent to double pass transmission and the measured ToF delay is
therefore increased by a factor of two.
The use of a beam splitter in the reflection sensor allows the transmitting and reflecting THz
pulses to remain collinear throughout the inspection (see Figure 2-1). Therefore, the sensor
operates best when aligned orthogonal to the inspection surface. However, for illustration
purposes, an angle is often shown between the transmitter and receiver (see Figure 2-2). This
display method helps to clearly separate the incoming and reflecting THz pulses and thus better
illustrates the origin and timing of the reflection pulses.
-------
Sample
TVfe Pulse
Frcot
• :
• !•.;
'•• .
Surface
Reflection
Figure 2-1. Collinear THz Reflection Sensor (with Beam-Splitter)
Figure 2-2. Illustration of Photon Pulse's Origin and Detections
Because the most common use of nuclear gauges in industrial settings is the measurement of
basis weight, this verification test compared basis weight values determined by these two
systems vs. the laboratory-generated basis weights measurements.
The fundamental method of nuclear gauges to find basis weight is to calibrate the measured
attenuation of the nuclear particle flux when passing through a sample against a standard-
5
-------
method-determined or accepted basis weight value. If the average density of the product is
known and remains constant, then the average sample thickness can be calculated from the
measured basis weight value.
The fundamental measurement with a THz sensor is the delay in the ToF of the THz pulse as it
passes through a sample. In a transmission measurement, the delay can be directly measured and
then calibrated against basis weight, a similar process to nuclear gauges. The sample's ToF is
calibrated against a standard method determined or accepted basis weight values.
Figure 2-3. Picometrix, LLC, T-Ray 4000® Time-Domain THz System
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
This verification test was conducted according to the technical and quality assurance/quality
control (QA/QC) procedures specified in the Quality Assurance Project Plan (QAPP) for
Verification ofPicometrix, LLC, T-Ray 4000R Time-Domain Terahertz System" ^ and complied
with the data quality requirements in the AMS Center Quality Management Plan (QMP).*^ As
indicated in the QAPP, the testing conducted satisfied EPA QA Category II requirements, which
establishes QAPP requirements for important, highly visible Agency projects involving areas
such as supporting the development of environmental regulations or standards.. The QAPP and
this verification report were reviewed by:
• Paul Thomas, 3M
• Mike Barlament, Kimberly-Clark
• Temeka Taplin, Mele Associates
• Madeleine Nawar, U.S. EPA.
Battelle conducted this verification test with funding support from the EPA's Office of Air and
Radiation.
3.2 Test Design
A non-radio isotopic source THz technology (Picometrix, T-Ray 4000® Time-Domain Terahertz
System) was tested at Appleton Paper Company, in Appleton WI. This allowed for performance
evaluation under real world manufacturing conditions. The performance of the THz technology
was verified based on accuracy, precision, comparability, and operational factors. The test
design consisted of a production line and a laboratory phase.
The performance of the technology was tested using two different weights of paper -mid-weight
and heavy weight. For each paper type, five separate rolls (treated as lots by Appleton) were
tested resulting in ten sets of synchronized nuclear gauge and THz sensor data collected during
the actual production process (i.e., the production line phase). All steps within a single lot run
were completed within a single roll of product. A paper sample was collected from the end of
each roll used for production line testing. The 10 samples collected from these runs were used
for testing during the laboratory phase. The paper sample basis weight from each lot was
measured using standard TAPPI gravimetric protocols, cut into small squares, stacked, and the
7
-------
ToF measured for each stack using the THz technology on a stable mount. A qualified Appleton
technician conducted all reference measurements for this verification test according to
procedures described in Appleton Standard Test Method 10001.00 Basis Weight - Laboratory
Determination of Coated and Uncoated Paper^ as modified in the QAPP.
The nuclear gauge technology was a Measurex, MX Open system, and was owned by Appleton
and operated by qualified Appleton staff according to standard Appleton procedures. The
Picometrix THz technology was operated by the vendor for both the production line and
laboratory phases. Testing was conducted over three days. Production line and laboratory
measurements for medium-weight paper took place on February 16 and 17, 2011. Testing on the
heavy-weight paper took place on February 23, 2011.
3.2.1 Production Line Testing
Production line testing involved measuring paper basis weight simultaneously using the nuclear
gauge and a THz sensor installed in the line. For testing purposes, the THz sensor was bolted to
a mounting bracket at a fixed position approximately 2-5 inches from the edge of the moving
paper sheet. The production nuclear gauge system was mounted on a scanner frame which
normally moves back-and-forth across the sheet. For verification testing, the nuclear gauge
system was "parked" at the edge of the paper and the THz sensor aligned with the nuclear gauge
between 2-5 inches from the edge of the sheet. Both the THz sensor and nuclear gauge remained
stationary during testing. Figure 3-1 shows a schematic of the gauge positions and Figure 3-2
shows the nuclear gauge and THz sensor during production line testing. Figure 3-3 shows a
close-up of the THz sensor during testing.
Beta
Gauge
Sheet
Machine
Direction
Sheet
THz
2" - 5"
THz Sensor
Figure 3-1. Relative Position of Sample Paper Sheet, Nuclear (Beta) Gauge, and THz
Sensor
-------
Figure 3-2. THz Sensor and Nuclear Gauge During Production Line Testing. The THz sensor is
seen to the right on a mounting bracket (Yellow oval). The nuclear gauge housing is seen to the left
of the roller (red oval). The arrow denotes paper flow.
Figure 3-3. Close-up of THz Sensor in the Production Line. The paper on the production line is
seen between the two lower plates of the gauge. The yellow arrow indicates the THz photon path.
-------
Ten lots of paper were tested in a standard paper coating production line; five mid-weight and
five heavy weight paper products. The process was identical for each lot. Production of either
the mid-weight or heavy-weight was initiated; production parameters were monitored via
Appleton's computer-based data collection system. During the monitoring period, the nuclear
gauge was sweeping back and forth across the moving paper sheet collecting continuous
measurements. This process took from three to six hours, depending on the number of coatings
being applied and production issues during any of the runs. As part of their standard
manufacturing procedure, each production run was assigned a Reference Roll Number. This
number was logged in the trial notebooks and used to track individual samples.
Once proper manufacturing stability was confirmed by the Appleton production representatives,
the nuclear gauge was moved to the edge of the paper and parked as close to the edge as
possible; the THz sensor was then aligned as precisely as possible, using manual inspection with
a 1/16 inch scale ruler, to the same location on the sheet (cross direction) as that of the nuclear
gauge. A difference between the THz and nuclear scan spot of 1A inch was possible. The nuclear
scan spot was typically 2 inch in diameter, and the THz sensor spot was 1A of an inch, so an
offset of the 1A inch from the center of the nuclear gauge scan spot was not thought to be
significant. During the 10 runs, the sensor positions (together) ranged between 2 and 5 inches
from the edge of the sheet. The sheet edge position varied from product to product, hence the
variation in the distance from sheet edge. The THz sensor was positioned as close as possible
along the direction of material manufacture (machine direction), within five feet in front
(downstream) of the nuclear gauge.
Immediately before sample data collection for each lot, the THz sensor assembly was rotated
"off-sheet", i.e., so that there was no paper in the measurement path. In this off-sheet position, a
system check was conducted that made ToF measurements of the air space between the upper
and lower plates of the sensor assembly. These 'open-air space' data were collected for
approximately two seconds and stored for use in the basis weight calculation. The sensor was
then rotated back to the fixed position aligned with the nuclear gauge. On-line data collection
began within one minute after the completion of this procedure, and was simultaneously
collected and logged for both the nuclear and THz systems for a continuous period of at least
five minutes. The nuclear gauge was moved off-sheet and parked 16 seconds before the end of
the rolll and an end-of-run two-second ToF open air space THz system check was performed
(Figure 3-4). At the end of each lot roll, an end-of-roll "tear-off paper sample was collected.
These samples were marked and saved for laboratory and THz inspection. To ensure data
comparability, the time of each of these events was documented in the data log.
The nuclear gauge moved off-sheet (i.e., ending on-line measurements) 16 seconds before the end of a roll. This
step was required to allow an individual to paste the end of the previous paper roll to the beginning of the next roll.
Once the start of the new production roll was past the inspection point, the nuclear gauge returned to scanning the
product.
10
-------
Figure 3-4. THz sensor rotated 'off-sheet' for air space reading
The data collected throughout each run was time stamped by each technology; nuclear gauge to 5
second increments and THz results to 0.01 second increments. This time stamp data allowed the
two technology results to be more precisely aligned. Within 10 minutes after each data collection
run, the two technology system clocks, used to timestamp the nuclear gauge and THz data were
compared and recorded to within a ±1 second increment for both technologies.
The nuclear gauge acquired measurements at a 1/5 Hz rate. Thus, a 5-minute period would result
in a log of 60 nuclear gauge measurements. The THz system acquired measurements at a 100 Hz
rate, thus 30,000 measurements were collected over the 5-minute period. At the mid-point of
each day of testing, the THz system time measurement calibration was checked (System Check).
This calibration procedure followed protocols established by Picometrix.
The nuclear gauge was operated and calibrated according to manufacturer recommendations and
established Appleton protocols.
Environmental conditions during the production line testing were not controlled for ETV testing
purposes. Thus, both gauges operated in a manufacturing factory environment within the
Appleton production facility. Environmental conditions during testing are presented in Table 3-1
based on independent observations made by a Battelle staff member who was present throughout
testing and who conducted independent monitoring of temperature and relative humidity on the
production floor and in the laboratory. A hand-held Hobo/Onset Model HI4-002 monitor was
used to monitor production floor temperature and relative humidity. The instrument was
11
-------
calibrated in Battelle's ISO 17025 calibration laboratory prior to testing. Data were recorded via
a data logger.
3.2.2 Laboratory Testing
The purpose of laboratory testing was to provide a direct comparison between THz ToF basis
weights and the basis weights determined using standard TAPPI gravimetric analysis made on
the "tear-off samples collected at that end of each roll of paper used for production line testing.
Five tear-off samples were collected for each paper weight (i.e., one from each of five mid-
weight rolls and one from each of five heavy-weight rolls tested). The same THz sensor that was
used in production line testing was moved to the TAPPI room for static off-line THz
measurements. Laboratory testing was performed in a TAPPI room located on the manufacturing
floor. The environment of this room was tightly controlled (72 °F and 50% RH). Laboratory
basis weights were measured by trained Appleton laboratory personnel. Each "tear-off paper
sample was folded into four layers and cut precisely into 9.5 x 12.5 inch squares, creating a 475
square-inch sample. The sample was weighed on an analytical Mettler balance calibrated to read
basis weight (Ib/ream) directly for this sample size. The laboratory-measured basis weights were
recorded by a Battelle staff member into the project logbook. The ongoing and overall basis
weight for each paper lot (roll) was also determined on the production line by the nuclear gauge
and the Appleton production process control software. Figures 3-5 through 3-7 show pictures
from the laboratory testing phase.
Once basis weight determination was complete, the mid-weight paper was cut into 96, 2-inch
squares. Further details on the cutting size are provided in Section 6.2. Eight stacks of 12
squares were then created. Each stack was placed in the THz sample holder and measured for
approximately 30 seconds, or until the ToF value was stable. The same process was used for
both paper weights except that for the heavy-weight paper the 2-inch squares were measured in
16 stacks of six pieces each because the extreme thickness of the paper resulted in a noisy signal.
Thus eight replicate measurements for the mid-weight paper and 16 replicate measurements for
the heavy-weight paper were collected by the THz technology. These results were used in the
precision calculations. The precision results of these measurements are discussed in Chapter 6.
For one mid-weight sample, three replicate samples were cut for basis weight measurement to
assess cutting variability. For one heavy-weight 6-piece stack, three separate THz measurements
were collected to assess measurement reproducibility.
The QAPP specified that the laboratory results from the first data collection run would be used to
temporarily calibrate the THz sensor to output basis weight values. This proved impractical due
to the production schedule and availability of Appleton staff to perform the basis weight
measurements. Therefore, a deviation was prepared, and all "raw" THz measurement values
were saved and post-calibrated once all measurement results were available.
12
-------
Figure 3-5. Standard Basis Weight Determination using an Analytical Balance. The
balance is calibrated to read basis weight when four 9.5 inch x 12.5 inch sheets are weighed.
Figure 3-6. THz sensor system in the Laboratory. One stack of mid-weight paper is seen
in the sample holder (highlighted in a yellow oval).
13
-------
Figure 3-7. Close-up of THz sensor system in the Laboratory
3.3 Test Samples
Laboratory samples were collected at the end of each roll for which a set of nuclear and THz
sensor measurements were collected. Thus, five laboratory samples were collected from both the
mid-weight and heavy-weight paper stocks. Approximately 50 feet of paper were removed and
discarded from the end of each roll to eliminate any end-of-roll artifacts. Then approximately 6
feet from the 12 foot wide roll were collected, folded, labeled and transferred to the production-
floor laboratory within the Appleton production facility. The samples were then conditioned for
at least 10 hours in the room where they would be evaluated before laboratory analysis.
3.4 Test Conditions
Temperature and relative humidity can impact basis weight values. Therefore, TAPPI standard
T402 sp-03 Standard Conditioning and Testing atmospheres for paper, board, pulp handsheets,
and related products (2003) specifies the environmental conditions under which basis weight
should be determined:
Temperature: 23±1.0 °C (73±1.8 °F)
Relative Humidity: 50%±2.0%
A hand-held Hobo/Onset Model H14-002 monitor was used by Battelle to measure temperature
and relative humidity throughout testing. These values were monitored on both the production
floor and the room used for laboratory analysis. The Hobo monitor gathered and stored data at
30-second intervals. The files were downloaded for analysis at the end of each test day. Table 3-
1 summarizes the actual environmental conditions measured by Battelle during testing. Shaded
values indicate environmental condition in excess of the TAPPI standards.
14
-------
Table 3-1. Environmental Conditions During Testing.
Sample ID
ACA11B16026
ACA11B16027
ACA11B16028
ACA11B16033
ACA11B16035
ACA11B16037
ACA11B23035
ACA11B23036
ACA11B23042
ACA11B23043
ACA11B23044
Production Line
Sample
Run
Time
Temperature
(°F)
Relative
Humidity
(%)
Mid-weight Feb 16, 2011
15:32
15:55
16:16
17:20'
18:20
19:29
77.3
78.0
76.6
n/a-2
n/a-
71.8
24
23
25
n/a-2
n/a-2
53
Laboratory
Sample
Run
Time
Temperature
(°F)
Relative
Humidity
(%)
Mid-weight Feb 17, 2011
09:56
10:16
10:36
09:14
11:06
11:20
75.2
75.2
75.2
71.8
75.2
75.2
46
47
46
41
46
46
Heavy-weight Feb 23, 2011
11:54
12:14
14:12
14:32
14:48
75.9
76.6
75.9
74.5
75.2
17
18
17
19
19
20:44
18:33
19:52
20:10
20:30
72.5
72.5
72.5
72.5
72.5
50
52
50
49
50
Only laboratory data were collected for this lot; the THz system collected data to the wrong channel thus there is
no associated THz data.
2 The Hobo unit was moved to the Laboratory to monitor the TAPPI room conditioning environment during
collection of these samples.
For most production line testing and mid-weight paper laboratory testing, temperature exceeded
the TAPPI standard for basis weight measurement and relative humidity was lower than the
standard. It should be noted that the temperature and relative humidity of paper in the
production line was controlled through the process and that air temperature is not a reliable
surrogate for paper conditions. During laboratory testing on February 17, 2011, conditions did
not meet TAPPI standards. This was due to the frequent opening of the laboratory door by
Appleton staff that monitored production problems. While this room's environment may have
had an impact on the testing results, observations during measurements indicate that prolonged
handling of the paper stacks had a much larger effect. For some paper stack samples, a much
longer data collection time was used to allow the THz reading to stop decreasing and stabilize. It
has been assumed that this was due to moisture in the paper stacks, gained during handling,
being lost while equilibrating in the room environment. Thus, the minor room environmental
condition inconsistencies were thought to be inconsequential. The increased time for data
collection (up to 30 seconds) for some laboratory paper stack measurements was also considered
inconsequential.
3.5 Testing Parameters
Test parameters for this validation test included: accuracy, precision, comparability, and
operational factors. The results for each parameter are discussed below.
15
-------
3.5.1 Accuracy
Accuracy was assessed by evaluating the basis weight determined by the Picometrix, LLC, T-
Ray 4000® TD-THz System against basis weight measured using the standard laboratory
method. For the production line samples, the comparison of accuracy was limited because the
THz system ToF measurements made during production line runs were not made on the exact
same location on the paper as the laboratory samples collected from the end of the roll. To
address this complication, off-line static THz measurements were made on the paper samples in
the laboratory. By making a large number of measurements over the spatial area of the lab
sample, the accuracy of the THz measurement could be assessed. Thus, only offline THz
measurements and laboratory basis weight results were used to assess the accuracy of the
Picometrix THz technology. The results are summarized in Chapter 6.
3.5.2 Precision
The precision of the Picometrix, LLC, T-Ray 4000 TD-THz System was assessed by triplicate
THz measurements of one stack of the heavy paper used for laboratory basis weight
determination. The stack contained six, 2 inch square samples. In addition, the cutting
precision of the 9.5 inch x 12.5 inch paper sample was assessed by cutting and weighing three
mid-weight samples from Sample ACA11B16028. The results are summarized in Chapter 6.
3.5.3 Comparability
This test assessed the performance of the Picometrix, LLC, T-Ray 4000® TD-THz System based
on the measurement of whole sample basis weight as compared to laboratory basis weight
values. A second comparison was also made between the THz sensor and nuclear gauge based on
production line testing results. For this comparison, results from the nuclear gauge and THz
sensor were directly evaluated against each other for comparability. Data obtained by each
technology were assessed on a roll by roll basis for comparability. Because of the operational
need to remain static for this testing, cross-sheet scanning was not conducted. An additional
factor limiting comparison was the difference in measurement spot size between the two
technologies. The results of the comparison are presented in Chapter 6.
3.5.4 Operational Factors
Operational factors such as maintenance needs, power needs, calibration frequency, data output,
consumables used, ease of use, repair requirements, training and certification requirements,
safety requirements and image throughput were evaluated based on Battelle staff testing
observations and input provided from Picometrix staff. The results are summarized in Chapter 6.
16
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures were performed in accordance with the QMP for the AMS Center*-2-* and the
test/QA plan for this verification test/1-* QA/QC procedures and results are described below.
4.1 Quality Control
4.1.1 Instrument Calibration Checks
The THz system was set up at the beginning of each test day to optimize the signal. A standard
fused silica block was used to establish the initial signal output. As part of the QA/QC
requirements for the verification test, the system was checked mid-day (Section 4.2.2) to verify
that the system was still operating correctly and that no drift had occurred.
The balance, temperature measuring device, and hygrometer were calibrated according to the
manufacturer's specifications and Appleton procedures prior to testing. More details are
provided in Table 4-1. The nuclear gauge, a Measurex, MX-Open system, was calibrated by the
Appleton plant operator in accordance with the manufacturer's specifications and Appleton
procedures.
4.1.2 Laboratory Replicate Samples
The largest variable in laboratory basis weight determination was the precision with which the
sample was cut to an exact size for each basis weight measurement. To address this issue, three
test samples were cut from one of the mid-weight samples and weighed using the TAPPI
standard method to identify the error associated with the cutting precision. The average and
standard deviation of three replicate samples was 48.85 ± 0.18 Ib/ream and the coefficient of
variation 0.36%. This variability was considered low and was within TAPPI standard limits.
Thus, no adjustments were made to the laboratory data or considerations made for interpreting
the data analysis results.
4.1.3 Data Quality Indicators
The QAPP defined data quality indicators (DQIs) that would enable stakeholders to assess
whether the verification test provided suitable data for a robust evaluation of performance. DQIs
were established for paper basis weight measurements and the laboratory analyses that required
17
-------
control for this performance-based measurement. The DQI for these supporting measurements
are quantitatively defined in Table 4-1, along with the acceptance criteria and whether the
criteria were achieved.
Table 4-1. DQI and Criteria for Critical Supporting Measurements
Phase
Laboratory1
Confirmation
Laboratory
Confirmation
Laboratory
Confirmation
Laboratory
Confirmation
Laboratory
Confirmation
In-line
Production
In line
Production
In line
Production
Off-sheet
Production
DQI/Critical
Measurement
Accuracy/Balance2
Accuracy/TAPPI
Room
Temperature
Accuracy/TAPPI
Room Relative
Humidity
Hygrometer
Thermometer
Completeness/
Amount of THz
data collected per
second
Accuracy/THz
instrument
calibration
Accuracy/Amount
of accurate data
collected
Accuracy/mass in
chamber
Method of
Assessment
Certified
weights
Thermometer
Hygrometer
ISO 17025
Certified
Laboratory
ISO 17025
Certified
Laboratory
Time stamp
of THz data
stream
Calibration
standard
results vs.
initial
calibration
Review data
for anomalies
(> ±5% from
average)
Off-sheet
check of
optics
Frequency
Quarterly by
professional
balance service
and Prior to
testing
Continuously
during testing
Continuously
during testing
Annually and
within 1 week
of testing
Annually and
within 1 week
of testing
Each run:
number of data
points/ second
Daily Mid-day
check
Each run
Before and
after each run
Acceptance
Criteria
NIST
tolerances for
analytical
balances
23±1.0°C
(73±1.8°F)
50%±2.0%
±1.0%at23°C
and 50% RH
Graduated to
0.20 °C
(0.50 °F)
>99%
±20"15sec
<20%ofdata
average
< ±50"15 sec
difference
between
readings
Criteria
Achieved?
Yes.
Balance
calibrated
1-31-2011
Yes. Powers
monitor
calibrated
12-7-2010
Yes. Powers
monitor
calibrated
12-7-2010
Yes.
Hobo
calibrated
2-11-2011
Yes.
Hobo
calibrated
2-11-2011
Yes
100%
No.
Test devised
for trial was
not reliable
Yes
All > 80%
Yes
All results
<±5"15 sec
Laboratory Confirmation was the reference method for this test.
2 Basis weight determined using Appleton Spec. No. 10001.00 which was based on TAPPIT 410.
18
-------
4.2 Equipment Testing, Inspection, and Maintenance
All laboratory equipment was tested, inspected, and maintained according to Appleton internal
requirements and TAPPI standards to ensure that the performance requirements established in
the QAPP^ were achieved.
4.2.1 Hobo Continuous Reading Monitor
A hand-held Hobo continuous-recording monitor was used to verify the accuracy of the TAPPI
room temperature and relative humidity equipment. A comparison of seven concurrent readings
during testing verified that the temperature percent difference ranged from 0.14% - 4.21% and
that the relative humidity percent difference ranged from -5.58 to 5.63%. Based on these percent
difference values, the temperature and relative humidity from the TAPPI room and Battelle ISO
17025 certified monitors were considered comparable, and thus TAPPI room measurements were
considered representative of the laboratory testing environment.
4.2.2 THz Operation Verification
THz Systems Checks. THz instrument system checks were performed on each production line test
day and the laboratory static test day for the heavy-weight paper; however due to an oversight,
the THz system check was not performed on the day that the mid-weight paper was tested in the
laboratory. The QAPP specified that the checks should be ±20"15 seconds vs. the initial set-up
signal optimization. The actual check values exceeded these criteria; however the failure of this
check was not considered critical. Table 4-2 summarizes the results of these checks. As noted in
Section 4.4, the THz system checks did not meet this DQI.
The failure of the system check was not considered concerning or impactful. The empty air-
space sensor measurement was made before and after each sample measurement. These empty
air-space values are critical to a THz ToF measurement similar in the manner that tare weight
measurement are critical to determine addition weight in a gravimetric balance test. If these air
space measurements were inconsistent, then the system check was devised to help determine the
source of the inconsistency, whether control unit or external reference structure. The empty air-
space measurements proved consistent, indicating that the system and the external reference
structure were stable, and the system check devised for this trial, however, proved inconsistent.
These findings indicated that the system check method was flawed but this did not impact the
testing as the empty air-space measurements were consistent. The value of the system check was
never to be used in the calculation of any measurement parameter, thus the inability to hold
specification does not affect the measurement results.
19
-------
Table 4-2. THz Mid-Day System Checks vs. Daily Initial Set-up Values"
Date/Time
Feb 16,20117
Feb 16,2011/16:27
Feb 16, 2011/18;51
Feb 23, 2011/07:50
Feb 23, 2011/15:05
Feb 23, 2011/17:50
Initial Signal
(ps)
80.210
80.187
Mid-check Signal
(ps)
80.305
80.282
80.200
80.277
THz Location
Production
Production
Production
Production
Production
Laboratory
Difference (ps)
0.094
0.072
0.013
0.090
a - Shaded cells indicate those measurements were not made at that time.
THz Off-Sheet Air Space (Blank) Drift. Prior to each production line or laboratory test sample, a
5-second reading was collected by the THz sensor to establish the baseline ToF with no sample
in the beam path. Once sample measurements were complete, a 5-second reading was collected.
The QAPP acceptance criteria for the difference between the pre-and post-testing off-sheet optics
checks was < ±50~15 sec. The average of these two values was subtracted from each sample
reading as the baseline ToF value. Table 4-3 summarizes the results. The results from all tests
but one were within the acceptance criteria, with the maximum drift less than ±5~15 seconds
(0.005 ps).
Table 4-3. Summary of THz Air Space Samples
Sample
Production (ps)
Initial
Final
Difference
Laboratory (ps)
Initial
Final
Difference
Mid-Weight Paper
ACA1 IB 16026
ACA1 IB 16027
ACA11B16028
ACA11B16035
ACA11B16037
Missed
260.919
260.950
260.910
260.912
260.917
260.918
260.920
260.913
260.909
*
0.001
0.030
0.003
0.003
260.908
260.904
260.902
260.902
260.901
260.905
260.903
260.902
260.903
260.900
0.003
0.001
0
0.001
0.001
Heavy-Weight Paper
ACA11B23035
ACA11B23036
ACA11B23042
ACA11B23043
ACA11B23044
259.808
259.805
259.798
259.800
259.799
259.804
259.804
259.801
259.799
259.800
0.004
0.001
0.003
0.001
0.001
260.779
260.852
260.780
260.780
260.779
260.781
260.853
260.781
260.779
260.778
0.002
0.001
0.001
0.001
0.001
4.3 Audits
Two types of audits were performed during the verification test: a technical systems audit (TS A)
of the verification test performance and an audit of data quality (ADQ). Audit results are
discussed below.
20
-------
4.3.1 Technical Systems Audit
The Battelle Quality Manager, Rosanna Buhl, performed a technical systems audit (TSA)
at the Appleton Paper Plant during the production line and static laboratory testing of both mid-
weight (February 16-17, 2011) and heavy-weight paper (February 23, 2011) verification tests.
The purpose of this audit was to:
• Evaluate verification testing of the Picometrix Terahertz (THz) system to determine basis
weight in an actual paper production environment with in-line nuclear gauges;
• Evaluate verification testing of the Picometrix Terahertz system to determine basis weight
in the laboratory vs. Appleton and TAPPI procedures;
• Verify that temperature and relative humidity conditions in the laboratory met Appleton
and TAPPI requirements;
• Review calibration records of laboratory equipment (balance, temperature and relative
humidity monitor); and
• Verify that testing was compliant with QAPP requirements and that the required
documentation was being completed in real time to ensure data traceability.
The TSA consisted of observations of vendor technology operation and Appleton staff
performing routine procedures for laboratory and production line performance monitoring. A
TSA checklist was used to guide the audit. Procedures reviewed included set-up, calibration, and
testing using the THz system in the production area and laboratory, laboratory basis weight
measurements, laboratory equipment readings and calibration records. Four significant TSA
findings were identified.
• Relative Humidity: The relative humidity (RH) conditions in the TAPPI room during
laboratory testing at Appleton were not maintained at the 50.0%±2.0% range specified in
the QAPP DQI Table 2. Actual RH values, measured using the Hobo monitor, ranged
from 41-47%.
• THz Calibration: The QAPP states that the mid-day THz checks should be ±20~15
seconds vs. the initial calibration. The mid-day checks did not achieve this criterion.
(Picometrix staff indicated that the term "initial calibration" was a misnomer; a more
accurate term would be initial set-up to optimize the signal).
• Production On-line Testing THz Calibration Check: The THz system time
measurement calibration was not checked on the day that laboratory static testing was
performed on the mid-weight paper.
• Sample Design: The laboratory measurements of the heavy-weight paper did not follow
the sample design. Rather than 12 pieces of paper in eight stacks measured by the THz
during laboratory testing, 6 pieces of paper in 12 stacks were measured.
21
-------
The remaining observations noted documentation errors and issues that would not impact data
quality. A TSA report was prepared, and a copy was distributed to the EPA.
Details on how the THz sensor calibration and sample design findings were addressed are
described in Section 4.4 in Deviations 2 and 4, respectively. Relative humidity information as
recorded by the HOBO was provided in the report, with those values outside of the specified
range highlighted. Battelle does not believe that these values impacted the study or data quality.
The production on-line testing THz sensor calibration test was not checked due to a
misunderstanding by the Picometrix operator. It was determined in consultation with the vendor
that this missed calibration test did not impact the performance of the THz sensor or the resulting
data. All empty air space measurements were consistent. See Deviation 2 in Section 4.4 for
further details.
4.3.2 Data Quality Audit
The Battelle Quality Manager audited at least 25% of the sample results data acquired in the
verification test and 100% of the calibration and QC data versus the QAPP requirements. One
audit of data quality (ADQ) was conducted for this project encompassing the review of raw data,
synthesized data and the verification report. The ADQ was initiated within 10 business days of
receipt by the Quality Manager and assessed using a project-specific checklist. During the audit,
the Battelle quality manager, or designee, traced the data from initial acquisition (as received
from the Appleton or the Picometrix technology), through reduction and statistical comparisons,
to final reporting. Data underwent a 100% validation and verification by technical staff (i.e.,
Verification Test Coordinator (VTC), or designee) before it was assessed as part of the ADQ. All
QC data and all calculations performed on the data undergoing the audit were checked by the
Battelle Quality Manager or designee. Results of the ADQ were documented using the checklist
and reported to the VTC and EPA within 10 business days after completion of the audit.
The ADQ resulted in two findings and one observation. One finding noted that deviations (as
described in Section 4.4) noted in the TSA needed to documented in the report and submitted for
approval. The second finding noted that various calculation errors were found in the
spreadsheets used for data analysis. The observation indicated that a discussion of data
completeness for the mid-weight paper roll should be included in the report. All findings and
observations were addressed and corrected appropriately.
4.4 Deviations
Four deviations were documented during testing:
Deviation 1 (2-16-11): Testing and data collection began for the Verification of Picometrix,
LLC, T-Ray®™ 4000 Time-Domain Terahertz System before the QAPP was approved by the
EPA AMS Center Project Officer (PO). The QAPP was approved on 3-22-11. Impact: Battelle
believes that this did not impact the data quality of the test. The changes to the QAPP did not
include any items that would impact the test design or data collection.
Deviation 2 (2-16-2011): The QAPP states that the mid-day THz checks should be ±20"15
seconds vs. the initial set-up that optimizes the THz signal. The actual values are reported in
22
-------
Table 4-2 (Section 4.1.2). Impact: Battelle believes that the checks exceeding the specification
did not impact the measurement results. The test was devised for this trial to help determine the
source of variation, whether control unit or external reference structure, if the external reference
structure empty air-space measurements were not stable. The air-space checks were very stable,
thus the system and the external reference structure must have been stable. The result of the
system check was never to be used in the calculation of any measurement parameter.
Deviation 3 (2-16-2011): The QAPP states that data from the first run of each paper weight
(mid-weight or heavy-weight) should be used to temporarily calibrate the THz sensor to provide
real-time results for subsequent measurement runs. However, the production process and
schedule made this approach impractical. Rather, all data were collected electronically by the
THz data system and were post-calibrated against the laboratory basis weight data. Impact:
Battelle believes that this did not impact the data quality of the test. The initial plan would only
have provided interim basis weight data and generation of final data were not impacted.
Deviation 4 (2-16-2011): The QAPP states that 12 pieces of paper in eight stacks would be
measured by the THz during the off-line phase of the test to determine basis weight. However,
for the heavy weight paper, 16 stacks with 6 pieces of paper were measured. The total number of
pieces of paper measured (96) was not changed but the stack size varied. This deviation was
necessary due to the thickness of the heavy weight paper (3 mm) which created a lot of noise and
outliers in the THz scan. Impact: Battelle believes that this did not impact the data quality of the
test. The total number of paper pieces tested was the same and thus still represents the same
sample area.
Deviation 5 (9-26-2011): The QAPP states that two audits of data quality (ADQ) would be
conducted. An initial ADQ was to be conducted on the on-site collected data within 10 business
days of receipt by the Quality Manager. A second ADQ was to be collected on synthesized data
and verification report. Because of the significant data reduction required to analyze the data the
VTC and Quality Manager agreed that it was more practical to conduct one combined audit.
Battelle believes that this did not impact the data quality of the test.
23
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the quantitative performance factors listed in Section 3.5
are presented in this chapter. Qualitative observations were also used to evaluate verification test
data.
5.1 Accuracy
The accuracy of the results was assessed by calculating the percent error between the laboratory
measurements and the results from the offline THz technology readings. Percent error was
calculated using the following:
{Technology - Lab
%Error = l- — x 100 (1)
Lab
It should be noted that the laboratory measurement used for comparison in the accuracy statistic
was not without some measurement error itself, as was the THz sensor. As such, the difference
between the laboratory and THz measurement includes errors from both of these measurements.
5.2 Precision
The precision was calculated as the standard deviation of repeated measurements made by the
THz technology during laboratory testing using the following equation:
StdDev = I) -^ (2)
N-l
5.3 Comparability
Comparability between the technologies was assessed by calculating the percent difference
between the measurements made by the THz technology and the nuclear gauge while in the on-
line production mode. This evaluation helped in assessing the performance of the THz
technology in relation to that of the instrument typically used for production control processes
(i.e., the nuclear gauge). Percent difference was calculated using the following:
24
-------
„.„... (Terahertz Technology Result - Nuclear Gauge Result] ,„.
^Difference = ± — '- x 100 (3 )
Nuclear Gauge Result
Comparability results were calculated for each run evaluated during the verification test. A
paired t-test was used to determine if the percent difference between technology readings for a
given roll of paper were significantly different from zero.
25
-------
Chapter 6
Test Results
The results of the verification tests of the Picometrix T-Ray 4000 TD-THz System are
presented below for each of the performance parameters.
Raw THz ToF data were processed to generate basis weight values as described below. This
process was devised by Picometrix and is how they normally process similar data. The
measurement of interest was the "weight" of a continuously manufactured sheet material. The
factor chosen was the basis weight. Basis weight is the weight of a prescribed area sample. A
common example is grams per square meter (g/m or gsm). For the paper industry, the units
used are pounds per ream (Ib/ream) of paper. For these samples, a ream is 3,300 square feet of
paper.
The THz measurement of interest was the ToF for the THz pulse to travel through the sample
and be reflected back to the sensor. The ToF value, in ps, was the "raw" THz output value. This
value was calibrated against accepted values of basis weight for the sample under study. The
THz system used to make measurements in this study collected data at a 100 Hz rate (i.e., system
collects 100 data points per second).
The nuclear gauge used in this test measured the number of beta particles (electrons) passing
through the sample, indicating beta particle flux. Numerous factors affect the transmission of
beta particles through materials, including but not limited to the thickness, density, and water
content of the material. The beta particle flux values are calibrated to accepted values of basis
weight for standard test materials. The nuclear gauge used for this testing output data at a 0.2 Hz
rate (i.e., system collects one data point every five seconds).
It should be noted that cross sheet scanning was typically performed by the nuclear gauge during
production at the Appleton facility. The scanning was required to monitor the uniformity of the
coating process. As the nuclear gauge scanned across the sheet, the product was divided into a
number of cross direction "bins". The nuclear gauge scans across the web at approximately 6
inches per second, producing 96 bin measurements for each single 15 second scan across the
product. Then, a number of repeated scans were averaged before a measurement result for that
bin was reported. This means for routine operation of a nuclear gauge, a single bin measurement
was an average of a number of sets of measurements with each individual measurement covering
a certain amount of product. These values were running averages. These average results were
then reported through the system to help monitor the production process. The averaging of both
across the sheet (cross direction) measurements and along the production direction (machine
direction) measurements was used to maintain controlled measurement feedback to the coating
devices. However, for this test, the gauges must remain in "parked" mode. Thus, the cross web
26
-------
averaging cannot be accomplished. However, the data acquired during this test was the closest
approximation to the typical operation of the nuclear gauge and THz sensor in a controlled
setting.
To generate THz basis weight values, 100 THz data points were averaged to provide
measurements at a 1 second rate. The nuclear gauge values were accepted as accurate, and the 1
second increment THz ToF values were calibrated to the nuclear gauge basis weight (pound per
ream of paper) values. This operation was conducted as follows.
1) A single THz ToF to nuclear gauge basis weight calibration factor (units of Ibs/ream/ps)
was found for each data set. This was done by finding the average nuclear gauge basis
weight value, the average THz ToF value over the two entire data sets and then simply
finding the ratio of these two values.
2) The five individual data sets calibration factors were then averaged to generate a single
calibration factor for each sample type.
3) Using this single calibration factor, the reverse process was carried out for which the 1
second increment basis weights were recalculated for each THz data set. The results
reported are the measurement time (1 second increment), associated nuclear gauge basis
weight measurement (if it existed), the "raw" THz ToF measurement (if it existed) and
the THz calculated basis weight.
These THz basis weight values were then used for the calculations described in the following
sections.
The THz data were also examined for outliers during processing. The THz errant readings are
most often caused by an incorrectly assigned finding when determining the time and amplitude
of the waveform pulses. As multiple outliers were possible, a Grubb's t-test was used to
determine outliers. First the mean and standard deviation were calculated (Xbar and s). Then,
starting with the largest outlier, T was calculated, where
„ _
If T> 3.017, then Xt was rejected. That is, that result was regarded as an outlier and removed
from the data set. Then Xbar and s were recalculated from the remaining data and the Grubb's li-
test was used to determine the next largest outlier. Some outliers were found in the THz
datasets. In any instance, less than 0.1% of the data (30 out of 30,000 measurements) was
considered to be outliers through this process.
6.1 Accuracy
Accuracy was assessed by comparing the results from the measurement of basis weight using the
TAPPI gravimetric reference method to the offline (laboratory) THz basis weight measurements
on tear-off samples from the end of each roll of paper evaluated in the verification test. Both
gravimetric and offline THZ measurements were made in the temperature and humidity
controlled TAPPI room. The results of the two basis weight measurements were compared by
27
-------
evaluating the percent error between the laboratory results and offline THz results. Table 6-1
summarizes the accuracy of offline THz basis weight data compared to the TAPPI gravimetric
reference method.
The percent error for the THz basis weight, as compared to the laboratory gravimetric reference
method values, ranged from 0.1-2.8% for the mid-weight rolls and 0.3-2.9% for the heavy-
weight rolls. The percent error for the offline THz sensor readings, based on the averages across
all five rolls for a particular weight paper, was 0.02%. The average percent error for a mid-
weight ream measurement, calculated as the average of all of the mid-weight paper percent
errors presented in Table 6-1, was 1.4%, significantly higher than the average across all five
rolls. The average percent error for a heavy-weight ream was similar at 1.3%.
The standard deviations of the average laboratory gravimetric measurements as well as the THz
offline basis weight measurements are also provided in Table 6-1 for each paper weight. The
standard deviation of the mid-weight paper basis weight for the offline THz sensor results was
0.64 Ib/ream. This was similar to the standard deviation of the laboratory gravimetric reference
method basis weight measurements across all five mid-weight paper rolls (0.56 Ib/ream). The
standard deviations across all heavy weight paper rolls varied between the measurement
techniques. The standard deviation for the THz heavy weight measurements was approximately
twice that of the laboratory values. According to the vendor, typical standard deviations
expected for the THz technology generally range from 0.1% to 0.2%. It was determined after the
verification test that the design of the offline THz measurements may not have been ideal and
might explain the higher than expected standard deviation found for the THz offline results. This
is discussed further in the next section (Section 6.2).
28
-------
Table 6-1. Accuracy of THz Basis Weight Values vs. Standard Laboratory Measurements.
Sample ID
Laboratory
Gravimetric Basis
Weight
(Ib/ream)
THz Basis
Weight (mean ±
Std)
THz Basis
Weight
% Error
% THz to Laboratory
Basis Weight
Mid- Weight Paper
ACA1 IB 16026
ACA1 IB 16027
ACA1 IB 16028
ACA11B16035
ACA11B16037
Average
STD
50.05
49.83
48.77
50.02
49.57
49.65
0.56
49.59
48.58
50.15
50.06
49.93
49.66
0.64
0.9%
2.5%
2.8%
0.1%
0.7%
0.02%a
99%
97%
103%
100%
101%
100%
Heavy Weight Paper
ACA11B23035
ACA11B23036
ACA11B23042
ACA11B23043
ACA11B23044
Average
STD
157.25
157.54
157.01
155.00
156.05
156.57
1.04
156.66
152.96
158.36
158.44
156.59
156.60
2.22
0.4%
2.9%
0.9%
2.2%
0.3%
0.02%a
100%
97%
101%
102%
100%
100%
a - Average percent error based on percent error of average basis weight measurements, not average of all percent
error calculations.
6.2 Precision
The precision of the THz technology was assessed on offline replicate measurements made on
cut samples of heavy weight paper. Triplicate measurements were made on one stack of heavy
weight paper. The results are summarized in Table 6-2. The standard deviation was 0.0115 ps
across the replicate measurements with a coefficient of variation of <0.005%.
Table 6-2. Precision of Replicate THz Readings
Sample ACA11B23035
Rep 1
Rep 2
Rep 3
Average
STD
Coefficient of Variation
THz Results (ps)
266.917
266.917
266.897
266.910
0.0115
0.0043%
The vendor noted that, based on experience and past-performance of the technology, that the
accuracy and precision of the laboratory THz basis weight measurements and replicated THz
readings were higher than expected (by approximately 20x). A review of the laboratory testing
29
-------
method revealed a flaw in the test design used that likely led to these higher-than-expected
offline THz results.
As noted in Section 3.2.2, the paper sample size for the gravimetric analysis was four precisely
cut 9.5 x 12.5 inch squares, creating a 475 square inch sample. When developing the test plan
for the offline THz measurements, two main concerns were taken into consideration. First, that
the THz measurements needed to be taken over a set of points well distributed across the 475
square inches. Second, that the normal spatial variability in basis weight of the paper (i.e., its
formation) will require a large number of THz measurements to eliminate this variability.
The offline THz measurement procedure followed was to cut the entire sample into 96 four
square inch (2 inch by 2 inch) pieces. These individual pieces were then stacked into groups of 8
or 16 and the THz ToF was measured for the whole stack. The theory was that the stacking of
samples would randomize the sample's basis weight variation and speed the measurement
process.
This measurement method was dependent on the positioning, curvature and spacing between the
sample sheets. The variation in these parameters generated an unintended significant variation in
the ToF measurement results. The assumption was that the main THz pulse would pass through
the stack, reflect off the external reference structure (ERS) rear surface and pass back through the
stack. In this configuration, the measurement of this "first light" main pulse would represent the
total delay through the stack. The vendor indicated that the main pulse behaved as expected.
However, upon analyzing the data, the vendor explained that because of variations in the
positioning, curvature, and spacing between the sheets of paper in the stack, a very large number
of small amplitude peaks (<5%) also occur due to reflections between the sheets of paper of the
stack (see Figures 6-1 and 6-2).
Figure 6-1. Laboratory sample
measurement method
Multiple
reflections
Figure 6-2. Zoom on multiple reflections
from paper sheet samples
30
-------
These reflections always originate from the main pulse, thus they always will be present after
this main pulse. These small amplitude reflection pulses become convolved with the trailing
edge of the main pulse. Thus, this inter-sample sheet reflection changed the shape of the pulse.
In order to determine a reflection pulse ToF value, the entire pulse, typically around 4 ps was
used. Thus, changes in the trailing edge of the pulse will affect the measurement of the pulse's
time-of-flight. The vendor indicates that these changes will be variable, depending on the
configuration of the stack of the individual sheets. Curved samples, like paper, will exhibit more
variability.
Because stacks of paper were used, the vendor believes that this testing procedure affected the
laboratory THz basis weight measurements, resulting in a reduction in the accuracy and precision
of the laboratory stacked paper testing results. In retrospect, the vendor indicated that a more
robust measurement method would have been to cut the gravimetric samples into strips and drag
a single strip through the measurement spot while collecting continuous data. By cutting strips
throughout the sample, the desired spatial coverage would have been achieved.
6.3 Comparability
The comparability of the THz to the nuclear gauge was assessed by evaluating the online THz
basis weight results against the online nuclear gauge basis weight results. An initial exploratory
analysis was performed by plotting the basis weight results for each technology to determine
how well the online basis weight values track each other.
Figures 6-3 - 6-7 show the THz and nuclear gauge basis weight data plotted together for each
roll of mid-weight paper roll. The Y-axis scale spans 1-1.5 Ibs/ream for the midrange products.
Figures 6-8 - 6-12 show the THz and nuclear gauge basis weight data plotted together for each
roll of heavy weight paper. The Y-axis scales span 3.5 - 5.5 Ibs/ream for these products.
These data represent all of the THz and nuclear gauge data collected during the production line
testing. As noted previously, the THz data shown in each figure were averaged to provide
measurements at a one second rate. The nuclear gauge data are at a frequency of once every five
seconds. For Figures 6-3-6-12, the red data represent the nuclear (beta) gauge results and the
black data represent the THz sensor results.
As noted in Section 3.5.3, a factor limiting the comparison between the nuclear gauge and THz
sensor was the difference in measurement spot size between the two technologies. A typical
nuclear gauge inspection spot was 25 millimeter (mm), while a typical THz sensor inspection
spot was 2 mm. At a production line speed of 4000 feet/minute, these "spots" become spread to
25 mm x 1041 mm for the nuclear gauge's 50 millisecond measurement integration time and 2
mm x 125 mm with an 80 mm gap between THz measurements. Thus, these two systems
inspected the product in different ways. The nuclear gauge covered a larger area and thus could
provide an improved result for the average basis weight value. The higher measurement rate of
the THz system somewhat compensated for this difference. In addition, the smaller THz sensor
inspection spot allows for better streak detection and could possibly provide information on the
formation (uniformity of basis weight) of the sample.
-------
50.0-,
49.9-
49.8-
I 49.7-
ra
| 49.5 -
| 494^
1 493-
w
492-
49.1 -
49.0
-•-TH7
• Beta
0 100 200 300 400 500 600
Trial Run Time (seconds)
Figure 6-3. ACA11B16026 on-line results.
49.4-
49.2 -
T*m».
*
0 100 200 300 400 500
Trial Run Time (seconds)
Figure 6-4. ACA11B16027 on-line results.
49.1
49.0
~ 48.9 -
1 48.8 -
g
01 48.7 -
I
I
£ 48.5 -
Q.
£
ra
<» 484
48.3-
48.2
50 100
150 200 250 300 350
Trial Run Time (seconds)
Figure 6-5. ACA11B16028 on-line results.
50.4-
1
fi 50.0-1
I
49,4-
-•- THz
-•-Befa
0 100 200 300 400 500
Trial Run Time (seconds)
Figure 6-6. ACA11B16035 on-line results.
50 8-,
50.6-
50.0-
49.8-
49.4-
0 200 400 600 800
Trial Run Time (seconds)
Figure 6-7. ACA11B16037 on-line results.
32
-------
160.5 -,
160.0-
159.5-
| 159,0-
£ 158.5-
£
f 158.0-
•S 157,5-
£ 157,0-
« 156.5-
156.0-
155.5
Figure
0 100 200 300 400 500 600 700
Trail Run Time (seconds)
6-8. ACA11B23035 on-line results.
160.5-,
160.0-
159.5-
?
£ 159.0-
I
£ 158.5-
1
„ 158.0-
0 157.5-
| 157.0-
156.5-
156.0
0 100 200 300 400 500 60(
Trial Run Time (seconds)
Figure 6-9. ACA11B23036 on-line results.
.2 156.0-
•3. 155.5
I
155.0-
154.5-
154.0
0 100 200 300 400 500 600
Trial Run Time (seconds)
Figure 6-10. ACA11B23042 on-line results.
159-
£ 157-
100
600
200 300 400 500
Trial Run Time (seconds)
Figure 6-11. ACA11B23043 on-line results.
159.5-
159.0-
158.5-
158.0-
1575-
157.0-
156.5-
156.0-
0 100 200 300 400 500 600 700 800
Trial Run Time (seconds)
Figure 6-12. ACA11B23044 on-line results.
33
-------
Of note is the less-varied, more piece-wise linear fit appearance of the production nuclear gauge
measurements. Appleton expressed surprise that the nuclear gauge data appeared as linear and
smooth as shown in Figures 6-3 through 6-12. It was suggested that this was a result of some
filtering on the nuclear gauge results. Note that the nuclear gauge does not normally operate in a
parked position as was used in the verification test. This filtering introduces a significant change
in the "shape" of the production basis weight that was useful for production needs but, according
to Appleton, does not represent the actual second-by-second basis weight values. Unless the
THz measurements are filtered in the same manner, this change will introduce some error in the
quantitative comparison between the two data sets. The Appleton staff did not know the exact
filtering used on the nuclear gauge online basis weight data. Thus, it was determined that the
THz data would be treated with a relatively weak filter, an 11 point adjacent average, to attempt
to match the gross features of the nuclear gauge data. Thus, the THz results are presented in a
less processed manner to better illustrate the sensor's "real time" measurement capabilities.
Picometrix has indicated that the results can be further processed to introduce the necessary
damping needed for production feedback control.
As shown in Figures 6-3 through 6-12, in most cases, the THz data track the nuclear (beta) gauge
results well, following the trend of the nuclear gauge data. There are some instances, however,
where the data diverge. For the mid-weight paper, Figure 6-3 shows that the THz basis weight
measurements show a steeper rise at around 400 seconds while the nuclear gauge results rise at a
slower rate. Figure 6-4 also shows a slight divergence, with the nuclear (beta) gauge basis
weight values rising slightly while the THz results fall slightly at the beginning of the roll.
For the heavy weight paper, Figures 6-8 and 6-10 show divergence between the results of the
two technologies. In Figure 6-8, the overall trends in basis weight results are similar for both
technologies. However, the THz results are consistently lower than the nuclear gauge results. In
Figure 6-10, the nuclear gauge basis weight values peak noticeably around 100 seconds while the
THz values do not show such a change.
It is important to note that the y-axis was generally on a small scale and that the differences
being viewed are small in magnitude. For example, the offset between the basis weight results in
Figure 6-3 when the THz sensor results rise faster than the nuclear gauge results was
approximately 0.2 Ibs/ream. The offset in Figure 6-8 was larger, closer to 1-2 Ibs/ream.
To further explore the comparability between the THz and nuclear gauge production line
measurement results, especially quantitatively, the percent difference between paired
measurements for each technology on each roll of paper were evaluated. As noted previously,
the THz sensor recorded measurements significantly more frequently than the nuclear gauge and
were averaged during data preparation such that there were five THz measurements to every one
nuclear gauge measurement. These data were then used for pairing the nuclear and THz sensor
results. For these comparisons, the THz observation times were rounded to the nearest integer
and matched to the corresponding nuclear gauge times. This was accomplished using Excel's
VLOOKUP feature. In cases of ties, VLOOKUP captured the earliest observations of the two.
The percent difference was then calculated for each matched pair of THz and nuclear gauge
production line data.
Figures 6-13 through 6-32 show the percent difference between the THz and nuclear gauge
results for each roll of paper evaluated. For each production roll, two figures highlight the
34
-------
percent difference between the THz and nuclear gauge results. One figure is a function of point-
by-point percent difference for the entire trial run time, based on the matched pairs discussed
above. The second figure is a histogram of the same percentage difference results. In the
histograms, the percent difference results for a particular roll of paper are binned together in
0.1% difference increments and show the varying amounts of levels of percent difference within
the given roll of paper.
Figures 6-13 - 6-22 present results for the mid-weight rolls and Figures 6-23 - 6-32 are for the
heavy weight rolls. Note that in most cases, the histogram has Gaussian shape with the peak
centered on 0% difference. Many of the histogram figures, however, appear flattened which was
likely due to the comparison of the THz results and the forced linear fit sections of the nuclear
gauge values. The percent difference for the heavy weight roll ACA11B23035 shows a
consistently negative percent difference, highlighting the consistent offset of the THz and
nuclear gauge results shown in Figure 6-8. Note that in all cases, the percent difference was
<2% for the basis weight measurements by these two technologies. In fact, for many cases, the
percent difference was <1%. All percent differences were well below the 10% specified in the
QAPP.
35
-------
16026
16026
100 200 300 400 500
Trial Run Time (seconds)
Figure 6-13.ACA11B16026 Percent
Difference Trial Run Time
0 .5
Percent Difference
Figure 6-14. ACA11B16026 Percent
Difference Histogram
16027
16027
100 200 300 400
Trial Run Time (seconds)
Figure 6-15.ACA11B16027 Percent
Difference Trial Run Time
Percent Difference
Figure 6-16. ACA11B16027 Percent
Difference Histogram
16028
16028
100 200
Trial Run Time (seconds)
Figure 6-17.ACA11B16028 Percent
Difference Trial Run Time
-.2 0 .2
Percent Difference
Figure 6-18. ACA11B16028 Percent
Difference Histogram
36
-------
16035
16035
100 200 300 400 500
Trial Run Time (seconds)
Figure 6-19.ACA11B16035 Percent.
Difference Trial Run Time
-.5 0 .5
Percent Difference
Figure 6-20. ACA11B16035 Percent
Difference Histogram
16037
16037
200 400 600 800
Trial Run Time (seconds)
Figure 6-21.ACA11B16037 Percent.
Difference Trial Run Time
11
Eb
-.5 0 .5
Percent Difference
Figure 6-22. ACA11B16037 Percent
Difference Histogram
37
-------
23035
23035
200 400 600
Trial Run Time (seconds)
Figure 6-23.ACA11B23035 Percent.
Difference Trial Run Time
-.5
Percent Difference
Figure 6-24. ACA11B23035 Percent
Difference Histogram
23036
23036
200 400
Trial Run Time (seconds)
Figure 6-25.ACA11B23036 Percent
Difference Trial Run Time
-.5 0 .5 1
Percent Difference
Figure 6-26. ACA11B23036 Percent
Difference Histogram
23042
23042
200 400
Trial Run Time (seconds)
Figure 6-27.ACA11B23042 Percent.
Difference Trial Run Time
-1.5 -1 -.5 0
Percent Difference
Figure 6-28. ACA11B23042 Percent
Difference Histogram
38
-------
23043
23043
200 400
Trial Run Time (seconds)
Figure 6-29.ACA11B23043 Percent
Difference Trial Run Time
-1.5 -1
-.5 0 .5 1
Percent Difference
Figure 6-30. ACA11B23043 Percent
Difference Histogram
23044
23044
200 400 600
Trial Run Time (seconds)
Figure 6-31.ACA11B23044 Percent
Difference Trial Run Time
0 .5
Percent Difference
1 1.5
Figure 6-32. ACA11B23044 Percent
Difference Histogram
39
-------
The average basis weight value for each roll of paper as measured by the THz and nuclear gauge
technologies along with the average percent difference for each roll is provided in Table 6-3.
Standard deviations of each mean are also presented. The mean percent difference was
calculated by averaging the percent difference for each roll. To evaluate whether the mean
percent difference was significantly different from zero, a t-test was used.
Table 6-3. Comparison of Production Measurements Between THz and Nuclear Sensors
Roll
Number
Beta
Mean
Basis
Weight
THz THz
Mean SD
Basis
Weight
Mean
Percent
Difference
Percent
Difference
SD
T-test
P-value2
16026
16027
16028
16035
16037
23035
23036
23042
23043
23044
49.54
49.38
48.65
49.98
50.10
158.06
158.20
156.00
156.96
157.64
0.096
0.072
0.048
0.218
0.175
0.734
0.625
0.805
0.562
0.557
49.61
49.31
48.62
49.96
50.08
157.03
158.39
156.04
156.65
157.72
0.123
0.131
0.134
0.199
0.211
0.501
0.686
0.556
0.842
0.629
0.14%
-0.13%
-0.06 %
-0.04 %
-0.03 %
-0.65 %
0.12%
0.03 %
-0.20 %
0.05 %
0.282
0.275
0.260
0.449
0.297
0.283
0.381
0.542
0.550
0.527
<0.001
<0.001
0.122
0.429
0.133
<0.001
<0.05
0.567
<0.05
0.210
1. SD: Standard Deviation
2. T-test of whether Mean Percent Difference is significantly different from zero. P-values less than 0.05 indicate a statistical
significant difference from zero.
Overall, the variability between the nuclear gauge and THz basis weights, as indicated in Figures
6-13-6-32 and in the standard deviation columns of Table 6-3, was comparable. However, there
seems to be greater variability among the nuclear gauge heavy weight roll measurements as
compared to the THz heavy weight roll measurements. Roll Number 23035 shows a slight
departure in measurements between the nuclear gauge and the THz. This resulted in the roll
having the largest percent difference among the other rolls of -0.65%. The reason for this single
production run to show a significantly larger percent difference error is unexplained.
The absolute average percent difference error for most production runs was less than 0.15%.
Using the Q-test (Q = 0.645 > Q99<>/0 = 0.568) to reject the 23035 result as an outlier results in an
average percent difference of less than 0.1% for all rolls.
For the mid-weight samples, three of the five production runs had a sufficiently high p-value
(p>0.05) to be considered to have the statistically same mean value. Two of the five production
runs for the heavy weight samples had statistically the same mean value. The mean percent
40
-------
difference between the nuclear gauge and THz for each roll was significantly different from zero,
according to the t-test, for Roll Numbers 16026, 16027, 23035, 23036, and 23043.
Note that, because of the time series nature of the data produced during this test, the data are
autocorrelated. This means that the data near in time are more likely to be similar that those far
apart in time. To account for this, the data could be reduced by the interval at which they
become independent and are no longer autocorrelated. This adjustment was not made in the data
from this test, however, as the difference between the two technologies was very small and,
regardless of the interval of data used, this relationship would still remain very small and well
within the defined criteria of acceptance.
41
-------
6.4 Operational Factors
Operational factors related to the THz technology were evaluated based on Battelle staff testing
observations and input from Picometrix, LLC. Table 6-4 summarizes these operational factors.
For the base system configured for industrial applications, the list price is $220,000.
Table 6-4. Summary of THz Operational Factors
Operational Factor
Start-up
Ease of Use
Clarity of Instruction
Manual
User-friendly Software
Conveniences
Daily Status of
Diagnostic Indicators
Maintenance Needs and
Level of Effort/Vendor
Effort
Downtime Causes and
Duration
Data acquisition failure
Power Supply Nature
and Needs
Calibration Frequency
Data Output
Consumable
Needs/Use/Replacement
Repair Requirements
Assessment
Factory installation, set-up and peak definition. No factory or operator actions
are required after setup. Picometrix performs the setup on site. If the filter
settings are accidently changed, the password will lock the user out of the
system to prevent collection of erroneous data.
Once set up is complete, the software protocol will prompt for product and
coating stage and convert ToF to basis weight.
Not assessed. The manual is very large and detailed.
The user interface has not yet been developed but the intent is that it will be
programmed for basic use.
Small size, single-sided, no regulatory oversight required due to radiological
concerns
The units have diagnostics to tell if and when re -calibration is required. A
warning message is displayed and transmitted to the unit.
The laser must be replaced every 5-10 years. The laser window must be
cleaned weekly by the user.
Significant static electricity was present during the Feb 16 production runs and
some occasional unexpected operation (active channel switched) was
observed, mostly on the laptop computer. The instrument should be grounded
when installed.
None observed to date.
90 - 264 volts; 47 - 63 Hz
<100 watts power use
No surge >320 watts
Automatically every scan. Will go off sheet and will calibrate vs. the air gap.
A NIST standard can be placed in line monthly at the user's discretion.
Digital output in real time; pulled into time "bins" for controllers. There is a
protocol to convert raw data to basis weight.
Few: Power fuses and laser
There is a diagnostics table for the customer. Units are returned to Picometrix
for repair.
42
-------
Chapter 7
Performance Summary
The performance of the Picometrix LLC THz sensor was verified based on accuracy, precision,
comparability, and operational factors. The THz sensor was compared to a nuclear gauge for the
measurement of basis weight in a production and offline, laboratory environment. Performance
parameters were determined based on data obtained using two different weights of paper - mid-
weight and heavy-weight.
Accuracy was assessed by comparing the results from the measurement of basis weight using the
TAPPI laboratory gravimetric reference method to the offline THz sensor basis weight
measurements on tear-off samples taken from the end of each roll of paper. The results of the
two basis weight measurements were compared by evaluating the percent error between the
laboratory and offline THz sensor results. The percent error for the THz sensor basis weight, as
compared to the laboratory gravimetric reference method values, ranged from 0.1-2.8% for the
mid-weight rolls and 0.3-2.9% for the heavy-weight rolls. The percent error for the offline THz
sensor readings, based on the averages across all five rolls for a particular weight paper, was
0.02%.
The precision of the THz sensor was assessed on replicate measurements made on cut samples of
heavy weight paper. Triplicate measurements were made on one stack of heavy weight paper.
The standard deviation for these replicate THz sensor measurements was 0.0115 ps with a
coefficient of variation of <0.005%. It was determined that a non-ideal testing method was used
for the THz sensor offline testing which lead to a much higher than expected (-20 times)
imprecision in the THz sensor results. The variability in the paper stack configuration was found
to contribute to the variability in the type of THz sensor measurement undertaken in this
verification test. The vendor felt that this sample position variability was significant and greatly
impacted the ability to draw meaningful conclusions from the static sample laboratory tests.
The comparability of the THz sensor to the nuclear gauge was assessed by evaluating the online
basis weight results against the online nuclear gauge results. Plots of these data showed that the
THz sensor data tracked the nuclear gauge basis weight trends for most rolls across both paper
weights. In four instances, the THz sensor basis weight results diverged at some point within the
production of a roll of paper. The offset in the two data sets at these points was 0.2 Ibs/ream for
most rolls and 1-2 Ibs/ream for one roll where there was a consistent offset between the THz
sensor and nuclear gauge data. In the other three instances, the data divergence was only over
the course of seconds at certain points in the production of a particular roll.
The percent difference between the mean basis weight values for the on-line THz sensor and
nuclear gauge results for approximately 5-12 minute data runs are highlighted in Table 7.1.
43
-------
The average percent difference in the mean values ranged from -0.65% to 0.14%. All but one of
the mean basis weight measurements of the THz sensor and nuclear gauge technologies was
within ±0.20%.
Table 7-1. Comparison of Online Production Mean Percent Difference Values for THz and
Nuclear Sensors Basis Weight
Roll
Number
Mean
Percent
Difference
T-Test
P-Value
Mid-Weight Paper
16026
16027
16028
16035
16037
-0.14%
-0.13%
-0.06 %
-0.04 %
0.03 %
<0.001
<0.001
0.122
0.429
0.133
Roll
Number
Mean
Percent
Difference
T-Test
P-Value
Heavy Weight Paper
23035
23036
23042
23043
23044
-0.65%
0.12%
0.03%
-0.20%
0.05%
<0.001
<0.05
0.567
<0.05
0.210
To evaluate whether the mean percent difference was significantly different from zero, a t-test
was used. Table 7.1 lists the p-values for these t-tests. P-values less than 0.05 indicate a
statistical significant difference from zero. For the mid-weight samples, three of five production
runs had p-values >0.05, indicating no statistically significant difference between the means of
the two. Similar statistical testing for two of the five production runs for the heavy-weight
samples indicated that there was no statistically significant difference between sample means.
The nuclear gauge data were processed to provide a piece-wise linear fit response that was useful
for production control. The THz sensor data were lightly smoothed to mimic this fit to some
degree, but the THz sensor data still retained its more variable measurement value behavior.
Operational factors related to the THz technology were evaluated based on Battelle staff testing
observations and input from Picometrix, LLC. Picometrix states that the THz sensor was
approximately 50 times smaller and lighter than the nuclear gauge enclosure. The Picometrix
LLC THz sensor operates on the principal of EM reflection and which makes system setup, use,
and maintenance easier than for the nuclear gauge. Also, the THz system does not rely on any
radiological sources and thus does not present any special safety concerns or any special
procurement, use, or disposal concerns (e.g., extra disposal costs or potential exposure hazards).
The laser must be replaced every 5-10 years; the laser window must be cleaned weekly, but no
factory or operator actions are required after the initial vendor setup. The instrument was
automatically calibrated during each scan; a NIST standard can be placed in-line monthly at the
user's discretion. The THz system has diagnostics to indicate to the user if and when
recalibration is required. The instrument should be adequately grounded when installed, as some
occasional interruptions in operation were observed due to static electricity. The instrument
requires 90-246 volt power supply and uses <100 watts of power. Data were output digitally in
real-time. The unit cost is $220,000.
44
-------
Chapter 8
References
1. Quality Assurance Project Plan for Verification of Picometrix, LLC T-Ray 4000* Time-
Domain Terahertz System, Battelle, Columbus, Ohio, February 23, 2011.
2. Quality Management Plan for the ETV Advanced Monitoring Systems Center, Version 7.0,
U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio,
November 2008.
3. Appleton Standard Test Method, "Basis Weight - Lab Determination of Coated and
Uncoated Paper," Spec. No. 10001.00, Revision 1, July 1993.
45
------- |