EPA/600/R-15/011
February 2011
Environmental Technology
Verification Report
Magee Scientific Model AE33 Aethalometer
Prepared by
Battelle
The Business of Innovation
Under a cooperative agreement with
CrTr\ U.S. Environmental Protection Agency
-------
Environmental Technology Verification
Report
ETV Advanced Monitoring Systems Center
Magee Scientific Model AE33 Aethalometer
By
Kenneth Cowen
Thomas Kelly
Amy Dindal
Battelle
Columbus, Ohio 43201
-------
Notice
The U.S. Environmental Protection Agency, through its Office of Research and Development, partially
funded and collaborated in the research described herein. This report has been subjected to the Agency's
peer and administrative review. Any opinions expressed in this report are those of the author(s) and do
not necessarily reflect the views of the Agency, therefore, no official endorsement should be inferred. Any
mention of trade names or commercial products does not constitute endorsement or recommendation for
use.
-------
Foreword
The EPA is charged by Congress with protecting the nation's air, water, and land resources.
Under a mandate of national environmental laws, the Agency strives to formulate and implement
actions leading to a compatible balance between human activities and the ability of natural
systems to support and nurture life. To meet this mandate, the EPA's Office of Research and
Development provides data and science support that can be used to solve environmental
problems and to build the scientific knowledge base needed to manage our ecological resources
wisely, to understand how pollutants affect our health, and to prevent or reduce environmental
risks.
The Environmental Technology Verification (ETV) Program has been established by the EPA to
verify the performance characteristics of innovative environmental technology across all media
and to report this objective information to permitters, buyers, and users of the technology, thus
substantially accelerating the entrance of new environmental technologies into the marketplace.
Verification organizations oversee and report verification activities based on testing and quality
assurance protocols developed with input from major stakeholders and customer groups
associated with the technology area. ETV consists of six environmental technology centers.
Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/.
Effective verifications of monitoring technologies are needed to assess environmental quality
and to supply cost and performance data to select the most appropriate technology for that
assessment. Under a cooperative agreement, Battelle has received EPA funding to plan,
coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air,
Water, and Soil" and report the results to the community at large. Information concerning this
specific environmental technology area can be found on the Internet at
http://www.epa.gov/etv/centers/centerl.html.
-------
Acknowledgments
The authors wish to acknowledge the support of all those who helped plan and conduct the
verification test, analyze the data, and prepare this report. Specifically the authors thank Andrea
Polidori of the South Coast Air Quality Management District, Joann Rice of EPA Office of Air
Quality Planning and Standards, and Zachary Willenberg, Battelle Quality Assurance Officer, for
their review of this verification report.
in
-------
Contents
Notice i
Foreword ii
Acknowledgments iii
List of Figures v
List of Tables vi
List of Abbreviations vii
Chapter 1 Background 1
Chapter 2 Technology Description 2
Chapters Test Design and Procedures 4
3.1 Introduction 4
3.2 Test Procedures 6
3.3 Field Site 7
Chapter 4 Quality Assurance/Quality Control 10
4.1 Amendments/Deviations 10
4.2 Reference Method 10
4.2.1 Reference Method Sampling 11
4.2.2 Reference Method Analysis 16
4.3 Audits 20
4.3.1 Performance Evaluation Audit 20
4.3.2 Technical Systems Audit 22
4.3.3 Data Quality Audit 23
4.4 QA/QC Reporting 23
4.5 Data Review 23
Chapters Statistical Methods 24
5.1 Comparability 24
5.2 Correlation 25
5.3 Precision 25
5.4 Data Completeness 25
5.5 Operational Factors 25
Chapter 6 Test Results 26
6.1 Comparability 27
6.1.1 Regression Analysis 30
6.1.2 Relative Percent Difference Analysis 32
IV
-------
6.2 Correlation 32
6.4 Data Completeness 36
6.5 Operational Factors 36
6.5.1 Maintenance 36
6.5.2 Consumables/Waste Generation 37
6.5.3 Ease of Use 37
Chapter 7 Performance Summary 38
List of Figures
Figure 2-1. Magee Scientific Aethalometer AE-33 3
Figure 3-1. Illustration of measurements of carbonaceous paniculate matter 5
Figure 3-2. Aerial photograph of test site and surrounding area 8
Figure 3-3. RAAS reference sampler installed at sampling site 8
Figure 3-4. BGI PQ200 reference samplers installed at sampling site 9
Figure 4-1. Duplicate Reference Method EC results from TOR analysis 12
Figure 4-2. Duplicate Reference Method EC results from TOT analysis 13
Figure 4-3. Differences between the duplicate reference method EC results from
TOR analysis 14
Figure 4-4. Differences between the duplicate reference method EC results from
TOR analysis 14
Figure 4-5. Regression of reference method results from TOR analysis by sampler type 15
Figure 4-6. Scatter plot of reference method results from TOT analysis by sampler type 15
Figure 6-1. Measured hourly average BC concentration from the duplicate Model AE33
Aethalometers during testing 26
Figure 6-2. Calculated differences between the hourly average BC concentration
from the duplicate Model AE33 Aethalometers during testing (Difference = SN 89 -
SN90) 27
Figure 6-3. Comparison of the 12-hour BC averages from the Model AE33
Aethalometers and the mean reference method TOR EC concentrations 28
Figure 6-4. Comparison of the 12-hour BC averages from the Model AE33
Aethalometers and the mean reference method TOT EC 29
Figure 6-4. Linear regression of 12-hour Aethalometer BC averages against
mean reference method TOREC results 30
Figure 6-5. Linear regression of 12-hour Aethalometer BC averages against
mean reference method TOT EC results 31
Figure 6-6. Linear regression of Aethalometer 1-hour averages (linear scale) 34
Figure 6-7. Linear regression of Aethalometer 1-hour averages (logarithmic scale) 34
Figure 6-8. Linear regression of the Aethalometer 12-hour averages (linear scale) 35
Figure 6-9. Linear regression of the Aethalometer 12-hour averages (logarithmic scale) 35
-------
List of Tables
Table 4-1. Summary of Reference Method Field Blank Results 16
Table 4-2. Auto-Calibration Results of DRIEC/OC Analyzer 17
Table 4-3. Full Calibration Results used for DRI Analyzers 18
Table 4-4. Results of Periodic Calibration Checks 18
Table 4-5. Temperature Calibration Results 19
Table 4-6. System Blank Results 19
Table 4-7. Laboratory Blank Results 20
Table 4-8. Replicate EC Analysis Results 21
Table 4-9. Summary of Flow Rate PE Audit 22
Table 4-10. Summary of Temperature and Pressure PE Audit 22
Table 6-1. Summary Regression Results of the Aethalometers and the Reference Method 31
Table 6-2. Summary of Regression Results of the Aethalometers and the Reference
Method Excluding Apparent Outlier Data Point 32
Table 6-3. Summary of Relative Percent Difference between the Aethalometers and
the TOR Reference Method Results 32
Table 6-4. Summary of Correlation Results of the Aethalometers and the Reference Method .. 33
Table 6-5. Summary of Calculated Unit-to-Unit Precision Results of the Aethalometers 33
Table 6-6. Summary of Unit-to-Unit Regression Results of the Aethalometers 35
Table 6-7. Summary of data completeness for the Aethalometers 36
Table 6-8. Summary of Maintenance Performed on Aethalometers 37
Table 7-1. Summary of Verification Test Results for the Model AE33 Aethalometer 38
VI
-------
List of Abbreviations
AMS Advanced Monitoring Systems Center
BC black carbon
CH4 methane
csv comma separated variable
CV coeffi ci ent of vari ati on
DL detection limit
DQO data quality objective
DRI Desert Research Institute
EC elemental carbon
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
He helium
IMPROVE Interagency Monitoring of PROtected Visual Environments
KHP potassium hydrogen phthalate
LAC light absorbing carbon
LPM liters per minute
Hg/m3 micrograms per cubic meter
NIST National Institute of Science and Technology
OC organic carbon
PE performance evaluation
ppm parts per million
QA quality assurance
QC quality control
QMP quality management plan
r2 coefficient of determination
RPD relative percent difference
TOR thermal optical reflectance
TOT thermal optical transmittance
TSA technical systems audit
Vll
-------
Chapter 1
Background
The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative environmental
technologies through performance verification and dissemination of information. The goal of
the ETV Program is to further environmental protection by accelerating the acceptance and
use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing
high-quality, peer-reviewed data on technology performance to those involved in the design,
distribution, financing, permitting, purchase, and use of environmental technologies.
ETV works in partnership with recognized testing organizations; with stakeholder groups
consisting of buyers, vendor organizations, and permitters; and with the full participation of
individual technology developers. The program evaluates the performance of innovative
technologies by developing test plans that are responsive to the needs of stakeholders,
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and
preparing peer-reviewed reports. All evaluations are conducted in accordance with rigorous
quality assurance and quality control (QA/QC) protocols to ensure that data of known and
adequate quality are generated and that the results are defensible.
The EPA's National Risk Management Research Laboratory and its verification organization
partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The
AMS Center recently evaluated the performance of the Magee Scientific Model AE33
Aethalometer at an ambient air monitoring site in Columbus, Ohio. Black carbon (BC)
monitors were identified as a priority technology category for verification through the AMS
Center stakeholder process.
-------
Chapter 2
Technology Description
The objective of the ETV AMS Center is to verify the performance characteristics of
environmental monitoring technologies for air, water, and soil. This report provides results
for the verification testing of the Magee Scientific Model AE33 Aethalometer. The following
is a description of the Model AE33 Aethalometer, based on information provided by the
vendor. The information provided below was not verified in this test. Details of the
installation and operation of the Aethalometers during the test period are included in Section
3.2 of this report.
The Aethalometer™ is used for the real-time measurement of optically-absorbing 'Black' or
'Elemental' carbon aerosol particles. The name "Aethalometer" is derived from the classical
Greek verb 'aethaloun' (asGoAouv), meaning 'to blacken with soot'. It was conceptualized
in 1979, commercialized in 1986, and has been under continuous development since that
date. The Aethalometer Model AE31 was verified by the ETV Program in 2001. The Model
AE33 Aethalometer was released in 2012, and incorporates many scientific and technical
improvements relative to earlier models.
The Aethalometer uses a continuous filtration and optical measurement method to provide a
continuous readout of real-time data for the concentration of 'BC', which is fundamentally
defined by 'blackness', an optical measurement. The optical analysis for BC is designed to
be consistent and reproducible, and may be validated by the use of Neutral Density optical
standards.
Aethalometers provide fully automatic, unattended operation. The sample is collected and
analyzed as a spot on a roll of filter tape: depending on location, one roll of tape may last for
several months. No other consumables are required. The instrument requires no calibration
other than periodic checks of the air flow sensor response.
The AE-33 performs optical analysis at seven discrete wavelengths from 370 nm to 950 nm.
These data can be interpreted to provide an indication of source apportionment, due to the
different spectral characteristics of diesel particulates versus biomass-burning smoke.1
In recent years, it became apparent that under certain conditions, at certain locations, filter-
based optical measurement techniques can be influenced by a saturation effect (also known
as the "loading effect") of variable magnitude. This effect, when present, can change the
-------
reported data by up to a factor of 2 or more, depending on the nature of the aerosol and the
settings of the instrument. At other locations, or at the same location under conditions of
different aerosol climatology, the effect may be reduced or completely absent. The fact that
the "loading effect" is variable and clearly dependent on some attribute of the aerosol
indicates that it is a combination of some aspect of the instrumental method, together with an
actual chemical or microphysical aspect of the aerosol. However, the "loading effect" is
always found to be linear with respect to the light attenuation measured on the filter spot.
The Model AE33 Aethalometer corrects for the "loading effect" by collecting two aerosol
spots in parallel, but at rates of accumulation that differ by a factor of two. Mathematical
combination of the data from the two parallel analyses permits reconstruction of the "ideal"
result, together with a report of the "loading compensation parameter" which may be
informative of aerosol properties in its own right.
Figure 2-1. Magee Scientific Aethalometer AE-33
-------
Chapter 3
Test Design and Procedures
3.1 Introduction
The ETV Program's AMS Center conducts third-party performance testing of commercially
available technologies that detect or monitor natural species or contaminants in air, water,
and soil. Stakeholder committees of buyers and users of such technologies recommend
technology categories, and technologies within those categories, as priorities for testing.
Among the technology categories recommended for testing are "black carbon" monitors.
Because of the nature of BC, this technology category includes monitors for both BC and
EC. Two stakeholders were selected to serve as peer reviewers for the quality assurance
project plan (QAPP)2 and this verification report. The responsibilities of verification test
stakeholders/peer reviewers included:
• Participate in technical panel discussions (when available) to provide input to
the test design;
Review and provide input to the QAPP; and
• Review and provide input to the verification report/verification statement.
The QAPP and this verification report were reviewed by experts in the fields related to black
carbon monitors. The following experts provided peer review:
• Andrea Polidori, South Coast Air Quality Management District
• Joann Rice, EPA.
The purpose of this verification test was to generate performance data on BC monitors so
organizations and users interested in installing and operating these systems can make
informed decisions about their use. Black carbon is a term that is commonly used to describe
strongly light absorbing carbon (LAC), which is thought to play a significant role in global
climate change through direct absorption of light, interaction with clouds, and by reducing
the reflectivity of snow and ice. BC is formed from the incomplete combustion of fossil
fuels, biofuels, and biomass and can be emitted from both anthropogenic and natural sources.
It is a primary component of soot and has been linked to adverse health effects and visibility
reduction. Consequently, there is a great deal of interest in monitoring BC in the atmosphere.
However, differences in measurement techniques result in measurements that are
operationally defined and characterize the paniculate matter based on either its light
absorbing properties (leading to determination of BC) or its refractory properties (leading to
-------
determination of EC), as illustrated in Figure 3-1. In this figure, the use of the subscript a
denotes that the measurements are technique specific and result in estimations of BC or EC
that are "apparent" based on the technique being used. The methods used to determine EC
are termed thermal-optical in Figure 3-1 because they involve conversion of particulate
carbon to gaseous form under varying temperatures and controlled atmospheres while the
parti culate sample is monitored by either transmission or reflection of light.
This verification test was conducted according to procedures specified in the peer-reviewed
ETV Quality Assurance Project Plan (QAPP)for Verification of Black Carbon Monitors.2
Deviations from this QAPP are described in Section 4.1 of this report.
Light-Absorption Classification
More
light-absorbing
Light-
absorbing
carbon
(LAC)
Black
carbon
Brown *
carbon
(BrC)
Less
light-absorbing
Thermal-Optical Classification
More refractory
Elemental
carbon
Organic
carbon
(OCJ
Less refractory
* Measurement technique-specific split point
Figure 3-1. Illustration of measurements of carbonaceous particulate matter.
(Source: U.S. EPA)3
The performance of the Model AE33 Aethalometer was verified by evaluating the following
parameters:
• Comparability with collocated reference method results
• Precision between duplicate units
• Data completeness
• Reliability
• Operational factors such as ease of use, maintenance and data output needs, power
and other consumables use, and operational costs.
-------
3.2 Test Procedures
The test was conducted at the Battelle Columbus Operations Special Support Site (BCS3)
located at 2555 International St., in Columbus, OH. For this test, duplicate AE33
Aethalometers continuously sampled ambient air for approximately 33 days, during which
period filter samples were collected on thirty days to use as a basis of comparison for the
analyzers being tested. Specifically, during each test period duplicate integrated filter
samples were collected over successive 12-hour periods each day (except as noted below)
using commercially available air sampling equipment. The duplicate reference samples were
collected from 7:00 am to approximately 6:50 pm and from 7:00 pm to approximately 6:50
am daily. From April 5 through April 25, a single four-channel Anderson Model RAAS-400
speciation sampler was used to collect the duplicate reference filter samples using separate
channels of the sampler. Samples were not collected during one of the planned 12-hour
sampling periods (beginning on the evening of April 16) because of adverse weather
conditions. On April 25, a failure in the electronics of the RAAS sampler rendered it
inoperable and the lack of replacement parts resulted in the RAAS being inoperable for the
remainder of the test period. Duplicate BGI Model PQ200 samplers were immediately
available and were used in replacement of the RAAS for the collection of the reference
samples from April 26 to May 7.
The reference samples were collected on pre-cleaned quartz fiber filters at a nominal flow
rate of 16.7 LPM with both the RAAS and BGI samplers. The change in samplers was not
expected to result in any differences in the measured EC concentrations since the samples
were collected at the sample flow rate and with similar PM2 5 size selective inlets. Actual
differences that may have been observed are likely the result of differences in the calibrated
flow rates of the samplers. Over the 3 3-day field period, a total of 118 filter samples were
successfully collected (i.e., duplicate samples during 59 of the 60 12-hour sampling periods).
The reference samples were analyzed for EC by Desert Research Institute (DRI) using the
DRI Model 2001 EC-OC analyzers implementing the Interagency Monitoring of PROtected
Visual Environments (IMPROVE) thermal/optical reflectance (TOR) method, which
monitors the filter sample by means of optical reflectance.4 DRI also reported EC results
from the IMPROVE method with the filter monitored by thermal/optical transmission
(TOT).4 Results from the Aethalometers were compared to these filter sample results to
assess the comparability of the Aethalometer results to the filter sample results. It should be
noted that the Aethalometer measures BC using light absorption techniques whereas the
TOR/TOT methods measure EC by thermal optical techniques in which CO2 is generated
from the oxidation of carbonaceous species which have been thermally liberated from the
collected sample and the optical transmittance or reflectance of the filter is used to correct for
residual carbon on the filter.
The precision of the Model AE33 Aethalometer was determined from comparisons of paired
data from the duplicate units. Other performance parameters such as data completeness,
maintenance requirements, ease of use, and operational costs were assessed from
observations by the Battelle field testing staff. This test was not intended to simulate long-
term (e.g., multi-year) performance of BC monitors. As such, performance and maintenance
issues associated with long-term use of the Model AE33 Aethalometer are not addressed in
this report.
-------
Note that in this report the filter samples will be referred to as "reference samples."
However, it should be noted that the IMPROVE method is not a true Reference Method in
that it is not recognized as an absolute standard. Nonetheless, it is used within the
IMPROVE network as the standard method for EC analysis. Thus the method was used in
this test as an analytical technique used for comparison to the BC monitors. Other
thermal/optical reference methods such as the NIOSH 5040 method may result in different
results.
For the verification test, duplicate Model AE33 Aethalometers were installed inside an
environmentally controlled instrument trailer at the monitoring site. The duplicate Model
AE33 Aethalometers were installed by the vendor on March 27-28 and operated continuously
at the site throughout the verification test, with the exception of several brief periods during
which maintenance activities were performed.
Each Aethalometer drew sample air at 5 liters per minute (LPM) through approximately 3
meters of conductive (i.e., static-free) tubing connected to a PM2.5 sharp cut cyclone inlet.
Other than an initial flow rate check performed by the vendor, on March 27, no other quality
control activities were performed on the Aethalometers for the duration of the verification
test. The two inlet cyclones for the Aethalometers were positioned at one corner of the
platform, at the same height as the reference sample inlet and at least one meter from each
reference sampler. The RAAS-400 and the duplicate PQ200 samplers were installed on the
platform such that each inlet was more than one meter from the other inlets. Battelle staff
were trained on the operation of the analyzers and performed the maintenance on both units
during the verification test. These activities were documented and are reported in Section 6.5
of this report.
3.3 Field Site
The test was conducted at the BCS3 facility located at 2555 International St., in Columbus,
OH. Figure 3-2 shows an aerial photograph of the test site (red marker "A") and the
surrounding area. The test site is located approximately 1A mile north of a rail yard and in the
vicinity of multiple industrial and shipping facilities which result in frequent truck traffic past
the site. The site also receives regionally transported air pollution due to its location on the
western side of the Columbus metropolitan area. An environmentally controlled mobile
laboratory was installed at the site to serve as a shelter for the Model AE33 Aethalometers
and as work space for the testing staff. Figures 3-3 and 3-4 show the sampling trailer with
RAAS sampler and BGI samplers, respectively, installed on a platform next to the trailer.
-------
Figure 3-2. Aerial photograph of test site and surrounding area.
Figure 3-3. RAAS reference sampler installed at sampling site.
-------
.
Figure 3-4. BGI PQ200 reference samplers installed at sampling site.
-------
Chapter 4
Quality Assurance/Quality Control
QA/QC procedures and all verification testing were performed in accordance with test/QA
plan for this verification test1 and the quality management plan (QMP) for the AMS Center6
except where noted below. QA/QC procedures and results for the reference method are
described below. Other than an initial flow check during installation of the Aethalometers no
additional QA/QC activities were performed on the Aethalometers. Maintenance activities
performed on the Aethalometers are included in Section 6.5.1.
4.1 Amendments/Deviations
Two deviations to the test/QA plan were prepared, approved, and retained in the test
documentation. The deviations established the following modification to the test/QA plan
and the test procedures:
• The RAAS speciation sampler was replaced with duplicate BGIPQ200 samplers
approximately halfway through the sampling period because of a failure in the
RAAS sampler. This change did not adversely affect the data quality since the new
samplers functionally performed the same as the RAAS samplers. See Section 4.2.1
for a comparison of the results from the two sampler types.
• Routine flow checks were not performed as specified in the test plan for a portion of
the test period. This deviation is not expected to adversely affect the results since
subsequent leak checks met the acceptance criteria and no systemic bias was observed
in the reference results.
4.2 Reference Method
The following sections describe the QA/QC procedures employed in the collection and
analysis of reference samples. Only the results for the EC analyses are presented since OC
results are not used to evaluate the performance of the Aethalometers.
10
-------
4.2.1 Reference Method Sampling
This verification test included a comparison of Aethalometer results to those of the reference
measurements described in Section 3.2. Figures 4-1 and 4-2, respectively, show the results of
the TOR and TOT measurements of the duplicate reference samples collected during the
testing period. In these figures, one of the two channels used in the RAAS for the collection
of the reference samples was designated as the "primary sample" and the other channel was
designated as the "collocated sample" to show potential biases between the two channels.
Similarly, one of the BGI samplers was designated as the "primary sample" and the other as
the "collocated sample." Figures 4-3 and 4-4 show the corresponding differences between the
primary and collocated reference sample measurements and indicate that there is no clear
systematic bias observed in either the TOR or TOT results for either the RAAS sampler or
the duplicate BGI samplers.
During the verification test the mean of the EC measurements for the duplicate reference
samples ranged from <1.5 to 32.9 micrograms per filter (|j,g/filter) which corresponds to
airborne concentrations ranging from <0.13 to 2.7 micrograms per cubic meter (|j,g/m3) based
on the sample volumes of-12 m3. According to DRI's documentation, at concentrations of
greater than 10 times the method detection limit (10 x MDL -1.5 |j,g/filter) the expected
precision between duplicate samples is - 10%.3 In general, the majority of the reference
samples had EC concentrations that were below 10 times the MDL, consequently the percent
difference between the duplicate samples was typically greater than 10%. Reference results
that were reported as less than the MDL were assigned a value of l/2 MDL for this report.
Over all results with both reference sampler types, the calculated percent differences (i.e., the
difference between the two duplicate results divided by their mean) ranged from -98% to
109%, with an average of 3%. The EC concentrations measured from the filters collected
with the RAAS sampler ranged from <0.13 to 1.1 ng/m3, with a mean concentration of 0.35
± 0.28 ng/m3. The average percent difference between the duplicate samples collected with
the RAAS was 21% ± 71%. The EC concentrations measured from the filters collected with
the BGI PQ200 samplers ranged from <0.13 to 2.7 |J,g/m3, with a mean concentration of 0.64
± 0.62 ng/m3. The average percent difference between the duplicate samples collected with
the BGI samplers was 1% ± 50%.
Figure 4-5 shows a scatter plot of the TOR results for the reference method samples
indicating in which sampler type used to collect the filters (e.g., RAAS and BGI PQ200).
Figure 4-6 presents a similar plot for the TOT analyses. These figures show that the TOT
results exhibit a slope closer to 1.0, an intercept closer to zero, and a higher r2 value, relative
to the TOR results.
During testing a total of 12 reference method field blank samples were collected,
representing 10% of the total reference method samples. The field blanks were installed in
the filter cassettes and loaded into the reference method samplers without drawing air
through the filters. Table 4-1 presents a summary of the field blank results including the
results for the EC measurements for both the TOR and TOT analyses by the DRI Model 2001
analyzer. Table 4-1 shows that in nearly all cases no detectable EC was found on the
reference method blank filters.
11
-------
CO
"B)
C
O
I2
c
0)
o
c
o
o
o
LLI
O 1 -
0)
o
c
0)
a:
0
• Primary Sample-RAAS
Q Primary Sample-BGI
• Collocated Sample-RAAS
H Collocated Sample-BGI
II
ul
CO
CO
CNJ
CO
o5
CO
CD
CNI
CO
CO
Date
Figure 4-1. Duplicate Reference Method EC results from TOR analysis.
12
-------
"B)
c
o
:*= 2
•4-1
C
0)
o
c
o
o
O
LU
•o
0)
o
c
0)
0)
0
• Primary Sample-RAAS
0 Primary Sample-BGI
• Collocated Sample-RAAS
H Collocated Sample-BGI
liu
CO
o5
CO
CD
CO
CO
Date
Figure 4-2. Duplicate Reference Method EC results from TOT analysis.
13
-------
0.5
^^
-§ 0.3 -
1
0
is
i 0.1 -
8
0
g 4/e
c -0.1 -
8
0)
0)
t _03 _
_n R -
JL
•i.l
ri* ' iii
nsl
r r r f i i T
4/12/13
• Difference-RAAS
• Difference-BGI
Jj^J
J19/T3 I 4/26/1 1
Date
Figure 4-3. Differences between the duplicate reference method EC results from TOR analysis.
0.5
o
+J
ns
O
O
0.3 -
0.1 -
-0.1 -
5
-0.5
, 1
l/'ll
• Difference-RAAS
|
. ullil
1
4/12/13
• Difference-BGI
J IF L I III III I
1 f ' ' IFF rii ^ir
f19/13 1 4/26/131 IM/13 |
Date
Figure 4-4. Differences between the duplicate reference method EC results from TOT analysis.
14
-------
00
_o
a.
E
ns
W
-------
Additionally, flow rate checks of the reference samplers were performed at least every three
days during the sampling period to ensure that the reference samplers were operating within
5% of their nominal flow rate.
Table 4-1. Summary of Reference Method Field Blank Results
Filter ID
BTOQ001
BTOQ002
BTOQ011
BTOQ055
BTOQ072
BTOQ073
BTOQ074
BTOQ075
BTOQ090
BTOQ097
BTOQ109
BTOQ110
EC (ng/filter)
TOR
0.00
0.00
0.05
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.10
TOT
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
4.2.2 Reference Method Analysis
Routine calibrations of the DRI Model 2001 carbon analyzers used to analyze the reference
samples were performed at the beginning and end of each day by injecting known volumes of
either CH4 or CO2 with nominally the same amount of carbon (approximately 21.4 |j,g) and
comparing the resulting OC3 and EC1 measurements4. The acceptable level is ±5%
difference between peaks injected during the OC3 and EC1 temperature step. Table 4-2
presents a summary of these routine calibrations for each of the carbon analyzers (identified
as uniquely numbered CA units below) and the calculated percent differences between
carbon measurements from the OC3 and EC1 measurements for each calibration.
Exceedences of the acceptance criterion require recalibration of the analyzer.
16
-------
Table 4-2. Auto-Calibration Results of DRIEC/OC Analyzer.
Date
4/23/2013
5/17/2013
5/18/2013
5/20/2013
5/21/2013
4/23/2013
5/17/2013
5/18/2013
5/20/2013
5/21/2013
4/23/2013
5/17/2013
5/18/2013
5/20/2013
5/21/2013
OC3 (jig)
CA#6
21.80
19.46
CA#7
20.90
22.75
CA#8
22.35
20.10
20.31
20.27
CA#9
20.22
22.33
22.37
22.29
22.24
CA#10
21.67
21.15
CA#11
21.08
21.89
21.87
21.85
CA#12
21.06
21.10
21.17
21.01
21.00
CA#13
22.59
CA#16
17.67
22.50
22.37
22.39
22.41
EC1 (ng)
CA#6
21.11
20.10
CA#7
20.53
22.31
CA#8
22.02
20.26
20.09
20.25
CA#9
20.78
22.04
22.00
21.95
22.00
CA#10
21.29
21.70
CA#11
20.91
21.27
21.17
21.24
CA#12
20.94
21.00
20.82
20.94
20.92
CA#13
21.40
CA#16
19.48
21.43
21.40
21.46
21.52
%Diff
CA#6
3.3%
-3.2%
CA#7
1.8%
2.0%
CA#8
1.5%
-0.8%
1.1%
0.1%
CA#9
-2.7%
1.3%
1.7%
1.5%
1.1%
CA#10
1.8%
-2.5%
CA#11
0.8%
2.9%
3.3%
2.9%
CA#12
0.6%
0.5%
1.7%
0.3%
0.4%
CA#13
5.6%
CA#16
-9.3%
5.0%
4.6%
4.3%
4.1%
CA#16 was taken offline on 4/23/2013 for FID and laser issues.
fMissing values indicate the instruments were offline during the day.
Full instrument calibrations are performed semiannually or after major maintenance or
repairs and are used to establish the calibration slope used in converting CC>2 detector counts
to ug of carbon. Four types of standards are used for full instrument calibration: 5% nominal
methane (CFLi) in helium (He), 5% nominal CC>2 in He, potassium hydrogen phthalate (KHP),
and sucrose. Instrument calibration involves spiking pre-fired quartz punches at four different
levels with 5.0 to 20.0 microliters (ul) of the 1,800 parts per million (ppm) KHP and sucrose
solutions and injecting CH4 and CC>2 gases at four levels from 200 to 1,000 uL of the. The
calibration slopes derived from the two gases and the KHP- and sucrose-spiked filter punches
are averaged together to yield a single calibration slope for a given analyzer. This slope
represents the response of the entire analyzer to generic carbon compounds and includes the
efficiencies of the oxidation and methanator zones and the sensitivity of the FID. Table 4-3
presents a summary of the recent full calibrations of the analyzers used to analyze the
reference samples. Note that the treatment of the calibration data ensures that the data passes
through the origin, so no intercept is presented. The calculated slopes are compared to
previous calibration results and should be with 10% of previous calibrations if no major
changes to the instrument have been made. If the differences in the slope exceed 10%, the
calibration must be repeated to verify the results.
17
-------
Table 4-3. Full Calibration Results used for DRI Analyzers
Analyzer
CA#6
CA#7
CA#7
CA#7
CA#8
CA#8
CA#9
CA#10
CA#10
CA#10
CA#11
CA#12
CA#13
CA#16
Date
4/3/2013
5/7/2013
4/26/2013
10/11/2012
5/13/2013
12/7/2012
4/18/2013
5/21/2013
5/4/2013
3/29/2013
3/14/2013
4/1/2013
11/19/2012
3/10/2013
Slope
21.482
19.436
22.000
20.513
20.094
20.483
22.040
21.457
23.228
20.502
21.320
20.904
20.818
22.685
r2
0.991
0.976
0.996
0.986
0.981
0.987
0.988
0.982
0.996
0.991
0.987
0.989
0.985
0.987
The instrument calibration was verified several times a week using sucrose and KHP
standards near the midpoint of the calibration curve (18 ugC). Table 4-4 presents a summary
of the calibration checks performed. In all cases the agreement between the measured and
standard concentration was within the ±1 ugC acceptance criterion.
Table 4-4. Results of Periodic Calibration Checks
Date
4/23/2013
5/17/2013
5/20/2013
5/21/2013
CA#6 CA#8
18.718 17.729
17.433
17.505
17.739
CA#9
18.441
17.783
17.447
17.523
CA#10
17.450
17.223
CA#11
18.119
18.149
18.439
17.332
CA#12
17.876
17.688
17.692
17.402
CA#13 CA#16
18.071 17.761
17.749 18.104
18.595
18.147
fMissing values indicate the instruments were offline during the day.
Temperature calibrations are performed at least semiannually on all instruments to verify
that the sample temperature is as accurate as possible. Quick-drying temperature-indicating
liquids of different melting points, Tempilaq°G (Tempil, Inc., South Plainfield, NJ, USA),
were used as temperature indicators. Temperature indicators of 121, 184, 253, 510, 704, and
816 °C were chosen for the IMPROVE_A protocol temperature calibration. The accuracy of
Tempilaq°G is certified within 1% of its designated temperature and is traceable to the
National Institute of Standards and Technology (NIST). Table 4-5 shows the results of the
most recent temperature calibrations of the analyzers used to analyze the reference samples.
In all cases the linear relationship between the thermocouple and standard Tempilaq°G values
met the r2 > 0.99 acceptance criterion.
18
-------
Table 4-5. Temperature Calibration Results
CA#6
CA#6
CA#7
CA#7
CA#8
CA#8
CA#9
CA#9
CA#10
CA#11
CA#12
CA#13
CA#13
CA#16
Date
11/26/2012
5/16/2013
4/29/2013
6/3/2013
12/7/2012
5/13/2013
4/5/2013
4/22/2013
2/19/2013
3/15/2013
2/25/2013
11/19/2012
5/21/2013
3/11/2013
Slope
1.016
1.027
1.041
1.01
1.018
1.037
1.012
1.026
1.015
1.019
1.02
1.012
0.995
1.012
Intercept
8.3
19.0
8.9
3.0
5.8
7.6
8.7
1.8
8.0
11.7
7.1
9.7
12.7
11.2
r2
0.999
0.999
0.991
1.000
1.000
0.997
0.999
0.999
0.999
0.999
1.000
0.999
0.999
1.000
System blanks were performed once a week without filter punches in the analyzer to
determine the instrument baseline. Calculated carbon concentrations from the system blank
should not be more than 0.2 ug carbon. Table 4-6 presents a summary of the EC system
blank results for the analyzers used to analyze the reference samples. Table 4-6 shows that
the great majority of the EC system blanks showed no detectable carbon, and all EC blanks
easily met the 0.2 ugC acceptance criterion.
Table 4-6. System Blank Results
EC (ug)
4/28/2013
5/5/2013
5/12/2013
5/19/2013
5/26/2013
CA#6
0.000
0.000
0.002
0.004
CA#8
0.000
0.000
0.000
0.000
CA#9
0.003
0.000
0.000
0.000
CA#10
0.068
0.000
CA#11
0.000
0.000
0.000
0.000
0.000
CA#12
0.000
0.000
0.000
0.000
CA#13
0.000
0.000
0.000
CA#16
0.000
0.000
0.000
System blanks on 5/12/2013 were high due to instruments being idle for the weekend and subsequent
laboratory blank checks indicated normal conditions.
Laboratory blanks were performed daily to check for system contamination and evaluate
laser response. If total carbon exceeded 0.2 ugC, values were voided and additional
laboratory blanks were run after performing the oven bake procedure until the system is clean
(i.e., OC < 0.2 ug C/cm2 and no EC). Analyzers exceeding the limit for laser drift,
reflectance, transmittance, total carbon, and calibration peak area after three laboratory blank
19
-------
runs must be taken offline for maintenance. Table 4-7 presents the results of the laboratory
blanks for the analyzers used to analyze the reference samples. One of the analyzers (CA#7)
repeatedly failed the blank check and was taken offline on 5/18/2013 for maintenance.
Table 4-7. Laboratory Blank Results
EC (ng)
4/23/13
5/17/13
5/18/13
5/20/13
5/21/13
CA#6
0.000
0.063
0.000
CA#7
0.017
0.005
0.000
CA#8
0.000
0.000
0.000
0.000
0.011
CA#9
0.
0.
0.
0.
0.
.000
.000
.000
.000
.000
CA#10
0.000
0.000
0.000
0.012
CA#11
0.
0.
0.
0.
.000
.000
.000
.000
CA#12
0.000
0.000
0.000
0.003
0.021
CA#13 CA#16
0.000 0
0.000 0
0
0
0
.000
.000
.000
.000
.005
CA#7 was taken offline for 5/18/2013.
Replicates of analyzed samples were performed at the rate of one per group often samples.
The replicate was selected randomly and run immediately after each group often was
completed. The random analyzer for the replicate was identified using a chart created in
Microsoft Excel using the random number generator, which results in replicate analysis on
the same and different analyzers. The ug/cm2 values for EC were compared with the original
run for both the TOR and TOT analysis. Precision was determined from replicate
measurements as the average fractional difference between original and replicate analysis
concentrations. Concentration uncertainty is the fractional precision times sample
concentration. If sample concentration times fractional precision is zero, then the detection
limit is used as concentration uncertainty. Table 4-8 shows the results of the replicate
analyses. The results of the replicate analyses ranged from 0% to 52% for the TOR analysis
and from 0% to 36% for the TOT analysis. In general, the percent difference exceeded the
goal of 15% for the majority of the duplicate analyses indicating a lower degree of data
quality than desired.
4.3 Audits
Three types of audits were performed during the verification test: a performance evaluation
(PE) audit of the reference method sampling, a technical systems audit (TSA) of the
verification test performance, and a data quality audit. Audit procedures are described further
below.
4.3.1 Performance Evaluation Audit
A PE audit of the RAAS reference method sampler was performed by measuring the sample
flow rate through the two channels used for collection of the reference samples. The flow
rate through each channel was measured using a NIST-traceable flow transfer standard
(BIOS DryCal, Serial No. 103777). After installation of the BGI PQ200 samplers, the flow
rates of those samplers were audited using a BGI DeltaCal calibrator (Serial No. 001255).
The results of those checks are summarized in Table 4-9, and indicate that the sampler flow
rates were well within the target ±5% tolerance of the nominal 16.7 L/min flow rate.
20
-------
Table 4-8. Replicate EC Analysis
Results
Runl
Filter ID
BTOQ037
BTOQ041
BTOQ107
BTOQ114
BTOQ133
BTOQ086
BTOQ055
BTOQ057
BTOQ081
BTOQ009
BTOQ078
BTOQ095
TOR
6.97
7.77
2.55
3.81
4.37
4.33
0.00
5.22
8.21
3.44
4.76
8.68
TOT
4.55
6.06
1.89
3.00
2.43
2.52
0.00
4.17
6.09
1.99
3.30
6.91
Run 2
BTOQ037
BTOQ041
BTOQ107
BTOQ114
BTOQ133
BTOQ086
BTOQ055
BTOQ057
BTOQ081
BTOQ009
BTOQ078
BTOQ095
5.85
6.40
1.50
4.38
3.89
3.47
0.00
3.96
6.96
3.27
6.16
7.50
3.76
5.21
1.46
3.93
2.50
1.76
0.00
3.23
4.43
2.03
3.81
6.08
Percentage Diff.
BTOQ037
BTOQ041
BTOQ107
BTOQ114
BTOQ133
BTOQ086
BTOQ055
BTOQ057
BTOQ081
BTOQ009
BTOQ078
BTOQ095
17.5%
19.3%
52.1%
14.0%
11.8%
22.1%
0.0%
27.3%
16.5%
5.2%
25.6%
14.7%
19.0%
15.1%
25.8%
26.9%
3.1%
35.5%
0.0%
25.3%
31.6%
2.1%
14.2%
12.7%
21
-------
Table 4-9. Summary of Flow Rate PE Audit
Date
4/5/13
4/26/13
Reference Sampler
RAAS - Channel 1
RAAS - Channel 4
BGI - Sampler 1
BGI - Sampler 2
Measured
Flow
(L/min)
16.63
16.67
16.82
16.83
Difference from
Nominal
-0.6%
0.0%
0.9%
1.0%
Additionally, the temperature and pressure sensors of the reference samplers were audited
using NIST-traceable transfer standards. A summary of those audit results are shown in
Table 4-10 and indicate that both the temperature and pressure sensors in the reference
samplers were within the acceptance criteria for the verification test (i.e., ± 2°C for
temperature, and ± 5 mmHg for pressure).
Table 4-10. Summary of Temperature and Pressure PE Audit
Date
4/5/13
4/8/13
4/26/13
Reference
Sampler
RAAS
RAAS
BGI - Sampler 1
BGI - Sampler 2
Sampler
Temp. (°C)
15.3
12.6
13.0
Audit Temp.
15.6
12.1
12.8
Sampler
Pressure
(mmHg)
736
749
748
Audit
Pressure
(mmHg)
736
749
749
4.3.2 Technical Systems Audit
A Battelle QA Officer performed one ISA as part of this verification test. The TSA was
performed at the BCS3 site in Columbus, OH. The TSA focused on observation of the
reference method sampling and field QA/QC procedures in preparation for the field test. The
purpose of the audit was to ensure that the verification test was being performed in
accordance with the AMS Center QMP, and the test/QA plan for this verification test. In the
audit, the Battelle QA Officer observed the reference method sampling and sample recovery,
compared the actual test procedures being performed to those specified or referenced the
test/QA plan, reviewed data acquisition and handling procedures, inspected documentation of
reference sample chain of custody, performance of flow, pressure, and temperature PE audits,
and reviewed test record books. One finding and five observations were noted requiring two
deviations. The first deviation pertained to leak checks not occurring at the recommended
frequency after each flow check. The VTC started performing leak checks after each flow
check. It is not expected that the failure to conduct the leak checks had a substantial impact
on the results since subsequent leak checks passed the acceptance criteria and there were no
systemic biases between the reference results prior to conducting regular leak checks. The
second deviation was to address the change from the Anderson RAAS to the two BGI PQ
200 samplers. While not a finding, this deviation was necessary to document the change in
22
-------
samplers and did not impact the quality of results. The remaining observations noted were
minor and did not impact the quality of results.
4.3.3 Data Quality Audit
At least 10% of the data acquired during the verification test were audited. Battelle's Quality
Manager traced the data from the acquisition, through reduction and statistical analysis, to
final reporting, to ensure the integrity of the reported results. All calculations performed on
the data undergoing the audit were checked and QC results verified. Corrections from the
other vendor report were made to the Magee data prior to QA review, resulting in minor
comments. A few data issues were noted in the data quality audit, with minimal to no effect
on the overall quality of the verification results. A few data issues were noted in the data
quality audit, with minimal to no effect on the overall quality of the verification results.
4.4 QA/QC Reporting
Each audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the
ETV AMS Center.6 The results of the audits were submitted to the EPA.
4.5 Data Review
All data received from DRI for the reference measurements underwent 100% review and
validation by Battelle technical staff before being used for any statistical calculations. This
review included a review of the data files containing the measured EC values from the
individual thermal steps for each filter and tracing of the calculated total EC measurements
for both the TOR and TOT methods. The Aethalometer data were reviewed to ensure that
the minute by minute data were appropriately averaged into hourly and 24-hour values.
Based on review of Aethalometer data files and operator logs, a small number of 2 hour
measurements were missing because of instrument maintenance activities. Those data are
detailed in Section 6.5.1. All reference data were found to be valid and were included in the
data analysis.
Records generated in the verification test received a one-over-one review (e.g., review by
staff not involved in the generation of the record, but with at least the same technical
expertise as the person generating the record) before these records were used to calculate,
evaluate, or report verification results. Data were reviewed by a Battelle technical staff
member involved in the verification test. The person performing the review added his/her
initials and the date to a hard copy of the record being reviewed.
23
-------
Chapter 5
Statistical Methods
The statistical methods used to evaluate the quantitative performance factors listed in Section
3.1 are presented in this chapter.
5.1 Comparability
The Aethalometers were evaluated for comparability in two ways. Firstly, comparability was
determined from a linear least squares regression analysis of the measured BC concentrations
from the Aethalometers against the corresponding EC results from the reference method. For
comparison to the reference results, average concentrations from each of the Aethalometers
were determined separately for each of the 12-hour sampling periods, by averaging the
monitor's individual 1-minute readings into hourly averages and then averaging the hourly
results into 12-hour averages over the corresponding reference method sampling period. The
12-hour averages from the two Aethalometers were plotted against the mean of the
corresponding duplicate reference method measurements. The slope and intercept of these
plots was determined from a linear regression analysis and reported independently for each
of the monitors.
Additionally, comparability was determined in terms of the relative percent difference (RPD)
between the mean value of the reference measurements and the results from each
Aethalometer tested. The RPD was calculated using Equation 1:
.100 (1)
n re
where: Ct is the average BC concentration measured by the BC monitor
during the ith reference sampling period, and
C(ref)t is the mean of the duplicate reference method BC
concentrations for the ith reference sampling period.
24
-------
Both measures of comparability were determined relative to the mean results from the
duplicate reference results.
5.2 Correlation
The degree of correlation of the results from each Aethalometer to the reference method
results were determined based on the coefficient of determination (r2) value of the linear
regression performed to assess comparability (Section 5.1). Correlation was determined
separately for each unit of each BC monitor undergoing testing, and relative to the results
from the reference method.
5.3 Precision
Precision (P) was determined based on a comparison of paired measurements from the
duplicate BC monitors being tested. For this assessment of precision, the P between the
paired measurements from the duplicate BC monitors was calculated using Equation 2:
where C(l), and C(2), are the BC concentrations measured by the first and second of the two
duplicate Aethalometers. Precision was calculated for each set of duplicate BC monitors for
each reference sampling period, and the overall mean precision is also reported. For this
calculation, measurement data below the vendor's stated instrumental detection limit was
excluded from the analysis.
5.4 Data Completeness
Data completeness was assessed in two ways, based on the overall data return achieved by
each Model AE33 Aethalometer during the testing period. First, for each of the BC monitors
data completeness was calculated as the total hours of apparently valid data reported by the
monitor divided by the maximum total possible hours of monitoring data in the entire field
period. Also, for each Model AE33 Aethalometer data completeness was calculated as the
percentage of 12-hour reference method sampling periods in which the monitor provided at
least 9 hours of valid data (75%). The causes of any substantial incompleteness of data
return was established from operator observations or vendor records, and noted in the
discussion of data completeness results.
5.5 Operational Factors
Operational factors such as maintenance needs, data output, consumables used, ease of use,
repair requirements, etc., were evaluated based on observations recorded by Battelle staff,
and explained by the vendor as needed. A laboratory record book was maintained at the test
site, and was used to enter daily observations on these factors. Examples of information
recorded in the record book include the daily status of units, maintenance performed, and
observations recorded during the installation and removal of the units.
25
-------
Chapter 6
Test Results
Figure 6-1 shows the hourly average data from the duplicate Aethalometers during the
verification testing period. These data are corrected to standard temperature and pressure
conditions (1 atmosphere, 25 °C). The two units are identified by their serial numbers (089
and 090, respectively). To facilitate comparisons to the reference method results the
concentrations are reported in |J,g/m3 rather than ng/m3 as reported in the raw Aethalometer
data files. The calculated differences (expressed in terms of SN089 - SN090) are presented
in Figure 6-2, and show a general tendency for SN 089 to read lower than SN 090.
6 -
c
OJ
u
o
u
u
OQ
4 -
2 -
SN 089
SN 090
0
4/5/13
4/12/13 4/19/13 4/26/13
Date
5/3/13
Figure 6-1. Measured hourly average BC concentration from the duplicate Model AE33
Aethalometers during testing.
26
-------
0.5 -
I
01
u
c
s
0 -
-0.5
-1
4/5/13
4/12/13
4/19/13
Date
4/26/13
5/3/13
Figure 6-2. Calculated differences between the hourly average BC concentration from the
duplicate Model AE33 Aethalometers during testing (Difference = SN 89 - SN 90).
6.1 Comparability
The comparability of the Model AE33 Aethalometers with the reference method was
determined in two ways. Firstly, comparability was determined from a linear least squares
regression analysis of the BC concentrations measured by the Model AE33 Aethalometers
and the EC concentrations measured by the reference methods as described in Section 5.1.1.
Also, comparability was determined from the RPD of the 12-hour Aethalometer averages and
the mean of the reference method data for each sampling period, as described in Section
5.1.2. For these calculations, the 12-hour results for the Model AE33 Aethalometers were
calculated as the averages of the hourly Aethalometer averages from the respective reference
method periods. Comparability was determined independently for each of the Model AE33
Aethalometers, and comparability was calculated with respect to both the TOR and TOT
reference method results. The results of these analyses are presented below.
Figures 6-3 and 6-4 show time series plots of the 12-hour average Aethalometer results and
the corresponding mean TOR and TOT reference method results, respectively. The dates
shown correspond to the start of the respective sampling periods, with the first sampling
period beginning on April 5 at 7:00 pm and ending on April 6 at 6:50 am.
27
-------
CO
1
C
o
"•p
£
+•»
0)
o
C
o
o
o
CD
3 -
2 -
1 -
ISN089
ISN090
Mean Reference Method
0
4/5/13
0
4/12/13
4/19/13
Date
4/26/13
5/3/13
Figure 6-3. Comparison of the 12-hour BC averages from the Model AE33 Aethalometers and the mean reference method TOR EC
concentrations.
28
-------
-. 3
CO
E
1
55
i 2
a>
o
c
o
o
o
CD
1 -
SN089
SN090
Reference Method
0
4/5/13
4/12/13
4/19/13
Date
4/26/13
5/3/13
0
Figure 6-4. Comparison of the 12-hour BC averages from the Model AE33 Aethalometers and the mean reference method TOT EC.
29
-------
6.1.1 Regression Analysis
Figures 6-4 and 6-5 show linear regressions of the 12-hour BC averages from the
Aethalometers with the corresponding TOR and TOT reference method results, respectively.
These figures show that there is a positive bias (i.e., slope > 1) of the Aethalometer results
relative to the reference method results, as well as a positive intercept of-0.3 ng/m3 for each
of the regression lines.
Note that a single data point at the highest observed BC concentration appears to be an
outlier and significantly impacts the calculated slopes and intercepts for the regression lines.
A linear relationship appears to exist below 1.5 ng/m3 and that additional measurements at
higher concentrations are needed to confirm a linear relationship at higher concentrations. It
is not apparent whether this point is an outlier or is indicative of a non-linear correlation
between the IMProVE method results and the Aethalometer at high concentrations. This data
point is clearly apparent in each of the regression plots and consistently falls below the
regression lines in each plot.
C
O
0)
u
C
O
O
O
00
L.
0)
0>
E
£
re
I
* Apparent
Outlier
SN 089 = 1.277x + 0.286
R2 = 0.875
SN090= 1.350X+0.309
R2 = 0.880
01234
Reference Method EC Concentration (|j.g/m3)
Figure 6-4. Linear regression of 12-hour Aethalometer BC averages against mean reference
method TOR EC results.
30
-------
c
o
re
o>
u
c
o
O
O
00
0)
E
£
re
I
SN089= 1.701X +0.305
R2 = 0.906
SN090 = 1.795X+0.330
R2 = 0.908
1234
Reference Method EC Concentration (|j.g/m3)
Figure 6-5. Linear regression of 12-hour Aethalometer BC averages against mean reference
method TOT EC results.
Table 6-1 presents a summary of the regression results from the Aethalometers relative to
both the TOR and the TOT reference method results. The uncertainties (one standard
deviation) of the calculated slopes and intercepts are included parenthetically. These results
show a positive bias (slope > 1, positive intercept) for the Aethalometers relative to the
reference method for both the TOR and TOT analyses. All of the regression slopes in Table
6-1 are significantly different from 1.0 at the 95 percent confidence level, and all of the
intercept values are similarly significantly different from zero at that confidence level. These
regression results include the apparent outlier data point at the highest observed
concentration.
Table 6-1. Summary Regression Results of the Aethalometers and the Reference
Method
SN089
SN090
TOR
Slope
1.277(0.064)
1.350(0.066)
Intercept (|ag/m3)
0.286 (0.041)
0.309 (0.042)
TOT
Slope
1.701 (0.072)
1.795 (0.076)
Intercept (ng/m3)
0.305 (0.034)
0.330(0.036)
Table 6-2 presents revised regression results excluding that data point. In general, the
exclusion of the apparent outlier data point resulted in an increase in the slopes of the
regression results and a decrease in the intercepts. All of the regression slopes in Table 6-2
are significantly different from 1.0 at the 95 percent confidence level, and all of the intercept
values are similarly significantly different from zero at that confidence level.
31
-------
Table 6-2. Summary of Regression Results of the Aethalometers and the Reference
Method Excluding Apparent Outlier Data Point
SN089
SN090
TOR
Slope
1.585 (0.061)
1.661 (0.064)
Intercept (jig/m3)
0.172(0.032)
0.194(0.034)
TOT
Slope
2.013 (0.067)
2.105(0.074)
Intercept (jig/m3)
0.224 (0.027)
0.249 (0.030)
6.1.2 Relative Percent Difference A nalysis
Table 6-3 presents a summary of the calculated RPD between the 12-hour averages for the
the Aethalometers relative to the TOR and the TOT reference method results. For these
calculations, reference method results below twice the method detection limit were excluded.
For perfect agreement between the Aethalometers and the reference method results, the RPD
would be zero. In general, the measured concentrations from the Aethalometers were
approximately twice as high as those from the reference method resulting in positive RPD
values. It should be noted that only about two thirds of the TOR reference method results
and fewer than half the TOT reference method results were above twice the detection limit.
Table 6-3. Summary of Relative Percent Difference between the Aethalometers and the
TOR Reference Method Results
RPDa
SN089
SN 089 - RAAS
SN 089 - BGI
SN090
SN 090 - RAAS
SN 090 - BGI
TOR
95.6%(N=39)
100.9% (N=23)
88.1%(N=16)
109.5% (N=39)
115.3%(N=23)
101.0%(N=16)
TOT
149.7% (N=26)
168.0% (N=15)
124.7% (N=ll)
163.7%(N=26)
183.9%(N=15)
136.1% (N=ll)
a: Includes only reference data that exceeded twice the method detection limit.
6.2 Correlation
Table 6-4 presents a summary of the coefficient of determination (r2) for the results from the
Aethalometers relative to both the TOR and the TOT reference method results (see Figures 6-
2 and 6-3). The correlation results were calculated including all of the data and also with the
apparent outlier shown in Figure 6-4 removed. In all cases, exclusion of the apparent outlier
increased the calculated r2 value, with a larger increase in r2 for the TOR results than in r2 for
the TOT results. With both data sets the correlation of Aethalometer results with TOT results
was higher than with TOR results.
32
-------
Table 6-4. Summary of Correlation Results of the Aethalometers and the Reference
Method
SN089
SN090
r2 (All data)
TOR
0.875
0.880
TOT
0.906
0.908
r2 (Outlier removed)
TOR
0.924
0.922
TOT
0.941
0.936
6.3 Precision
Table 6-5 presents a summary of the calculated unit-to-unit precision for the duplicate
Aethalometers for the calculated hourly and 12-hour averages. The total number of paired
measurements in which the readings from both analyzers exceeded the detection limit of
0.005 ng/m3 is included parenthetically for each calculation.
Table 6-5. Summary of Calculated TJnit-to-TJnit Precision Results of the Aethalometers
RPD
1-hour
12-hour
8.5%(N=756)
7.1%(N=63)
Figures 6-6 and 6-7 show linear regressions of the hourly averages from the duplicate
Aethalometers on a linear and logarithmic scale, respectively. Figures 6-8 and 6-9 show
similar plots for the 12-hour averages. Table 6-6 summarizes the results of the unit-to-unit
linear regressions of the Aethalometer data.
33
-------
o 6
o>
o
"S 4
c
o
z
V)
c
o
1 -
0.1 -
o
CO
0.01
y = 1.063x-0.000
R2 = 0.990
N=756
0.01
0.1 1
BC Concentration - SN 089 (|ig/m3)
10
Figure 6-7. Linear regression of Aethalometer 1-hour averages (logarithmic scale).
34
-------
3 -
1
o
o>
o
w
c
g
is
o
o
o
00
2 -
1 -
y = 1.051x + 0.010
R2 = 0.992
N=63
01234
BC Concentration -SN 089 (ng/m3)
Figure 6-8. Linear regression of the Aethalometer 12-hour averages (linear scale).
10
O)
I
0>
o
tn
0)
o
o
o
o
CO
0.1 -
0.01
y = 1.051x +0.010
R2 = 0.992
N=63
0.01
0.1 1 10
BC Concentration - SN 089 (ng/m3)
Figure 6-9. Linear regression of the Aethalometer 12-hour averages (logarithmic scale).
Table 6-6. Summary of Unit-to-Unit Regression Results of the Aethalometers
35
-------
1-hour
12-hour
Slope
1.063(0.004)
1.051(0.012)
Intercept
(Hg/m3)
-0.000 (0.005)
0.010(0.012)
2
r
0.990
0.992
Fort both the 1-hour and 12-hour averages, the regression slopes in Table 6-6 are
significantly different from 1.0 at the 95 percent confidence level, whereas the intercept
values are not significantly different from zero at that confidence level.
6.4 Data Completeness
Table 6-7 presents a summary of the data completeness for the duplicate Aethalometers
during the testing period. These results are based on the total number of 1-minute
measurements recorded during the testing period and the total number of valid 12-hour
averages that were calculated to correspond to the reference method sampling periods. Each
of the analyzers recorded valid data for at least 99.5% of the total 1-minute periods during
the verification test and valid 12-hour averages were calculated for 100% of the reference
method sampling periods. Other than the maintenance activities noted in Section 6.5.1, the
data loss observed in the 1-minute readings is attributed to periods during which the filter
tape in the Aethalometers was advanced.
Table 6-7. Summary of data completeness for the Aethalometers
Analyzer
SN089
SN090
1-minute
Total
Periods
45,360
45,360
Valid
Measure
ments
45,157
45,149
%
Complete
99.6%
99.5%
12-hour
Total
Periods
63
63
Valid
Measure
ments
63
63
%
Complete
100%
100%
6.5 Operational Factors
This section addresses the maintenance, consumables, waste generation, ease of use, and
other factors relevant to operation of the Model AE33 Aethalometer.
6.5.1 Maintenance
Table 6-8 shows the maintenance activities that were performed on the two Model AE33
Aethalometers during the verification test.
36
-------
Table 6-8. Summary of Maintenance Performed on Aethalometers
Date
3/29/13
4/17/13
Maintenance
Restore instrument defaults
from memory card for each
analyzer
Restore instrument defaults
from memory card for SN90
Approximate
time
-10 minutes
~5 minutes
Data loss
-15 minutes
-10 minutes
6.5.2 Consumables/Waste Generation
The Aethalometer uses rolls of filter tape to sample the ambient air. The tape was not
exchanged during the verification testing period and only a small fraction (estimated at less
than 10%) of the total amount of tape on the rolls in the two Aethalometers was used
between the time of installation and the end of the verification test (-43 days).
6.5.3 Ease of Use
Installation of the Aethalometers was straightforward and involved removal of the analyzers
from their respective shipping containers, placing the analyzers on a bench in the mobile
laboratory, and supplying power to the analyzers. Electrically-conductive ("static
dissipative") flexible tubing was used as sampling inlets for the Aethalometers and was
installed using a "push-in" fitting. Installation of the two units including the sample inlets
was completed in approximately 5 minutes. Installation of the sampling lines and inlet
cyclones for the two Aethalometers required approximately 10 minutes. After installation,
the units were allowed to operate overnight and the flow rates were checked the following
morning using a NIST traceable flow device. That flow calibration required less than half an
hour to complete. Routine operation required no effort other than brief daily instrument
checks and approximately weekly data downloads. The downloaded data files were in
comma separated variable (csv) format and were processed in Microsoft Excel.
37
-------
Chapter 7
Performance Summary
Table 7-1 presents a summary of the results of the verification of the Model AE33
Aethalometer during this verification test.
Table 7-1. Summary of Verification Test Results for the Model AE33 Aethalometer
Comparability-
Regression analysis
comparison to
reference samples
Analyzer
SN089
SN090
TOR
Slope
1.277(0.064)
1.350(0.066)
Intercept
0.286(0.041)
0.309 (0.042)
TOT
Slope
1.701 (0.072)
Intercept
0.305 (0.034)
1.795(0.076) 0.330(0.036)
Analyzer
RPD
Comparability- Calculation of
RPD between Aethalometer results
and reference method results
TOR
SN089
95.6%(N=39)
SN090
105.9% (N=39)
TOT
149.7%
(N=26)
163.7%
(N=26)
Correlation - Regression analysis
comparison to reference samples
Analyzer
TOR
SN089
0.875
SN090
0.880
TOT
0.906
0.908
Precision - Comparison of results from duplicate
monitoring systems
RPD (# of Observations)
1-hour
8.5% (N=756)
12-hour
7.1%(N=63)
Precision - Regression analysis of results
from duplicate monitoring systems
Period
Slope
Intercept
(Hg/m3)
1-hour
1.063
(0.004)
-0.000 (0.005)
12-hour
1.051
(0.012)
0.010(0.012)
0.990
0.992
Data Completeness
Analyzer
Period
Total
Periods
Valid
Measurements
%
Complete
SN089
1-minute
45,360
45,157
99.6%
12-hour
63
63
100%
SN090
1-minute
45360
45,149
99.5%
12-hour
63
63
100%
38
-------
Table 7-1 (continued). Summary of Verification Test Results for the Model AE33
Aethalometer
Maintenance
• Default instrument settings restored from internal memory card
twice during testing.
• No routine maintenance performed during testing.
Consum ables/waste
generated
Filter tape required.
Ease of use
Installation of two units without inlets completed in ~5 minutes.
Installation of inlets and sampling lines completed in ~10 minutes
Calibration of flow rates completed in less thanSO minutes, after
allowing the units to operate overnight.
Routine operation required no effort other than brief daily instrument
checks and approximately weekly data downloads.
Data exported as csv files and processed using Microsoft Excel.
39
-------
Chapter 8
References
1. Allen, G.A., Babich, P., and Poirot, R.L., "Evaluation of a New Approach for Real Time
Assessment of Wood Smoke PM." Found at:
http://www.ct.gov/deep/lib/deep/air/regulations/proposed_and_reports/pm25/appendix_2c.pdf
2. Battelle, Quality Assurance Project Plan for Verification of Black Carbon Monitors: Version
1, Battelle, Columbus, Ohio, April 12, 2013.
3. Report to Congress on Black Carbon, EPA-450/R-12-001, U.S. Environmental Protection
Agency, March 2012. Available at http://www.epa.gov/blackcarbon/.
4. Desert Research Institute, DRI Model 2001 Thermal/Optical Carbon Analysis
(TOR/TOT) of Aerosol Filter Samples - Method IMPROVE_A, DRI SOP#2-216r3,
prepared by DRI, Reno, NV, October 22, 2012.
5. U.S. EPA, Environmental Technology Verification Program Quality Management Plan,
EPA Report No: EPA 600/R-08/009, U.S. Environmental Protection Agency, Cincinnati,
Ohio, January 2008.
6. Battelle, Quality Management Plan for the ETV Advanced Monitoring Systems Center,
Version 8.0, U.S. EPA Environmental Technology Verification Program, prepared by
Battelle, Columbus, Ohio, April 5, 2011.
40
------- |