oEPA
EPA 600/R-14/143 | May 2014 | www.epa.gov/ord
United States
Environmental Protection
Agency
Sensor Evaluation Report
Office of Research and Development
National Exposure Research Laboratory
-------
EPA/600/R-14/143 May 2014 www.epa.gov/ord
Sensor Evaluation Report
Ron Williams, Russell Long, and Melinda Beaver
National Exposure Research Laboratory
Office of Research and Development
U.S. Environmental Protection Agency
Research Triangle Park, NC, USA 27711
Amanda Kaufman
ASPPH Environmental Health Fellow hosted by EPA
Association of Schools and Programs of Public Health
Washington, DC, 20036
Florian Zeiger
AGT International (AGT Group R&D GmbH)
64295 Darmstadt, Germany
Michael Heimbinder, HabitatMap
669 Carroll St, Suite #3, Brooklyn, NY 11215
lem Heng, Manhattan College
Raymond Yap, Manhattan College
Bharat R. Acharya, Bart A. Grinwald, Kurt A. Kupcho, Sheila E. Robinson.
Platypus Technologies, LLC
Madison, Wl 53711
-------
Olivier Zaouak and Bruno Aubert
Cairpol
Zac La Novialle
10 rue de la Serre
63670 La Roche Blanche, France
Michael Hannigan, Ricardo Piedrahita, Nicholas Masson
University of Colorado Boulder
Boulder, CO 80309
Bob Moran and Malcolm Rook
Weather Telematics
Ottawa, Ontario, Canada K2P OG5
Paul Heppner and Cathy Cogar
Global Science and Technology
Greenbelt, Maryland 20770
Nima Nikzad and William G. Griswold
University of California, San Diego
La Jolla, CA 92093-0404
-------
Disclaimer
Some of the information reported here was obtained through the U.S.
Environmental Protection Agency EPA Contract # EP-D-10-070 to Alion Science
and Technology. It has been subjected to the Agency's peer and administrative
review and has been approved for publication as an EPA document. Mention of
trade names or commercial products does not constitute endorsement or
recommendation for use.
-------
Acknowledgments
The U.S. EPA wishes to thank the collaborating institutions and their staff members for
their contribution to the success of this research. While this research was originally conducted
as part of a confidential collaboration to provide timely feedback to sensor manufacturers with
the goal of decreasing development time of sensors with improved sensor performance, it has
resulted in performance characteristics of the tested devices being openly shared with others.
These institutions responded to a world-wide open invitation to be a part of this research
endeavor. The importance of their willingness to openly share their emerging technology with
us and acceptance of both positive and negative feedback on their sensors cannot be
underestimated. This effort involved a highly iterative process of direct communication between
the U.S. EPA and these institutions resulting in collaborating institutions often making numerous
changes in hardware and software components to accommodate the data collection protocol.
Ms. Kathleen Graham and Sarah Bauer provided FTTA oversight, guidance, and
encouragement in the establishment of the cooperative agreements used in the performance of
this effort. The NERL's Quality Assurance Manager (Sania Tong-Argao) and associated staff
(Ms. Monica Nees) are acknowledged for laboratory data audits as well as their excellent
contributions to the development of sophisticated standard operating procedures used in
collection of the data. Sam Garvey, Robert Mickler, Zora Drake-Richmond, and Keith Kronmiller
(Alion Science and Technology) are acknowledged for their contributions in supporting the U.S.
EPA in the execution of complex laboratory data collections and summary analyses. Linda
Sheldon, Roy Fortmann, Peter Preuss, Stacey Katz, Emily Snyder, and Gail Robarge (U.S.
EPA) are acknowledged for their efforts to ensure the success of the research effort reported
here.
IV
-------
-------
Abstract
This report summarizes the results of low cost air quality sensor performance trials
conducted in the NERL's on-site laboratories located in Research Triangle Park, NC during
2012-2013. Such trials were viewed as highly valuable for all parties following the conclusion
of the U.S. EPA's Air Sensor and APPs conference conducted in the spring of 2012.
Conference attendees from a wide range of interests (government, sensor development, citizen
scientists, etc) concluded that basic sensor performance characteristics of available sensor
technologies were unknown. Many potential users of these technologies were unaware that
data quality needed to be established or how one might perform such determinations. Many
conference attendees shared that the U.S. EPA needed to take a leadership role to both
educate sensor developers and end users about needed performance standards, share
technical information about how one would establish such characteristics, and foster continuing
dialogue between all interested parties on this subject matter.
As a result of the aforementioned conclusion, the U.S. EPA initiated a research program
in the summer of 2012 to survey sensor developers about potential technology-sharing with the
ultimate goal of conducting exhaustive laboratory-based sensor performance trials.
Collaborating institutions responded to an open, web-based invitation to submit applications to
participate. Applicant screening was performed that ensured the sensors being offered for
evaluation were available to others for use and that they had not been previously examined via
any third-party laboratory. In addition, there had to be a lack of publically-available information
on sensor performance for inclusion in the research.
A total of nine institutions of multi-national origin agreed to establish confidential
MCRADA research with the U.S. EPA associated with the performance of Os and NC>2 gas
phase air quality sensors. These sensor types being selected due to their prevalence in the
market place and the availability of established test regimens that might be applied. Findings
associated with sensors shared by seven of these parties are described in this report and are
being voluntarily shared here by the collaborating institutions. These institutions were a
combination of academic, private sector, and non-profit organizations that had responded to the
open challenge of participating in exhaustive laboratory-based trials of basic sensor
performance characteristics.
Performance traits such as response linearity, response reaction times, detection limit,
and response to interfering agents, among others were established. In addition, characteristics
important to potential users such as battery life, ease of operation, data storage and/or
communication protocols were investigated. Laboratory data was collected using an exposure
chamber linked with either FRM or FEM instrumentation. Sensors were individually challenged
using a variety of test conditions and their resulting response recorded. Replicate trials of each
challenge were performed to provide needed statistical strength in establishing performance.
Select findings from these trials are as follows:
(1) Many of these sensors, having commercial values of < $1000, demonstrate some
performance characteristics that often rival those of FRM/FEM instrumentation. The sensors
often exhibited very fast response times with minimal rise and lag times which suggests
potential use for continuous or near-continuous environmental monitoring.
(2) Many of the sensors had a high degree of linearity over their full response range at
concentrations often well above normally observed environmental concentrations.
(3) While the sensors often did not have the detection limits as low as FRM/FEM
instrumentation, they often achieved levels near these concentrations that would appear to
meet a wide variety of environmentally-relevant monitoring needs.
VI
-------
(4) Some of the sensors revealed some unwanted co-response to interfering agents (e.g., NC>2,
Os, SO2). Likewise, extremes of RH and temperature often resulted in some undesirable
response characteristics.
(5) Establishment of data collection/recovery protocols involved a wide array of approaches to
achieve success (e.g., WiFi hot spots, cellular telephone, SD card, proprietary web data
portals). This often involved iterative upgrades to communication protocols, hardware and
other integrated systems as testing was initiated on each device.
VII
-------
Table of Contents
List of Tables x
List of Figures x
Abbreviations xv
Executive Summary xi
1.1ntroduction 1
1.1 Background 1
1.2 Research Initiative 2
1.3 Call for Sensor Developer Applications 2
1.4 Study Inclusion Criteria 3
1.5 Collaborative Research Agreement 3
2. Study Objectives 4
2.1 Primary Goals 4
2.2 Laboratory Testing 4
2.3 Specific Objectives 4
2.4 Secondary Goal Objectives 5
3. Study Approach 6
3.1 Background 6
3.2 Evaluation Procedures 6
3.2.1 Exposure Chamber 6
3.2.2 Physical Parameters (Temperature and Relative Humidity) 6
3.2.3 Continuous Gas (Reference) Monitors 7
3.2.4 System Characterization 8
3.2.5 Sensor under Test Samples 8
3.3 Test Procedures 8
3.3.1 Linearity (Range) 9
3.3.2 Precision of Instruments 9
3.3.3 Lower Detectable Limit (LDL) 9
3.3.4 Concentration Resolution 9
3.3.5 Response Time 10
3.3.5.1 Lag Time 10
3.3.5.2 Rise Time 10
3.3.6 Interference Equivalent 10
3.3.7 Relative Humidity and Temperature Influences 11
3.3.7.1 Relative Humidity (RH) 11
3.3.7.2 Temperature 11
4. Quality Assurance 12
4.1 Development of QA/QC Materials 12
4.2 Data Collection 12
4.3 QA Systems Audit 12
4.4 Data I nclusion Process 13
5. Results 14
5.1 Ozone Sensors 14
VIII
-------
5.1.1 AGT 14
5.1.2 CairClip 16
5.1.3CitiSense 18
5.1.4 Dynamo 18
5.1.5 U-Pod 19
5.2 Nitrogen Dioxide Sensors 21
5.2.1 AGT 21
5.2.2 AirCasting 22
5.2.3 CairClip 24
5.2.4 CitiSense 25
5.2.5 Platypus 27
5.2.6 U-Pod 28
6. Study Limitations 30
6.1 Resource Limitations 30
6.1.1 Minimal Findings on Intra-Sensor Performance Characteristics 30
6.1.2 Minimal Environmental and Interfering Agent Testing Conditions 30
6.1.3 Limited Number of Sensors 30
7. Conclusions 33
Appendix A: Technical Aspects- FRM/FEM Performance Parameters 35
Appendix B: Photographs of Sensors 36
Appendix C: List of Research Operating Protocols and Quality Assurance Project Plans Used in
Support of This Research 40
IX
-------
List of Tables
Table 1. Reference Analyzers Used in Sensor Evaluation Study 7
Table 2. Interference Test Atmospheres for Apps/Sensors 10
Table 3. Summary of AGT Ozone Environmental Sensor Testing 15
Table 4. Summary of CairClip Ozone SensorTesting 17
Table 5. Summary of CitiSense Ozone Sensor Testing 18
Table 6. Summary of Dynamo Ozone SensorTesting 19
Table 7. Summary of U-Pod Ozone SensorTesting 20
Table 8. Summary of AGT Nitrogen Dioxide SensorTesting 22
Table 9. Summary of AirCasting Nitrogen Dioxide SensorTesting 23
Table 10. Summary of CairClip Nitrogen Dioxide SensorTesting 25
Table 11. Summary of CitiSense Nitrogen Dioxide SensorTesting 26
Table 12. Summary of Platypus Nitrogen Dioxide SensorTesting 28
Table 13. Summary of U-Pod Nitrogen Dioxide SensorTesting 29
Table 14. Sensor Specifications 31
List of Figures
Figure 1. Ace Glassware Exposure Chamber Showing Input and Output Ports and Caps 7
Figure 2. Example AGT ATS-35 Response for Os Under Normal Challenge Conditions 16
Figure 3. Example CairClip Response for Os under Normal Challenge Conditions 17
Figure 4. Example Dynamo Response for Os under Normal Challenge Conditions 19
Figure 5. Example AGT ATS-35 Response for NO2 under Normal Challenge Conditions 21
Figure 6. Sensor Characteristics for AGT NO2 Sensor 22
Figure 7. Example AirCasting Response for NO2 under Normal Challenge Conditions 24
Figure 8. Example CairClip Response for NO2 under Normal Challenge Conditions 25
Figure 9. Example CitiSense Response for NO2 under Normal Challenge Conditions 27
-------
Executive Summary
Background
Low cost air quality sensors are indicative of emerging technologies that have a wide
appeal to both professional researchers and citizen scientists. They exist in numerous
configurations (e.g., cell phone, hand-held) and are often available with a wide assortment of
sensor configurations. Many of these configurations include the ability to measure gas phase
air pollutants such as NC>2 and Os. While the commercial availability of such devices has
increased dramatically in the last five years, uncertainty about the quality of data such devices
might be capable of providing has been raised.
Study Objectives
The U.S. EPA as part of its Air Climate & Energy (ACE) research program on emerging
technologies (ACE EM-3), developed a research initiative that would seek to survey both NC>2
and Os low cost sensors as to their availability and then work in collaboration with their
developers on understanding basic performance characteristics of such sensors. A world-wide
survey of potential sensor candidates for laboratory-based evaluations was conducted.
Applicant sensors were screened for inclusion in the research based upon publicized selection
criteria. Evaluation of each application by the U.S. EPA's Federal Technology Transfer Act
(FTTA) office was performed to ensure the integrity of the collaborative agreement and the
confidential nature of technology-sharing. As a result, a total of nine Material Cooperative
Research and Development Agreements (MCRADAs) were established during the 2012-2013
time period where sensor developers submitted their technologies to the U.S. EPA for
exhaustive laboratory trials. The work would be collaborative in nature between the U.S. EPA
and the various institutions but confidential in that findings and proprietary information related to
sensor design and performance would not be shared outside of the one-on-one MCRADA
relationship. The aforementioned trials would be performed at near-Federal Reference and
Equivalent Method (FRM/FEM) criteria in U.S. EPA's Research Triangle Park, NC laboratories.
The comparison of performance as it related to this very high level of data quality was
conducted to provide the greatest amount of feedback possible to the sensor developers as to
the value of their device for a wide range of air quality applications.
Study Approach
Collaborative research was initiated during the fall of 2012 and was completed by the
summer 2013. The U.S. EPA's existing FRM/FEM laboratory was modified to accommodate
sensor testing. A quality assurance program was designed and instituted that featured definitive
operating procedures for all reference monitors, gas generation systems, test chamber
operating parameters, sensor operation, and data recovery/analysis procedures. Sensors were
received by the U.S. EPA and integrated into the established testing regimen agreed upon by all
parties. Third party (external to the study investigators) quality assurance audits were
performed during the study period to ensure data collection procedures met all established
protocols. An iterative process of dialogue was maintained by all parties during this time period
featuring both group (all MCRADA collaborators) as well as individual communication between
the U.S. EPA and any respective institution to ensure full disclosure of the research protocol,
progress, difficulties, and achievements that occurred as a result of the collaboration.
XI
-------
Following the completion of the data collection, validation, and analysis, a summary
report was provided to each MCRADA institution as to the basic performance characteristics of
their sensor(s). Such feedback included but was not limited to data on response time, response
linearity, influence of interfering agents and ease of use. The feedback provided to each
collaborator was defined by numerous factors such as already established inability of the sensor
to operate under certain test conditions (e.g. extremes of temperature, relative humidity);
inclusion in a specific testing regimen when pre-test indicated it would not be fruitful; and lastly,
but equally important, the availability of U.S. EPA resources to conduct this research. The
U.S. EPA was committed to provide some feedback to all parties concerning performance and
therefore decisions were made in the execution of the test regimen to best balance resources
versus time expended with each sensor.
Sensor Performance Results
Discreet statistical evaluation of sensor performance was provided to each sensor
developer as well as ancillary information pertaining to ease of use features. Each sensor
tested had unique qualities of both the discreet as well as ease of use features.
Ease of Use Features Evaluation
Concerning ancillary information pertaining to ease of use features, several key findings
were evident. In general these included but not limited to:
• Power Requirements: Battery capability varied widely between sensors. Li-Ion and other
rechargeable media types were common features. Operating times of as short as 4 hours and
>24 hrs were evident or reported by the sensor developers. In order to ensure successful
testing of the sensors using the lengthy and often automated procedures, direct line voltage
was provided to each sensor to ensure their effective operation and data integrity.
• Data collection/transmission/storage/recovery: There were numerous data
collection/transmission/storage/recovery approaches observed between the various sensor
devices. Therefore, extensive efforts had to be performed to ensure data recovery to perform
the evaluations. Cellular communication, WiFi hot spots, direct storage via laptops, electronic
tablets, and even third party (proprietary web hosting) protocols had to be established,
developed, or in some cases unexpectedly refined as to the manufacturer's suggested
protocols. Data communication issues had to be fully vetted to ensure both consistent and
reliable data recovery.
• Data Schemes: Data schema was widely variable between the sensors evaluated. There
currently is no standardized approach for how data is communicated and the unique pattern of
data formatting (and the types of data being reported) makes evaluations complicated.
Individual data recovery programs had to be established for each sensor so that data could be
recovered and the U.S. EPA had sufficient knowledge of what data values were actually being
reported. As an example, some sensors directly reported an estimation of gas concentration
whereas others might only report a variable (e.g., change in resistance, voltage) that needed
to be translated using either an EPA-derived algorithm or preferably one provided by the
manufacturer to allow raw data to be reported as a concentration estimate.
• Response Range: Response range of the sensors varied widely. While some sensors
exhibited detection limits of < 10 ppb, others responded more favorably to higher
concentrations at the ppm level. The testing protocol was not changed because of these
differences but the variance in response when supposedly very similar gas detection
XII
-------
mechanisms were being employed by the sensors may indicate that the processing of raw
signal is a vital part of ensuring their overall utility.
Sensor Performance Characteristics
Many of the sensors tested exhibited qualities (e.g., response times, detection limits)
rivaling those of expensive FRM/FEM instrumentation. For example, it was not unusual to see
response times well under 1 minute with small rise and lag times following known changes in
challenge conditions. Some of these results are summarized below with more detailed
descriptions in Section 5:
• Precision: All of the sensors tested provided good to excellent precision - their ability to
reproduce a response at given challenge concentrations. It was not usual to see relative
standard deviations of response under 10% at both the low (near 0 ppb) and high (>200 ppb)
test conditions.
• Linearity: The sensors exhibited excellent linearity over a wide range of challenge
concentrations (0 to >200 to 500 ppb; the latter being dependent on the individual sensor and
gas). Coefficients of determination greater than 0.95 were often achieved.
• Relative Humidity and Temperature Changes: There was wide disparity in the response of
individual sensors to extremes of either RH or temperature challenge. Both minimal impacts
as well as extreme impacts were observed as they relate to the sensors successfully reporting
the challenge concentrations as environmental conditions changed. Some of this was
expected due to the very nature of the sensing mechanism (approach) often employed in low
cost sensors. Metal oxide and other electrochemical sensing membranes have physical
properties that need relatively stable operating parameters to remain effective. A number of
the sensors failed under such challenge circumstances with data responses outside of
acceptable reporting parameters. Resources were not available to allow for gradual changes
in environmental conditions under varying challenge gas concentrations and therefore, the full
scope of impact of changes in RH and temperature cannot was not established.
• Interferences: Wide capabilities of the sensors to respond favorably (minimally) to the
presence of a co-pollutant were observed. Some produced an interfering response of < 3 ppb
while others reported values that exceeded the ability of the sensors themselves to report an
output. Mixtures of either Os, NC>2 or SC>2 were used in various combinations of these test
challenges. Challenge concentrations of the interfering agent were established at a single,
relatively high environmental concentration that one might consider as a "worst case"
scenario. Therefore, the evaluations reported here merely provide a basic understanding of
whether a sensor might or might not (in its test configuration) be susceptible to interferents.
Conclusions
This summary should indicate that while both the discreet (performance characteristics)
and ease of use characteristics of each device were highly variable, there is strong evidence
that the sensors tested have immediate use for a wide array of environmental applications. In
fact, the overall pattern of performance for a given trait often rivaled (sometimes exceeded) that
of the FRM/FEM instrumentation while costing 2 to 4 orders of magnitude less in comparison.
Citizen scientists, academics, and others needing to obtain informative air quality data should
not discount the value in employing such devices to gain a general understanding of local air
quality. That being said, the issues observed with the sensors relating to communication
protocols, data storage, response range, environmental conditions, and specificity of response,
XIII
-------
among others, should not be discounted by potential end users. The information provided to the
collaborating institutions on sensor performance and which is summarized in this report,
represents a first step in helping to ensure the next generation of low cost air quality sensors
being developed have even more capabilities to meet a wide variety of air quality monitoring
needs. It also provides potential low cost sensor users with key information they need about
sensor performance and the criteria they need to understand in executing successful data
collection.
XIV
-------
Abbreviations
AC Alternating Current
ACE Air, Climate, and Energy
ASAP EPA's Air Sensor and APPs sensor conference
°C Degrees Celsius
CFR Code of Federal Regulations
cm Centimeter
cm s~1 Centimeters per Second
CO Carbon Monoxide
CC>2 Carbon Dioxide
DC Direct Current
EM Emerging Technology
FEM Federal Equivalent Method
FRM Federal Reference Method
FTTA Federal Technology Transfer Act
hr Hour
IDL Instrument Detection Limit
IE Interference Equivalent
in Inch
kOhm Kilo Ohm
LC Liquid Crystal
LCD Liquid Crystal Display
LDL Lower Detectable Limit
Li-ion Lithium Ion
Li-Po Lithium Ion Polymer
MCRADA Material Cooperative Research and Development Agreement
min Minute
mL Milliliter
MOS Metal Oxide Sensors
mV Millivolts
NA Not Applicable
NDIR Nondispersive Infrared
NERL National Exposure Research Laboratory
NIST National Institute of Standards and Technology
NO Nitric Oxide
NO2 Nitrogen Dioxide
NOX Generic term for mono-nitrogen oxides
Os Ozone
OD Outer Diameter
ORD Office of Research and Development
PC Personal Computer
PFA Perfluoroalkoxy
PID Photoionization Detector
ppb Parts Per Billion
ppm Parts Per Million
QAPP Quality Assurance Project Plan
QA/QC Quality Assurance/ Quality Control
RH Relative Humidity
ROP Research Operating Procedure
xv
-------
RTD Resistive Temperature Device
S Standard Deviation
SD Secure Digital
SIM Subscriber Identity Module
SO2 Sulfur Dioxide
SOP Standard Operating Procedure
urn Micrometer
URL Upper Range Limit
USB Universal Serial Bus
U.S. EPA United States Environmental Protection Agency
UV Ultraviolet
V Volt
VOC Volatile Organic Compound
XVI
-------
1. Introduction
1.1 Background
The U.S. EPA's Apps & Sensors for Air Pollution (ASAP) meeting conducted in the
spring of 2012 resulted in more than 80 invited participants from a wide variety of backgrounds
to discuss the state of sensor technology. Such technologies are being described in the
scientific literature and their general availability for air quality monitoring is being evaluated.1'2'3'4
During the meeting, general discussions on the availability of various sensors were reported as
well as how they were (or potentially could be) integrated into environmental air quality
research. Following the completion of this meeting, the U.S. EPA summarized key findings.5
One such finding was the uncertainty of data quality associated with low cost sensor
technologies. It was apparent that much attention had been given by sensor developers in the
development of devices that met criteria such as:
• Low cost.
• Lightweight and of small physical size (often cell phone size).
• Requiring minimal user knowledge or training to initiate data collection.
• Providing continuous or near-continuous pollutant data estimations.
• Reporting data either directly on the device or through some web-based portal.
• Utilizing emerging technologies such as Metal Oxide Sensors (MOS) theorized as having a
high degree of potential as a sensing element.
It was also apparent that little attention had actually been given to understanding the
data quality being produced by this class of air quality monitors. Often sensor developers were
relying upon sensor element manufacturers (the base technology that low cost sensors were
being designed around) to provide estimates of performance to the general public and potential
end users. This was being done not as a result of lax attitudes towards data quality assurance
issues but rather often a genuine lack of knowledge about what performance measures were
important. Likewise, low cost sensor developers often did not have the capability of scientifically
investigating this issue due to either lack of technical expertise or availability of proper testing
facilities. One recommendation, established as an action item for those in attending the 2012
ASAP meeting, was to develop a better understanding of basic sensor performance
characteristics and continue the dialogue between all parties (sensor developers, citizens,
community action groups, academics, and regulatory officials, and scientific investigators-at-
large).
As a direct result of the aforementioned recommendation, the U.S. EPA began an
internal dialogue about how best to facilitate the exchange of information on basic sensor
1 Roadmap for Next Generation Air Monitoring- U.S. Environmental Protection Agency.
http://www.epa.gov/research/airscience/docs/roadmap-20130308.pdf. March 8, 2013.
2 Snyder, E., Watkins, T., Thoma, E., Williams, R., Solomon, P., Hagler, G., Shelow, D., Hindin, D., Kilaru, V.,
Preuss, P. Changing the paradigm for air pollution monitoring. Environmental Science and Technology, 47: 11369-
11377(2013).
3 White, R., Paprotny, I., Doering, F., Cascio, W., Solomon, P., Gundel, L. Sensors and "Apps" for Community-Based
Atmospheric Monitoring. Environmental Manager. May 2012. 36-46 (2012).
4 Hall ES, Kaushik SM, Vanderpool RW, Duvall RM, Beaver MR, Long RW, and Solomon PA. Integrating Air Pollution
Sensors into Current Ambient Air Monitoring Networks: Practical Considerations, American Journal of Environmental
Engineering, (in press 2014).
5 Vallano, D., Snyder, E., Kilaru, V., Thoma, E., Williams, R., Hagler, G., Watkins, T. Air Pollution Sensors.
Highlights from an EPA workshop on the evolution and revolution in low cost participatory air monitoring.
Environmental Manager. December 2012. 28-33 (2012).
1
-------
performance and how to share that information with those at the cutting edge of low cost sensor
development. Such an endeavor might provide both immediate as well as long-term benefit to
all parties. In particular, the U.S. EPA might be in position to help facilitate improved low cost
sensor performance in this highly evolving area by sharing expertise in air quality monitoring. In
doing so, one might forecast faster development of improved air quality sensors, or a better
understanding of the capabilities of these devices being made readily available to the general
public.
1.2 Research Initiative
The U.S. EPA as part of its ACE research portfolio had an established program focusing
upon emerging technologies.6 Low cost sensor technology was an area of growing interest.
Based upon the recommendations of the ASAP 2012 and those associated with the ACE EM-3
research area (Emerging Technology), a research initiative was proposed that would integrate
the following components:
• Continue the dialogue between sensor developers and the U.S. EPA on sensor performance.
• Perform a market survey of low cost sensors associated with gas phase measurement of NC>2
and Os.
• Develop laboratory facilities and protocols that would be needed to evaluate such sensors for
basic performance characteristics.
• Establish a communication strategy for engaging sensor developers about the planned
laboratory research, and
• Seek out and then establish collaborative research agreements between the U.S. EPA and
interested sensor developers.
1.3 Call for Sensor Developer Applications
The U.S. EPA established a world-wide open call for sensor developers to submit an
application for inclusion in this research (http://www.epa.gov/airscience/air-sensor.htm).
Interested parties were requested to submit a statement of interest by June 30, 2012 and
provide basic information about their device. Interested parties were informed that due to
capacity constraints (resources), only a limited number of each sensor type would be accepted
for evaluation and that the scope of the evaluation would be limited to a practical
(environmentally-relevant) range of pollutant concentrations and environmental conditions (e.g.
humidity and potential interferences). Institutions submitting applications were informed that
they be invited to visit the U.S. EPA laboratories where the work would be conducted in the fall
of 2013 to discuss their instruments, the evaluation protocol, and receive a tour of the facility.
Applicants agreeing to the terms of the collaboration would receive information on the
performance of their device under known environmental conditions. Applicant's sensors were
asked to meet the following criteria:
• Technical feasibility to measure NC>2 and/or Os at environmentally relevant concentrations.
• Developers had at least some basic knowledge about their device's expected performance
characteristics that would minimize testing devices under disadvantageous circumstances
where the device might be damaged.
• No previous standardized evaluations under known challenge test conditions by any party.
6 Snyder, E., Watkins, T., Thoma, E., Williams, R., Solomon, P., Hagler, G., Shelow, D., Hindin, D., Kilaru, V.,
Preuss, P. Changing the paradigm for air pollution monitoring. Environmental Science and Technology, 47: 11369-
11377(2013).
-------
• Represent highly portable sensor and smart phone type applications featuring continuous
measurement capabilities.
1.4 Study Inclusion Criteria
The evaluation offered to applicants was intended to provide technical feedback to
developers of newer, low cost personal/portable sensors. It would not represent any U.S. EPA
endorsement of any device evaluated. The device would need to be fully operational at its time
of delivery and be at a development stage where it had its own power supply and
telecommunication tools (if applicable). The U.S. EPA requested that each applicant approved
for study inclusion:
• Provide clear, written, step-wise directions for operating the device (set-up/operation/data
recovery, etc) at its point of release to the US EPA. If such operating procedures were not
developed, they had to be agreeable to working quickly with the U.S. EPA on such
documentation.
• Provide access to the device for at least two months during the evaluation period.
• Provide self-powered devices capable of operation using internal (battery) power (either
replacement cells or rechargeable cells) and that if at all possible, it would need to operate at
a minimum of eight hours on internal power. If the device requires AC power for operation,
applicants were required to provide the necessary electrical transformer/connections.
• Provide a device housed within a break-resistant case or have its own external cover to permit
normal handling practices.
• Provide a device capable of accepting a wide range of expected temperature and relative
humidity conditions during testing and pose no intrinsic safety issues.
1.5 Collaborative Research Agreement
A total of nine applicants submitted proposals to join in the collaboration. Each of these
applicant's proposals were then thoroughly reviewed by the U.S. EPA technical staff as well as
officials associated with the FTTA to ensure applicable adherence to collaborative research
standards and confidential nature of the technology to be shared between parties. Applicants
were invited to the U.S. EPA'S Research Triangle Park, NC campus in October 2012 to tour the
facilities, discuss the technical nature of the proposed research, and to gain a fuller
understanding about the purpose of the collaboration and share their perspective about its
potential (http://www.epa.qov/nerl/features/sensors.html). Following this event, all nine of the
proposed applicants agreed to the formal collaborative (confidential) research with signatories
associated with each institution acknowledging the establishment of the MCRADAS. The
research reported here represents the first use of such agreements involving innovative
technology sharing between the U.S. EPA and various institutions.
The names of the collaborating MCRADA institutions and the findings reported in this
report represent the release on previously confidential and collaborative research data. Each
collaborating institution named in this report and technical information about their sensor and its
performance have agreed to public data sharing in a spirit of advancing sensor technology as a
whole and informing potential end users about the value and limitations of such current
technologies. This public data sharing was not a component of the MCRADA requirements and
the institutions providing this information are applauded for their willingness to share this
information with others.
-------
2. Study Objectives
2.1 Primary Goals
To address the issues discussed in the ASAP 2012 meeting, the U.S. EPA conducted an
Air Sensor/Application Evaluation and collaboration challenge. This challenge was viewed as a
high priority need for the U.S. EPA and one in which the NERL was asked to take a leadership
role in promoting. The NERL therefore sought out novel sensor technologies for the
measurement of ambient NC>2 and Os through a general appeal to inventors as discussed
earlier. These sensor types being selected due to their prevalence in the market place and the
availability of established test regimens that might be applied. The technology provided by the
Collaborator was temporarily transferred to the NERL where its performance was examined
under controlled laboratory conditions. The purpose of this collaboration was to provide the
NERL an opportunity to examine the emerging area of sensor technologies and to share
technical feedback to the Collaborator on the general performance characteristics of their
particular sensor as a means of advancing the general state-of-the-science.
2.2 Laboratory Testing
The NERL provided laboratory space, technical staffing, test atmosphere generation
equipment and reference analyzers for the research described below. Laboratory-based
research was initiated in October 2012 and completed in July 2013. The obtained data was
validated, tabulated and then summarized with a preliminary report on the sensor's general
performance characteristics produced during the summer 2013 and then shared with the
Collaborator for comment. A final report was provided to each collaborator in October 2013
following review by all parties and iterative co-development.
2.3 Specific Objectives
To achieve the primary goals discussed above, the specific objectives were to:
• Evaluate each transferred sensor being tested under known laboratory/chamber conditions
including relative humidity, temperature, pollutant challenge atmosphere and interfering
species concentration (e.g., NO2).
• Establish evaluation criteria for each device including: (a) linearity of response; (b) precision
at each known reference concentration; (c) determination of the lowest established
concentration in which a response was detected, (d) concentration resolution, (e) response
time, and (f) suggested range of operation to achieve best practical operation conditions. The
specifics of each criteria discussed above are defined in detail in the Study Approach section
of this report. Replicate trials were performed as a means to ensure data quality.
• Summarize data findings for a report detailing basic performance characteristic of each
sensor. These findings would describe not only how well a given sensor performed with
respect to a given challenge (e.g., response time), but also more generalized findings about
end user consideration. Of these, such topics as its ease of use were hoped to be defined.
-------
2.4 Secondary Goal Objectives
The aforementioned objectives were defined prior to the initiation of the collaborative
research. However, other secondary objectives became quickly evident as sensors were
received by the U.S. EPA and laboratory work was initiated. These secondary (but no less
important) objectives that needed to be achieved included:
• Development of more refined SOPs for many of the sensors to ensure the accurate operation
of the device and ensure data quality meeting the unique U.S. EPA Quality Assurance Project
Plan (QAPP) developed for this research. Each collaborating institution reviewed and
contributed to the QAPP prior to initiation of any data collection. While sensor developers had
some operating procedures in hand which they shared with the U.S. EPA, they sometimes
lacked sufficient details (e.g., photos, figures, guidelines) that instructed the end user. This
objective had to be achieved before QAPP-approved data could be collected.
• Refinement of communication or data/storage/gathering protocols. It was expected that data
recovery would be easily achieved since the sensors associated with this research were
available for others to use. However, each of the devices tested as part of this research often
had very unique protocols for how data was being gathered and communicated by the device.
Review of the SOPs revealed a wide range of needed systems to ensure timely and secure
data recovery. Some sensors stored data internally on SD cards, some directly transmitted
data to tablets or PCs. Other sensors used a variety of wireless communication protocols
(e.g., specific SIM card for cellular transmissions, local WiFi, Bluetooth, proprietary website
hosting). All of these protocols had to be considered in executing the study approach and
building up the necessary infrastructure to support the research. In some cases, multiple
iterations of either hardware or software had to be developed by both parties (U.S.
EPA/collaborating institution) to ensure data storage/recovery.
• Identification of sensor power requirements. The ability of the sensors to provide internal
power supplies (battery) sufficient to allow extended automated exposure chamber testing
was often lacking upon initial testing. Extended operations in the 12 to 24 hr range was
viewed as necessary to ensure replication of test conditions and efficient data collection.
Consequently, the sensor devices had to be re-wired from an internal power source to a direct
line source. This required working with sensor developers on the needed voltage and circuit
connections. In many cases, sensor developers provided direct wiring harnesses that met this
need.
-------
3. Study Approach
3.1 Background
As previously stated, one goal of this effort was to develop an understanding of what
technology might prove valuable in conducting the next generation of air monitoring. Upon their
discovery, such technologies were obtained in collaboration with inventors as well as
commercial and research organizations and examined under controlled laboratory conditions. If
found to be acceptable, devices might be recommended for incorporation in the next phase of
testing, short pilot studies involving direct environmental monitoring and comparison to
collocated reference methods.
The following evaluation criteria were established for each candidate sensor technology:
(1) linearity of response (range), (2) precision of measurements, (3) lowest established
concentration in which a response was detected (lower detectable limit [LDL]), (4) concentration
resolution, (5) response time, (6) interference equivalents, and (7) relative humidity (RH) and
temperature influences. This manuscript describes the experimental systems required to
assess the performance of such technologies.
3.2 Evaluation Procedures
3.2.1 Exposure Chamber
The exposure chamber (Ace Glassware) shown in Figure 1 was designed and
constructed for the evaluation testing. The internal diameter and length of the glass chamber
were large enough (15.2 cm X 91.4 cm) to accommodate at least one or more of the test
sensors. The chamber had four sampling ports spaced 3.8 cm apart. Reference analyzer
sample lines and sensor signal and power supply lines passed through Teflon-lined fitting caps
into the exposure chamber. Unused port positions were filled with solid Teflon plugs so that
laboratory air would not dilute the generated test atmosphere. Reference analyzer sampling
lines were made of 6.4-mm (0.25-in.) outer diameter (OD) perfluoroalkoxy (PFA) Teflon.
Particulate filters (5-um pore size) were fitted to each reference analyzer's inlet port. Air
containing known concentrations of the test atmosphere and/or interferents gas was provided to
the chamber inlet as needed to conduct the established protocols. An exhaust line was attached
to the chamber outlet and placed into the laboratory's 6-in. ceiling vent to allow a continuous
flow through design feature.
3.2.2 Physical Parameters (Temperature and Relative Humidity)
Temperature within the exposure chamber was controlled through the use of the
shelter's HVAC system and supplemented with heating pads and dry ice to obtain test
conditions. RH within the exposure chamber was controlled through the use of a de-ionized
water bubbler. Temperature and RH were measured with a temperature/RH probe designed by
Alion. The temperature sensor consists of a precision thin-film platinum 1000-Q resistive
temperature device (RTD) that employs a linear resistance change with temperature converted
to a 0-10 V DC output proportional to 0-100.0 °C. The sensor is calibrated (zero and span)
using a N 1ST-traceable reference thermometer. The RH sensor consists of a HyCal, Inc. IH-
3602-C monolithic integrated circuit capacitance sensor that produces a linear voltage
proportional to RH (0-10 V DC output directly proportional to 0-100% RH). The RH sensor is
calibrated using saturated salt solutions that have known RH over head space.
-------
Figure 1. Ace Glassware Exposure Chamber Showing Input and Output Ports and Caps
The temperature and RH signal response are shown on a liquid crystal display (LCD). Both
sensors share a common probe that was inserted into one of the ports of the sampling manifold,
typically at the end of the series of ports to lessen the chance of contamination from the probe's
stainless steel composition. The RH measurements require a minimal face velocity of 10 cm s~1
to be accurate. The analog output signals that correspond to the temperature and RH signals
were recorded by the data acquisition system during each experiment. Test atmospheres were
established using an NIST-traceable and programmable gas delivery system (Teledyne API
Model T700U) with constituents delivered to the system from either EPA reference gas
standards (862, NO2) or independent generation device (Os). Dilution air that had been
scrubbed of particulate matter, moisture, and hydrocarbons was delivered to the mixing system
to meet test gas dilution and chamber flow through needs.
3.2.3 Continuous Gas (Reference) Monitors
Samples for the continuous reference analyzers (NO/NO2/NOX, Os, and SO2) were
drawn to the monitors through the previously described sample ports and PFA lines at flow rates
from 500 to 1000 mL/min. Flow measurements were taken at the beginning of each test run to
ensure the total sampling flow requirements of the instruments did not exceed that of the source
atmosphere introduced into the sampling manifold during the test. The continuous analyzers
used in this study are given in Table 1.
Table 1. Reference Analyzers Used in Sensor Evaluation Study
Pollutant
NO2
03
SO2
Analyzer
Thermo Model 42C NO/NO2/NOX Analyzer
2B Model 205 O3 Analyzer
Thermo Model 43C SO2 Analyzer
Principle of Operation
O3-chemiluminescence
UV Absorption
Pulsed Fluorescence
-------
The operating procedure for the NC>2 analyzers is based on the NC>2 Federal Reference Method
(FRM)7 and recommendations given in the manufacturer's operator's manual. Likewise, the
operating procedure for the SC>2 analyzers is based on the SC>2 FRM8 and recommendations
given in the manufacturer's operator's manual. The operating procedure for the UV Os
analyzers (a Federal Equivalent Method [FEM]9) is based on the standard EPA ambient
measurements of Os and recommendations given in the manufacturer's operator's manual.
Data from the reference analyzers and a temperature and humidity monitor were continuously
recorded by the data acquisition system. The sensors' data were transmitted either by Blue
Tooth or USB/RS-232 directly to their special application software or smart phone applications.
The date, time, and results of each novel sensor test were documented.
3.2.4 System Characterization
Prior to initiating the sensor tests, the overall system was evaluated through a series of
experiments. The results of these characterization runs indicated the system was capable of
obtaining and maintaining (over a multiple day period) a temperature range of <5 °C->45 °C. In
addition, RH control in the range of 20% to near 100% was achieved. It was also determined
that the system residence time (amount of time it takes for a change in a system parameter to
be recorded by the corresponding reference instrument) was less than 5 minutes.
3.2.5 Sensor under Test Samples
The sensor(s) undergoing testing was placed inside the glass exposure chamber after
following start-up procedures found in either a standard operating procedure (SOP) or research
operating procedure (ROP) provided by each participating organization. Samples were drawn to
the reference monitor through a sample pump/fan located inside the instrument or by exposure
of its internal sensor to the chamber atmosphere. Positioning (orientation) of the sensor in the
test chamber was defined by the manufacturer to ensure representative data collections under
normal operating procedures. A list of the quality assurance procedures used in the execution
of this effort are reported in Appendix C.
3.3 Test Procedures
The following evaluation criteria were established for each device under each test
condition: (1) linearity of response (range), (2) precision of measurements, (3) lowest
established concentration in which a response was detected (lower detectable limit [LDL]), (4)
concentration resolution, (5) response time, (6) interference equivalents, and (7) RH and
temperature influences. A minimum of two RH conditions (e.g., dry air < 25% RH, humid air>
85% RH) were incorporated into the evaluation runs as well as a minimum of two temperature
ranges (e.g., low near 0 °C, high > 50 °C).
3.3.1 Linearity (Range)
Definition: Nominal minimum and maximum concentrations that a method is capable of
measuring.
7 40 CFR Part 50 Appendix F - Measurement Principle and Calibration Procedure for the Measurement of Nitrogen
Dioxide in the Atmosphere (Gas Phase Chemiluminescence)
8 40 CFR Part 50 Appendix A-1 - Measurement Principle and Calibration Procedure for the Measurement of Sulfur
Dioxide in the Atmosphere (Ultraviolet Pulsed Fluorescence Method)
9 2B Technologies Model 202 and 205 Ozone Monitors- Automated Equivalent Method: EQOA-0410-190 Federal
Register: Vol.75, pages 22126-22127, 04/27/2010
-------
Test procedure: Determine a suitable calibration curve showing the test analyzer's
measurement response over at least 95% of the required or indicated measurement range (e.g.,
0-500 ppb).
3.3.2 Precision of Measurements
Definition: Variation about the mean of repeated measurements of the same pollutant
concentration, expressed as 1 standard deviation.
Test procedure: Precision of measurements was evaluated by conducting three
replicates: (1) Sample test atmosphere Ao (zero air). (2) Measure test atmosphere, Ap, and
record the stable reading (in ppb) as PL (3) Repeat the second step two more times, record
values for P2 and PS after each test, and calculate precision (P) as follows:
P =
t=l
t=l
3.3.3 Lower Detectable Limit (LDL)
Definition: Minimum pollutant concentration that produces a measurement or
measurement output signal of at least twice the noise level.
Test procedure: Measure zero air and record the stable measurement reading in ppb as
Bz. Generate and measure a pollutant test concentration equal to the value for the lower
detectable limit specified in Table B-1 to Subpart B of Part 53 (table shown in Appendix A of
this report). Record the test analyzer's stable measurement reading (in ppb) as B/.. Determine
the LDL test result as LDL = BL- Bz.
3.3.4 Concentration Resolution
Definition: Smallest amount of input signal change that the instrument can detect
reliably. This term is determined by the instrument noise. Noise is the spontaneous, short-
duration deviations in measurements or measurement signal output about the mean output that
are not caused by input concentration changes. Measurement noise is determined as the
standard deviation (S) of a series of measurements of a constant concentration about the mean
and is expressed in concentration units.
Test procedure: (1) Measure zero air with the test analyzer. Record 10 test analyzer
concentration measurements with at least 2 minutes separating successive measurements.
Label and record the test measurements as ri, r2, K. . . r\. . .rio. (2) Repeat step 1 using a
pollutant test atmosphere 80% of the upper range limit (URL). Calculate So and Sso as follows:
S =
n-l
10
t=l
t=l
-------
3.3.5 Response Time
3.3.5.1 Lag Time
Definition: Time interval between a step change in input concentration and the first
observable corresponding change in measurement response.
Test procedure: Determine the elapsed time between the change in test concentration
from zero air to 80% URL and the first observable (two times the noise level) measurement
response.
3.3.5.2 Rise Time
Definition: Time interval between initial measurement response and 95% of final
response after a step increase in input concentration.
Test procedure: Calculate 95% of the measurement reading and determine the elapsed
time between the first observable (two times noise level) measurement response and a
response equal to 95% of the reading.
3.3.6 Interference Equivalent
Definition: Positive or negative measurement response caused by a substance other
than the one being measured.
Test procedure: (1) Sample and measure test atmosphere zero air. Allow for a stable
measurement reading and record the reading as R (in ppb). (2) Sample and measure the
interferent test atmosphere and record the stable reading in ppb as RI. (3) Calculate the
interference equivalent (IE) as IE = RI - R.
The test analyzers were challenged, in turn, with each potential interfering agent
(interferent) specified in Table 2 at a concentration substantially higher than that likely to be
found in the ambient air.
Table 2. Interference Test Atmospheres for Apps/Sensors
Interferent
Concentration
(ppb)
Os Sensors
SO2
NO2
>200
>200
NO2 Sensors
SO2
03
NO
>200
>200
>200
10
-------
3.3.7 Relative Humidity and Temperature Influences
3.3.7.1 Relative Humidity (RH)
Test procedure: Determine response to a known test concentration under dry (RH <
25%) and high humidity (RH > 85%) conditions.
3.3.7.2 Temperature
Test procedure: Determine response to a known test concentration under low (near 0
°C) and high (> 50 °C) temperature conditions.
11
-------
4. Quality Assurance
4.1 Development of QA/QC Materials
The U.S. EPA conducted all of the research presented in this report through an
integrated program of Quality Assurance/Quality Control (QA/QC) activities. Principals of each
major QA/QC activity are defined below. Quality assurance protocols were co-developed
between the U.S. EPA and the sensor developers. The documents had to be crafted so that
trained technicians knowledgeable about general air quality instrumentation procedures could
operate the devices and that non-trained QA representatives fully understood the intent of each
protocol. In other words, the SOPs had to be written so that others, even those not associated
with the immediate work but who had some scientific background, could understand how to
operate the devices. SOPs were developed, reviewed and ultimately approved by multiple U.S.
EPA staff members including the representative NERL Quality Assurance Manager.
The aforementioned SOPs were but one component of an U.S. EPA Quality Assurance
Project Plan (QAPP; NERL-QAPP-AB-12-03) entitled "ORD Sensor/Application test bed
challenge: Investigation of apps/sensors response under controlled laboratory conditions." The
QAPP defined such practical guidelines as defining those who would be involved in the
research (e.g., named U.S. EPA, Alion Scientific contract staff, sensor development
collaborators), data quality objectives (those that could be defined as in many cases the basic
performance characteristics of the device was unknown for a given area); the programs to be
ensure successful data collection/storage and integrity; data generation and analytical methods
to be employed (especially with respect to the utilization of FRM and FEM collocated monitor
and test chamber characterization); statistical treatment of collected data; and parties
responsible for overall project and data management. A full listing of relevant QA
documentation is provided in Appendix C.
4.2 Data Collection
Specifics of the study approach and how replicate data collections were used to address
data collection measurement uncertainty is defined elsewhere. At a minimum, replicates were
typically performed at least twice for each challenge condition and in the vast majority of cases
these were performed in triplicate. Laboratory and electronic notebook entries were made
simultaneous with each data collection event to fully document the test conditions. Digital
photographs were obtained showing each sensor in the test chamber and how it was positioned
to document its vertical and horizontal physicality. Raw data was processed following its
transfer into Microsoft Excel (v 2007). Second party review of data collection/transfer, record
keeping, as well as the summary data analyses performed using this software was performed
through trained QA staff (Alion Science and Technology).
4.3 QA Systems Audit
In addition to the QA oversight described immediately above, a formal QA systems audit
was performed by the NERL's QA Manager in January 2013. This audit involved an in-person
inspection of the test facility, the FRM/FEM instrumentation procedures being used to compare
individual sensor response, observance of actual sensor challenge operations and adherence to
SOPs/ROPs and specific QAPP requirements. Findings from the system audit indicated a high
degree of compliance with all SOPs/ROPs and QAPP requirements. A recommendation was
made that more extensive written documentation of daily test conditions in the laboratory
12
-------
notebook be performed which would be advantageous to the project and which would
supplement the electronic records already being obtained. The recommendation was
immediately implemented. This was not a negative finding but rather one which the auditor
indicated would minimize any future issue with describing test conditions and if needed, provide
the means to more easily reproduce the work.
4.4 Data Inclusion Process
Following the completion of data collection and its summarization, all of the MCRADA
collaborators received full copies of the raw data used to summarize findings relating to their
specific sensors. Data inclusion criteria were defined (revisited) and if for any reason data were
voided or excluded from inclusion the reason for such events were described. Sensor
developers were asked to review the statistical findings (tabular and graphical representations)
and approve the results as being fully representative of meeting the study objectives and
performance characteristic definitions established in the Study Approach and QAPP.
Collaborators were requested to provide alternative and other complimentary data analyses
using the raw data provided. Only one such event occurred. Review of the raw data from one
collaborator resulted in what was previously considered to be a negative integer (negative
concentration) in the raw data output and which was actually a delimiter and not defined to the
U.S. EPA prior to initiation of the data collection as non-value column descriptor. The
collaboration institution summarily performed the needed statistical calculations and graphics
development to correctly report findings associated with that given sensor treatment.
13
-------
5. Sensor Description, Results and Discussion
The specifications of each sensor evaluated in this challenge are summarized in detail in
Table 14. While some sensors had the capability to measure multiple pollutants, only NC>2 and
Os measurements and performance criteria were evaluated in this work. Comparative
information on acceptable FRM/FEM instrumentation performance characteristics for each
pollutant as listed in 40 CFR Part 53 Table B-1 (see Appendix A) is provided as the last entry on
each of the accompanying tables for the individual sensor tests.
It should be noted that in the figures for this and all other NO2 sensors, the FEM
employed to record the chamber conditions had a maximum reporting value of 200 pbb.
Therefore, there were situations where the test sensors yielded a response well in excess of the
FEM ceiling. This gives the appearance in the graphs that the sensors were reporting a
significant positive bias at the upper challenge concentration which would not be correct. Only
those values between 0 and 200 ppb were actually used in the development of statistics for this
and all other NO2 sensors. Actual sensor response is being shown here to reflect the ability of
the sensor to actually respond to challenge conditions at or near 400 ppb.
5.1 Ozone Sensors
5.1.1 AGT Environmental Sensor
The AGT Environmental Sensor10 received for evaluation and as described by the
collaborator, was designed for outdoor use by pedestrians and/or bicyclists and provides for
enhanced environmental measurements in urban areas and in the vicinity of industrial areas11.
The sensor device can be used to monitor air quality by measuring traffic pollutants and
industrial pollutants. This sensor is capable of measuring carbon monoxide (CO), carbon
dioxide (CO2), NO2, Os, particulate matter (PMio), temperature, humidity, noise, atmospheric
pressure and solar radiation (UV). Size and weight is comparable to other handheld devices
(e.g. GPS devices) and it is powered by batteries and solar panels. Pollutants like ozone,
carbon monoxide and nitrogen dioxide are detected using metal oxide semiconductor (MOS)
sensors. Air quality is sensed with a MOS based component to detect a broad range of VOCs
and correlates these measurements directly with CO2 levels in the room. Other environmental
parameters like temperature, humidity and atmospheric pressure are monitored using a
combined temperature/humidity sensor and a barometer. In addition, an ambient light sensor
can sense visible light, and the ultraviolet (UV) sensor is a broadband UVA-UVB-UVC sensor,
sensitive to wavelengths from 220 to 370 nm. Noise levels, measured in decibels (dB), are
detected using a microphone and electronics configured to measure sound using A-weighting.
In order to achieve fast response of concentration measurement, outside air needs to be
transported to all sensor components inside the housing. Therefore, fans are used to create this
airflow through the device. The primary use of this airflow is to pull in ambient air and deliver it
to the internal sensor components. The integrated 2000 mAh Lithium-Polymer (Li-Po) battery
offers 4-8 hours of autonomous device operation - depending on measurement and sampling
10 Florian Zeiger, Marco F. Huber; Demonstration Abstract: Participatory Sensing enabled Environmental Monitoring
in Smart Cities, 13th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN
2013), Berlin, Germany
11 AGT's sensor was developed within AGT's Environmental Sensing project. Acknowledgements to Zacharias
Boufidis, Marco Huber, Kostas Sasloglou, Georgios Mazarakis, Birte Ulrich, Ashok Kumar Chandra Sekaran, Roel
Heremans, Andreas Merentitis, Nikolaos Frangiadakis, Martin Strohbach, Kerron Soothe, Christian Debes, Maria
Niessen, James Rex, Environmental Sensing scrum team, Rutan GmbH, and the TECO research group
14
-------
intervals. Solar cells are used for energy harvesting from the sun light contributing to the
device's energy budget. It can be charged using a micro-USB cable. The device was also
equipped with a sensor for particulate matter.
The AGT Environmental Sensor has two buttons for standard operations and all detailed
configuration settings are performed via Smartphone app. The measurement range of the
internal sensor components were designed so that the device can be used in different settings
to enable citizens to monitor a variety of air quality conditions. The sensor device has no GPS
receiver or display and is designed to be used with a Smartphone for data transmission, adding
location/timestamp and sensor device configuration as part of its output. Communication
between Smartphone and the AGT Environmental Sensor is done via Bluetooth. This system
uses participatory sensing12, and thus enables this powerful technology for larger scale
environmental monitoring systems in the smart city context.13
The AGT Environmental Sensor was able to participate in all conducted tests. The
sensor system showed relatively good response times and small lag times during the tests (~ 1
min). Also the ozone response was highly linear (R2>0.98). It would appear that the sensor
showed the greatest degree of response variability (37.7 ppb) under challenge conditions
involving hot temperatures and high Os concentrations. Modest SC>2 positive interference was
observed (7.5 ppb).
Testing was completed for the AGT Environmental Sensor under all conditions. Note
that interference testing was performed only under normal conditions. Examples of the sensor's
response under normal challenge conditions are shown for Os in Figure 2. The response of the
AGT sensor is observed to be highly consistent with that of the reference monitor/trace over the
full range of challenge concentrations.
Table 3. Summary of AGT Environmental Sensor Testing
Analyte
03
CFROs
Conditions
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
0.9824
0.9933
0.9774
0.9772
NA
Precision
(ppb)
10.3
13.6
2.6
7.2
10
LDL
(ppb)
15.6
12.4
12.4
9.8
10
IDL
(ppb)
11.8
18.1
16.0
11.3
10"
Resolution
Low
(ppb)
8.3
6.8
5.9
2.6
5
High
(ppb)
14.1
37.7
4.0
6.1
5
Lag
Time
(min)
1
1
1
1
20
Rise
Time
(min)
5
6
4
3
15
S02
Int*
(ppb)
7.5
20
03
Int*
(ppb)
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were available
to establish the response comparison.
A Value specific to the 2B Model 205 Os Analyzer
12 Demonstration Abstract: Participatory Sensing enabled Environmental Monitoring in Smart Cities, Florian Zeiger,
Marco F. Huber. Accepted for the 13th ACM/IEEE International Conference on Information Processing in Sensor
Networks (IPSN 2013), Berlin, Germany
13 F. Zeiger and Z. Boufidis, "Towards Future Internet services through crowdsourcing-based sensor platforms,"
International Journal of Communication Networks and Distributed Systems, vol. 11, no. 1, pp. 4-10, Jun. 2013.
15
-------
AGTATS-35O3 Normal
•2BO3(ppb)
•AGT Response (ppb)
500.00
400.00
0.00
-100.00
Data Points
Figure 2. Example AGT ATS-35 Response for Os under Normal Challenge Conditions
5.1.2CairClip
The CairClip was composed of an amperometric sensor, a micro fan that enables
dynamic air sampling, a patented filter, and a highly-sensitive nanoamperes detection circuit.
Data are stored in an integrated datalogger capable of retaining more than 28,000 points (1-min
average). Sensor components were integrated into an aluminium-based casing cylinder (32 x 62
mm).
Testing was completed for the CairClip sensor under all conditions. The data are
summarized in Table 4. Data could not be processed for the Os hot conditions because of a
naturual increase in response due to temperatures and the sensor type (amperometric). Note that
interference testing was performed only under normal conditions. Examples of the sensor's
response under normal challenge conditions are shown for Os in Figure 3.
Results obtained during this study were comparable to those observed in Cairpol internal
test evaluations.14 Excellent linearity over the full operating range was observed (R2>0.99).
Limits of detection (<11 ppb) were evident with excellent precision observed under both high
and low challenge conditions (<4 ppb).
Lag and rise time were higher than those observed in internal Cairpol testing where
values typically < 3 mins have been obtained. These differences may be linked to the time it
takes for the sensor to condition itself with Os under test chamber conditions employed here as
compared to evaluations Cairpol has performed under ambient conditions.
As seen in Figure 3, the CairClip as tested showed highly similar trends with the
reference monitor but always with a small obvious positive effect. The observed offset is due to
the device's internal calibration coefficient (slope) that Cairpol uses to compensate for the lack
14 Zaouak, O., Aubert, B., Castang, J-B. Cost-efficient Miniature Sensors for Network Continuous Monitoring of
Diffuse Pollution at the Low ppbv Level. Invited Presentation to the 2013 Air Sensors Workshop. Research Triangle
Park, NC, March 20, 2013.
16
-------
of sensitivity of the electrochemical sensor to NC>2 in comparison to its sensitivity to Os (being
responsive to both
Cairpol has also suggested that the positive effect might be the result of a lack of
adequate conditioning time prior to start of the chamber trials. Exposure to zero grade air (as
was the case here) requires the sensor to become "disconditioned" due to a lack of oxidizing
species interacting with the sensor membrane. The need for proper conditioning is a noted
feature of this sensor for best response.
To date, no disconditioning phenomenon have been observed under real conditions
during actual monitoring events.15
Table 4. Summary of CairClip Sensor Testing
Analyte
03
CFROs
Conditions
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
0.9958
NA
0.9989
0.9905
NA
Precision
(ppb)
4.6
NA
2.8
9.5
10
LDL
(ppb)
10.8
NA
8.6
8.6
10
IDL
(ppb)
4.3
NA
4.6
3.9
10A
Resolution
Low
(ppb)
1.7
NA
1.7
2.6
5
High
(ppb)
3.4
NA
2.4
3.7
5
Lag
Time
(min)
1
NA
0
1
20
Rise
Time
(min)
4
NA
8
6
15
SO2
Int*
(ppb)
0.37
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were
available to establish the response comparison.
A Value specific to the 2B Model 205 Os Analyzer
CairClip O3 Normal
•2BO3(ppb) ^—CairClip O3 (ppb)
Data Points
Figure 3. Example CairClip Response for Os under Normal Challenge Conditions
15 Zaouak, O., Aubert, B., Castang, J-B. Cost-efficient Miniature Sensors for Network Continuous Monitoring of
Diffuse Pollution at the Low ppbv Level. Invited Presentation to the 2013 Air Sensors Workshop. Research Triangle
Park, NC, March 20, 2013.
17
-------
5.7.3C/f/Sense
Preliminary testing of the CitiSense with 500 ppb Os yielded no response. As such, all
further testing with Os was canceled. The believed failure of the device to respond to an initial
challenge was that related to an aged electrochemical sensor (> 2 years of age). This is not an
unexpected result as many MOS sensors have a limited lifespan regardless of their degree of
use during such a time period. A full description of the CitiSense device and its properties are
reported in section 6.2.4.
Table 5. Summary of CitiSense Testing
Analyte
03
CFROs
Conditions
NA
NA
Linearity
(R2)
NA
NA
Precision
(ppb)
NA
10
LDL
(ppb)
NA
10
IDL
(ppb)
NA
10A
Resolution
Low
(ppb)
NA
5
High
(ppb)
NA
5
Lag
Time
(min)
NA
20
Rise
Time
(min)
NA
15
S02
Int*
(ppb)
NA
20
03
Int*
(ppb)
NA
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were
available to establish the response comparison.
AValue specific to the 2B Model 205 Os Analyzer
5.1.4 Dynamo
Testing was completed for the Dynamo sensor under all conditions. The data are
summarized in Table 6. Note that interference testing was performed only under normal
conditions. An example of the sensor's response for Os under normal challenge conditions is
shown in Figure 4. The active range of the Dynamo sensor as reported by the developer is 10
ppb to 250 ppb. Any exposure to Os levels outside of this range would not be expected to be
reported accurately. The Dynamo sensor was designed for use on mobile platforms (trucking
fleets, etc.) that traverse areas expected to have Os concentrations well within this designated
range.
The Dynamo sensor typically can respond to a range of resistance values as Os
concentrations vary. The sensor was factory calibrated with exposure to 100 ppb of Os, and the
level of response set to equate to this concentration. The Os concentration was then reduced to
10 ppb, and the response adjusted to reflect this concentration response. These two factory
adjustments give the correct scaling and slope over the main range of interest (10-100 ppb).
While the typical factory calibration is a two point challenge (10 and 100 ppb), a much
larger range was applied in our testing (0 to 400 ppb). As observed in Table 6, acceptable
linearity was observed over the full range (R2> 0.95) but with significantly reduced response
characteristics as a result of the factory calibration procedure. Even so, data revealed a multi-
linearity curvature of the response and as stated above, this feature might have been related to
the two point factory calibration pre-set into the device. Good precision (< 7 ppb) over all test
conditions were observed. It should be noted that since the Dynamo is used in mobile data
collection scenarios, short lag and rise times are paramount for good performance
characteristics. As seen above, times < 5 min were observed. We did note minor positive SO2
(2.9 ppb) and sizeable NO2 interference (15.6 ppb) under the test conditions.
Further slope linearity and sensor response improvements have taken place in
factory since the time the Dynamo sensor unit was submitted for EPA testing.
18
-------
Table 6. Summary of Dynamo Sensor Testing
Analyte
03
CFROs
Condition
s
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
0.9466
0.9885
0.9795
0.9820
NA
Precision
(ppb)
7
4.3
3.3
6.1
10
LDL
(ppb)
11.5
11.7
9.9
10.9
10
IDL
(ppb)
4.1
5.4
6.0
4.4
10A
Resolution
Low
(ppb)
0
3.6
2.0
0
5
High
(ppb)
11.7
5.3
3.8
5.5
5
Lag
Time
(min)
1
1
1
1
20
Rise
Time
(min)
2
5
4
5
15
S02
Int*
(ppb)
2.9
20
N02
Int*
(ppb)
15.6
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were
available to establish the response comparison.
A Value specific to the 2B Model 205 Os Analyzer
Dynamo O3 Normal
•2BO3(ppb)
•Dynamo O3 (ppb)
500.00
450.00
400.00
v 350.00
| 300.00
« 250.00
^ 200.00
150.00
100.00
50.00
0.00
Data Points
Figure 4. Example Dynamo Response for Os under Normal Challenge Conditions
5.1.5U-Pod
Testing was performed for the U-Pod under normal and cold conditions. Data points
were produced every 5 seconds and included battery voltage; temperature; humidity; light
sensor output; and 63 sensor output. Gas sensor data are reported as the output voltage of the
e2v sensors. These values were then divided into a constant, 10,000, which was chosen
arbitrarily to achieve a positive correlation with set-point concentrations.
The Os sensor used in the U-Pod is a commercially available MOS that retails for $9 and
is about the size of a large peanut (SGX Sensortech MiCS-2611). This sensor has previously
been tested by the Hannigan group under a variety of ambient conditions, but never in a lab
19
-------
setting at the tested concentrations and dynamics. The sensor is known to suffer from cross-
sensitivity to other pollutants, to ambient conditions, and baseline drift. However, in 2-week co-
location calibrations of multiple MiCS-2611 sensors with Colorado Department of Public Health
and the Environment operated regulatory monitors; the sensors had median standard errors of
6.1 ppb16, on the order of the reported LDL here. Improved precision at lower temperature has
been previously observed in testing, and this effect could likely be overcome by controlling
sensor voltage. Converting sensor signal to concentration thus requires the use of multilinear
regression, which increases calibration time and analysis complexity.
Findings in the current testing indicate that under normal and cold test conditions, a
linear response (R2>0.88) was obtainable. The precision error under normal challenge was
somewhat high (46.2 ppb) but it did provide respectful LDL and IDL values (<11.9). The device
produced no measurable SC>2 interference.
The data are summarized in Table 7. Note that interference testing was performed only
under normal conditions. Proper conditioning was not achieved in two of the tests (Os hot, Os
humid) over the length of an entire test, and the variation between calibration sequences was
such that the data could not be used for statistical analysis. In such cases, the sensor failed to
provide an equilibrated starting output following installation into the chamber at the challenge
condition. The primary source of these problems is hypothesized to be conditioning times of the
sensor under chamber testing conditions.
The U-POD was received with an acrylic casing. Data we report here reflects testing
performed with no casing surrounding the sensor board. Extensive laboratory trials were
conducted that initially provided reduced sensor response or distorted response (data not
shown). It is hypothesized that the acrylic case to the U-Pod was absorbing ozone at a rate
sufficient to depress the levels present in the test chamber and that this rate requires time on
the order of hours to reach a state of equilibrium. We subsequently removed the case and
tested the sensor successfully over the full challenge range.
Table 7. Summary of U-Pod Testing
Analyte
03
CFROs
Conditions
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
0.8775
NA
NA
0.9546
NA
Precision
(ppb)
46.2
NA
NA
6.5
10
LDL
(ppb)
3.4
NA
NA
7.8
10
IDL
(ppb)
11.9
NA
NA
0.9
10A
Resolution
Low
(ppb)
0.3
NA
NA
0.1
5
High
(ppb)
1.6
NA
NA
3.6
5
Lag
Time
(min)
3
NA
NA
1
20
Rise
Time
(min)
8
NA
NA
5
15
S02
Int*
(ppb)
0
20
03
Int*
(ppb)
NA
20
*Blank cells indicate that interference testing was not conducted under that condition . NA indicates no data were
available to establish the response comparison.
A Value specific to the 2B Model 205 Os Analyzer
16 Piedrahita, R., Xiang, Y., Masson, N., Ortega, J., Collier, A., Jiang, Y., Li, K., Dick, R., Lv, Q., Hannigan, M., Shang,
L. The next generation of low-cost personal air quality sensors for quantitative exposure monitoring. Atmospheric
Measurement Techniques Discussion. 7; 2425-2457, 2014. Doi:10.5194/amtd-7-2425-2014.
20
-------
5.2 Nitrogen Dioxide Sensors
5.2.1 AGT Environmental Sensor
Detailed information about the AGT Environmental Sensor is provided previously in
Section 6.1.1. Primary testing was completed for the AGT environmental sensor under all
conditions. The data are summarized below in Table 8. Note that interference testing was
performed only under normal conditions. Examples of the sensor's response under normal
challenge conditions are shown for NC>2 in Figure 5. It should be noted that in the figures for this
and all other NC>2 sensors, the FEM employed to record the chamber conditions had a maximum
reporting value of 200 pbb. Therefore, there were situations where the test sensors yielded a
response well in excess of the FEM ceiling. This gives the appearance in the graphs that the
sensors were reporting a significant positive bias at the upper challenge concentration which
would not be correct. Only those values between 0 and 200 ppb were actually used in the
development of statistics for this and all other NC>2 sensors. Actual sensor response is being
shown here to reflect the ability of the sensor to actually respond to challenge conditions at or
near 400 ppb.
The device provided excellent linearity over the test range (R2> 0.99) with a very high
degree of precision (< 7.5 ppb). Detection limits were on the order of 10-20 ppb. It should be
noted that while the device reported concentrations in good agreement with reference monitors
in the 0 to 200 ppb (Figure 5), an artificial and distinct positive bias was evident under high
challenge conditions followed by a sharp drop to more appropriate concentration estimates.
There can be several reasons for this behavior. Taking into account the sensor characteristics
(Figure 6), disturbances in the 100-200 ppb concentration range can have significant effects on
the output of the sensor. One candidate for such a disturbance can be for example the
recharging of the battery which leads to non-constant temperature situations inside the sensor
case - and thus, can directly influence the raw sensor output. Nevertheless, this behavior is
subject to future research as we believe it can be resolved.
The device had fast lag times (~ 2/min) but rise times increased under hot challenge
conditions (20 min). A distinct positive SO2 bias was observed (19.5 ppb).
AGT ATS-35NO2 Normal
•Thermo 42 NO2 (ppb)
•ACT NO2 (ppb)
o
Q.
V)
01
cc
800.00
700.00
600.00
500.00
400.00
300.00
200.00
100.00
0.00
-100.00
Data Points
Maximum Response of
Reference Monitor
Figure 5. Example AGT ATS-35 Response for NO2 under Normal Challenge Conditions
21
-------
10000
1000
£ 100
10
10 100
NO2 concentration [ppb]
1000
Figure 6. Sensor Characteristics for AGT NO2 Sensor
Table 8. Summary of AGT Environmental Sensor Testing
Analyte
N02
CFR
N02
Conditions
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
0.9972
0.9919
0.9937
0.9917
NA
Precision
(ppb)
1.2
6.4
7.4
7.5
10
LDL
(ppb)
15.0
13.6
17.7
10.2
10
IDL
(ppb)
9.5
24.0
22.8
5.2
10A
Resolution
Low
(ppb)
1.8
5.7
2.7
0.8
5
High
(ppb)
2.3
8.1
5.2
6.8
5
Lag
Time
(min)
1
1
1
1
20
Rise
Time
(min)
5
20
7
6
15
S02
Int*
(ppb)
19.5
20
N02
Int*
(ppb)
NA
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were
available to establish the response comparison.
A Value specific to the Thermo Model 42C NO/NCb/NOx Analyzer
5.2.2 AirCasting
We experienced numerous lost data collection episodes in our testing of the AirCasting
monitor. Eventually, resources limited us to findings associated with NC>2 normal testing
conditions. We also experienced several spontaneous software crashes that impacted the
availability of data used in this analysis. In addition, we sometimes had difficulty establishing
and maintaining the Bluetooth connection between the AirCasting monitor and the Android
device. This might have been due to the signal being blocked by the walls of the test chamber
and the supporting testing hardware and not a feature of the device itself under normal ambient
operating conditions.
22
-------
Chamber tests performed by the developer have been previously reported.17 The
collaborator had determined that the MiCS-271 metal-oxide sensor provided reasonable
NC>2 response curves. It was noted that the response curves were not as sharp as
those produced by the tests reported here in the EPA laboratories. This might well have been
due to the small chamber system they used and which might have had inadequate mixing of the
challenge gas. The sensitivity response performed directly by the AirCasting collaborator
appeared to be slightly higher as compared to those in our EPA-performed tests. The in-house
AirCaster tests did however, note a word of caution. It was apparent that high out of the box
variability existed between NC>2 sensors from the manufacturer and therefore there is a need to
individually calibrate each sensor. In-house AirCasting tests also indicated the sensitivity of the
MiCS-2710 NC>2 diminished as the NC>2 concentration climbed above 100 ppb. Supporting
material for the sensor can be found in the reference section.18'19'20
The data are summarized below in Table 9. An example of the sensor's response to NC>2
under normal challenge conditions is shown in Figure 7.
As can be seen, the device provided good linearity (R2> 0.98) over the full test range
with excellent precision (3 ppb). It yielded extremely fast rise and lag times (< 1 min). As
depicted in Figure 7, the device showed excellent reproduction of the reference monitor
response until challenge concentrations exceeded 100 ppb.
Table 9. Summary of AirCasting Testing
Analyte
N02
CFR N02
Conditions
Normal
NA
Linearity
(R2)
0.9846
NA
Precision
(ppb)
3.0
10
LDL
(ppb)
11.6
10
IDL
(ppb)
14.6
10A
Resolution
Low
(ppb)
1.1
5
High
(ppb)
1.0
5
Lag
Time
(min)
0
20
Rise
Time
(min)
4
15
NA indicates no data were available to establish the response comparison.
A Value specific to the Thermo Model 42C NO/NCb/NOx Analyzer
17 AirCasting test results and analysis:
http://www.takingspace.org/evaluating-low-cost-gas-sensors/.
18 AirCasting instruction manual:
http://www.habitatmap.org/habitatmap docs/HowToBuildAnAirCastinqAirMonitor.pdf
19 AirCasting website: http://aircastinq.org/
20 AirCasting app download: https://plav.qooqle.com/store/apps/details?id=pl.llp.aircastinq&hl=en
23
-------
AirCasting NO2 Normal
•Thermo 42C (ppb) AirCasting (ppb)
250.00
200.00
Data Points
Maximum Response of
Reference Monitor
Figure 7. Example AirCasting Response for NC>2 under Normal Challenge Conditions
5.2.3 CairClip
Primary testing was completed for the CairClip sensor under all conditions. The data are
summarized in Table 10. Note that interference testing was performed only under normal
conditions. Examples of the sensor's response under normal challenge conditions are shown for
NC>2 in Figure 8.
As for the Os tests reported here, similar results as those obtained during in-house
Cairpol chambers tests were obtained. Excellent linearity was observed (R2> 0.99) with good
precision (< 9.3 ppb). It must be recognized that this Os/NCb sensor tested here is by nature,
less sensitive to NC>2 as compared to Os. That also explains why the value concerning Os
exposure was slightly higher than expected at the high test conditions (concentrations >150
ppb). The value of the calibration coefficient applied to the device's response has been
knowingly increased by Cairpol to compensate for this difference in sensitivity between the two
gases. We observed minimal SO2 interference (0.58 ppb) with this sensor.
Lag and rise time values on the order of 0-8 minutes, are higher than those generally
observed in internal Cairpol chamber testing. As can be seen in Figure 8, there was a constant
shift regarding the lag time and the rise time between the sensor and the reference monitor
output. A disfunction in the Cairsoft (software used for data downloading) has since been
detected as a result of the data provided here and is believed to have been responsible for this
shift. Cairpol indicates it has subsequently corrected this software function.
24
-------
Table 10. Summary of CairClip Sensor Testing
Analyte
NO2
CFR NO2
Conditions
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
0.9968
0.9978
0.9975
0.9896
NA
Precision
(ppb)
3.8
2.9
4.2
9.3
10
LDL
(ppb)
7.0
0
8.6
8.3
10
IDL
(ppb)
5.2
1.4
4.8
5.9
10A
Resolution
Low
(ppb)
3.6
0.6
2.4
2.3
5
High
(ppb)
2.0
5.4
2.3
3.2
5
Lag
Time
(min
)
2
4
4
0
20
Rise
Time
(min)
5
8
3
5
15
SO2
Int*
(ppb)
0.58
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were
available to establish the response comparison.
A Value specific to the Thermo Model 42C NO/NCb/NOx Analyzer
CairClip NO2 Normal
•Thermo 42 NO2 (ppb}
Maximum Response of
Reference Monitor
Figure 8. Example CairClip Response for NC>2 under Normal Challenge Conditions
5.2.4 CitiSense
The CitiSense air quality monitoring device was designed to be highly-portable and
operate for relatively long periods of time between battery charges. The device, which
communicates measurements to a paired Smartphone device over Bluetooth, can go
approximately five days between charges. The device measures temperature, humidity,
barometric air pressure, Os, CO, and NC>2. Measurements are taken every six seconds and
transmitted immediately over the Bluetooth channel to any paired devices. The device contains
no long-term storage and no display; all storage, processing, and visualization is done in the
25
-------
CitiSense application for Android phones. Further details about the design of the sensor node
and its uses can be found in the reference section.21
The sensor device and its protective enclosure are custom designs by the CitiSense
research team at the University of California, San Diego. The actual sensing elements used in
the device are off-the-shelf components provided by several manufacturers. The
electrochemical ozone sensor used in the device is the O3-3E-1 4-series adaption Sensoric
Ozone sensor from CityTechnology.22 The electrochemical carbon monoxide sensor, CO-AX,
and the electrochemical nitrogen dioxide sensor, NO2-A1, are both from Alphasense.23'24 Each
sensor on the device was calibrated when initially set up. It is recognized that the sensors must
be re-calibrated once every three to six months and replaced once every twelve to sixteen
months to ensure accurate results. Low-level details about the design of the sensor board and
its conditioning circuit can be found in the reference section.25
Primary testing was performed for the CitiSense with NO2 under all conditions. The data
are summarized below in Table 11. Note that interference testing was performed only under
normal conditions. Two of the tests (NO2 cold and NO2 hot) failed to produce viable data for
statistical analysis. An example of the sensor's response for NO2 under normal challenge
conditions is shown in Figure 9.
The device yielded good linearity over the test range (R2>0.97). Precision was notably
poorer under humid test challenge (23.3 ppb). Reasonable deviation in response was observed
at high and low NO2 challenges (< 10 ppb) but humid conditions impacted IDL performance
(68.5 ppb). While lag times were of no consequence (< 1 min), a rise time of 18 minutes was
reported under normal test conditions. The device had an appreciable bias to SO2 challenge
(34.2 ppb).
Table 11. Summary of CitiSense Testing
Analyte
N02
CFR
N02
Conditions
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
0.9772
NA
0.9722
NA
NA
Precision
(ppb)
4.6
NA
23.3
NA
10
LDL
(ppb)
7.7
NA
19.6
NA
10
IDL
(ppb)
13.4
NA
68.5
NA
10A
Resolution
Low
(ppb)
3.7
NA
8.4
NA
5
High
(ppb)
8.7
NA
10.0
NA
5
Lag
Time
(min)
0
NA
0
NA
20
Rise
Time
(min)
18
NA
8
NA
15
SO2
Int*
(ppb)
34.2
20
NO2
Int*
(ppb)
NA
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were
available to establish the response comparison.
A Value specific to the Thermo Model 42C NO/NCb/NOx Analyzer
21 Zappi, P., Bales, E., Park, J. H., Griswold, W., & Rosing, T. S. (2012, April). The citisense air quality monitoring
mobile sensor node. In Proceedings of the 11th ACM/IEEE Conference on Information Processing in Sensor
Networks, Beijing, China.
22 Sensoric O3-31-1 Ozone Sensor spec sheet, CityTechnology. http://www.citytech.com/PDF-Datasheets/o33e1.pdf
23 CO-AX Carbon Monoxide Sensor spec sheet, Alphasense. http://www.alphasense.com/WEB1213/wp-
content/uploads/2013/10/COAX.pdf
24 NO2-A1 Nitrogen Dioxide Sensor spec sheet, Alphasense. http://www.alphasense.com/WEB1213/wp-
content/uploads/2013/07/N O2A1.pdf
25 Zappi, P. The CitiSense Air Quality Sensor Board manual. Tech Report (in Press)
26
-------
CitiSense NO, Normal
•Thermo 42 N02
Maximum Response of
Reference Monitor
Figure 9. Example CitiSense Response for NC>2 under Normal Challenge Conditions
5.2.5 Platypus
The Platypus Technologies prototype NC>2 monitor uses a liquid crystal (LC) sensor to
detect the target gas. Under controlled conditions, a thin film of LC rotates the polarization of
light waves that pass through it. This rotation can be monitored visually or with photodiodes by
placing the LC film between crossed polarizing filters. This light rotating property is used in
liquid crystal displays for TVs and computers: each pixel is a tiny volume of liquid crystal, and
changes in electric fields control the orientation of the LC molecules, which in turn determines
amount of light passing through each pixel, thereby creating the picture on the screen. Platypus
has adapted this concept by inducing the change in orientation of LC molecules on a film,
supported on chemically functionalized surface, upon exposure to target gas.26
The sensor in the prototype NC>2 monitor is a thin circular film of LC 5 mm in diameter,
mounted on a metal strip that slides into the monitor without the need for tools (Appendix B).
Device electronics monitor the light intensity transmitted through the LC film. The sensor is
configured so that LC molecules orient parallel with the sensor surface when there is no NC>2
present, which allows light to pass through the LC layer when viewed through crossed
polarizers. Exposure to NC>2 causes the LC molecules to re-orient perpendicular to the surface,
blocking the transmission of light through the sensor. When NC>2 concentration is once more
below the level of detection, the LC molecules return to the original parallel alignment. Changes
in light intensity are converted to voltages that are used to calculate target concentration for
display using an algorithm. The prototype monitor is capable of detecting ppb levels of NC>2
26 Reviewed in Shah, R.; Abbott, N.L., "Principles for Measurement of Chemical Exposure based on Recognition-
Driven Anchoring Transitions in Liquid Crystals, Science, 293, 1296-1299, 2001
27
-------
when exposed for a few minutes. Further development is under way at Platypus to improve
response times, sensitivity and linearity for specific detection of a range of gas targets.
In the primary testing performed here, only normal NC>2 conditions could be easily
accommodated. Because the device was not amenable for direct chamber testing, chamber
challenge gas was supplied to the unit using a gas transfer kit provided by Platypus. Following
each challenge (change in chamber concentration), the LC film in the sensor was replaced and
the next measurement initiated. The monitor was tested for replicate exposures to 100, 150,
200, 300 and 400 ppb NC>2. Note that interference testing was performed only under normal
conditions.
The device provided reasonable linearity over the test range (R2> 0.79) with a precision
of 18 ppb. Because the device is not capable of collecting a continuous data output (one data
value per LC film) we were unable to report findings on many performance characteristics.
Extremely high Os bias was observed (405 ppb) which was not a characteristic Platypus had
observed in their own internal testing in developing the prototype tested here.
Table 12. Summary of Platypus Testing
Analyte
NO2
CFR NO2
Conditions
Normal
NA
Linearity
(R2)
0.7956
NA
Precision
(ppb)
18.0
10
LDL
(ppb)
NA
10
IDL
(ppb)
NA
10A
Resolution
Low
(ppb)
NA
5
High
(ppb)
NA
5
Lag
Time
(min)
NA
20
Rise
Time
(min)
NA
15
SO2
Int
(ppb)
0
20
03
Int
(ppb)
405
20
NA indicates no data were available to establish the response comparison.
A Value specific to the Thermo Model 42C NO/NCb/NOx Analyzer
5.2.6 U-Pod
Testing was performed for the U-Pod under all conditions. The data are summarized
below in Table 13. Note that interference testing was performed only under normal conditions.
Proper conditioning as evident in a stable response output at the outset of the test run, was not
achieved in two of the tests (NC>2 normal, and NC>2 cold) therefore data obtained during these
challenges could not be used for statistical analysis.
Similar to the Os sensor used in the U-Pod, the NC>2 sensor is a MOS that retails for $7
(SGX Sensortech MiCS-2710). In these lab tests the sensor displayed good performance,
similar to what has been observed in ambient co-location calibrations.27 The high linearity from
regression (R2> 0.98) is expected at such high concentrations, but this is often not the case at
lower concentrations, where there is some log-like curvature that must be modeled. As with the
Os sensor, this sensor performs well in colder temperatures, but this was not demonstrated in
the lab testing due to the aforementioned instability at the initiation stage (equilibration). The
slow rise times listed in these results are not apparent in co-location field calibrations, where lag
times do not appear to be an issue when using minute averaged data. Again, this may be
27 Piedrahita, R., Xiang, Y., Masson, N., Ortega, J., Collier, A., Jiang, Y., Li, K., Dick, R., Lv, Q., Hannigan, M., Shang,
L. The next generation of low-cost personal air quality sensors for quantitative exposure monitoring. Atmospheric
Measurement Techniques Discussion. 7; 2425-2457, 2014. Doi:10.5194/amtd-7-2425-2014.
28
-------
because concentrations seen in the field co-locations are usually much lower than the range
tested here. We observed no measureable SC>2 interference under the challenge testing.
Table 13. Summary of U-Pod Testing
Analyte
NO2
CFR
NO2
Conditions
Normal
Hot
Humid
Cold
NA
Linearity
(R2)
NA
0.9851
0.9815
NA
NA
Precision
(ppb)
NA
5.0
2.6
NA
10
LDL
(ppb)
NA
4.2
6.9
NA
10
IDL
(ppb)
NA
11.7
2.5
NA
10A
Resolution
Low
(ppb)
NA
1.3
0.1
NA
5
High
(ppb)
NA
4.1
1.7
NA
5
Lag
Time
(min)
NA
1
1
NA
20
Rise
Time
(min)
NA
33
21
NA
15
S02
Int*
(ppb)
0
20
N02
Int*
(ppb)
20
*Blank cells indicate that interference testing was not conducted under that condition. NA indicates no data were
available to establish the response comparison.
A Value specific to the Thermo Model 42C NO/NCb/NOx Analyzer
29
-------
6. Study Limitations
It must be recognized that the scope of this low cost sensor performance evaluation was
limited with respect to a number of primary parameters:
• The resources of the U.S. EPA to conduct the extensive laboratory tests defined herein, and
• The scope of the performance testing that could be performed while being extensive was not
meant to define full FRM or FEM qualifying conditions.
6.1 Resource Limitations
6.1.1 Minimal Findings on Intra-sensor Performance Characteristics
Resource limitations typically only allowed for a single sensor provided by the
collaborating institution to be examined. Therefore, this report has minimal findings on intra-
sensor performance characteristics. Such variability may be highly important as it was noted
during the testing that upon occasion, more than one provided sensor failed to respond during
initial rounds of testing and had to be replaced by the manufacturer. It is not known if the
failures in such events were in response to a poor sensing element (the basic pollutant
measurement interface), or some failure of the assembled sensor system. The U.S. EPA
attempted no diagnostics of such events and merely communicated with collaborators that a
failure had occurred and an alternative device (replacement) was needed if testing was to be
performed.
6.1.2 Minimal Environmental and Interfering Agent Testing Conditions
Resources also prevented the U.S. EPA from examining the sensors under a wide
variety of environmental and interfering agent conditions. Single "cold" and "hot" temperature
conditions as defined in the Study Approach were employed as well as a single, relatively high
(>200 ppb) challenge concentrations of either Os, NC>2 or SC>2 were performed. None of the
tests were conducted with co-varying environmental and co-pollutant test atmospheres.
Likewise, one would have liked to examine the response of these sensors to varying RH and
temperature with respect to pollutant concentration changes. Such examinations would have
had a significant impact on the total amount of laboratory trials that could be afforded in this
instance and as such were not performed.
6.1.3 Limited Number of Sensors
This work represented a limited survey of all low cost sensors available to interested end
users. Based on the U.S. EPA's own market survey prior to initiating the world-wide call for
applicants, those who eventually applied represented a large portion of the applicable sensors
meeting the study inclusion criteria but not all openly available in the market place. No stigma
should be attached to sensors/sensor manufacturers not appearing in this report. The parties
agreeing to share their devices and ultimately reporting their data findings with the general
public here are to be applauded for their openness and willingness to helping to advance low
cost sensor technology.
30
-------
Table 14: Sensor Specifications
Manufacturer
ACT
HabitatMap,
Manhattan
College, New
York City
College of
Technology
Cairpol
CitiSense
Weather
Telematics
Model #
ACT
Environ me
ntal Sensor
AirCasting
Air Monitor
Cairclip
CARTOLA
WT-SU1
Dynamo
Sensor type
MOS
MOS
3 electrodes
Electrochemical
cell
Electrochemical
MOS
Pollutant
monitoring
capability
O3, NOx,
CO, VOC,
participate
matter
NO2
Combined
O3/NO2
CO, NO2, O3
03
Data storage/
transmission
Bluetooth to
Smartphone or
Bluetooth/USB
connection to
PC
Data storage:
Android phone
or tablet,
AirCasting
server
Transmission:
Bluetooth,
Cellular
network, Wi-Fi
Internal storage
up to 20 days
(1min-mean).
USB data
transmission
using the
specific
software
Cairsoft
Streams data
wirelessly to
paired mobile
phone that runs
the CitiSense
application. No
on-device
storage, only
stored once on
phone.
Data collected
and
incorporated in
main weather
data string,
transmitted over
Battery
type
Li-Ion
Li-ion
Li-Po
Li-ion
N/A
Battery life
(hrs)
4-8 hrs
(depending
on
sampling
interval)
4 hrs
From 48 h
(continuous
monitoring)
to several
weeks
(stand by
mode)
Approx.
120 hours
N/A
Data output
Concentration
if calibrated,
raw data if not
calibrated
Voltage or
Analog Value
Concentration
(ppbv)
Concentration
(ppm)
Concentration
(PPb)
Size
(l/w/h)
(cm)
13x
5.8 x
2.5
7.62 x
6.35 x
7.62
diamet
er: 3.2
length:
6.2
10.8x
6.7 x
3.8
Part of
main
weathe
r
sensor
Weight
(g)
170 g
225g
55g
Approx.
200 g
160g
(excludi
ng
cable)
Notes
Sensor can be
configured via PC
or smartphone
Display:
LCD screen, Real
time.
Dynamic sampling
using microfan
Range (ppbv)
0-250
31
-------
Platypus
Technologies
LLC
CU Boulder
Hannigan Lab
Prototype
U-Pod
Liquid crystal
MOS,
electrochemical,
PID, NDIR
NO2
MOS: O3,
NO2, CO,
VOCs.
NDIR: CO2.
PID: VOCs.
Electrochemi
cal: CO, NO,
NO2, O3,
SO2
CANBUS
connection to
data hub then
RS232 to GSM
modem or PC
software
Display, RS-232
MicroSD
storage. Can
also be
configured with
WiFi, Bluetooth
NiZn
Can be
configur
ed for
use with
Li-ion
and
lead-
acid
batteries
2 hours w/
pump
running @
full speed
12-100
hours,
varies with
battery
selection
Display (ppm)
or voltage
Voltage
18x
5.5x5
15.5x
8.3 x
4.3
22x19
x10
508g
1000-
2000g
32
-------
7. Conclusions
A number of definitive conclusions are evident from this laboratory-based examination of
low cost sensor performance. First, it is apparent that direct challenge via a well characterized
exposure chamber was not only possible but proved to be a reasonable means of establishing
basic performance characteristics of the devices. With rare exception was there evidence of gas
phase interaction (reactivity) between sensor devices and the challenge atmosphere that might
have influenced test results. This indicates that the test chamber and its supporting
components (gas delivery system, environmental controls (RH, temperature), in-line test
atmosphere system) were of sufficient materials and protocols to ensure valid test results.
Careful characterization of the chamber prior to initiation of the testing (e.g., reference monitor
response versus change in test atmosphere, stability of test atmosphere under static conditions,
impact of changing environmental conditions upon reference monitors) ensured defensible
evaluation of the devices under a wide range of testing scenarios. This being said, it would
have been profitable for the test chamber to have had the capability of providing environmental
conditions more varied with respect to both temperature and relative humidity. Resources
limitations prevented design of more advanced environmental controls into the test chamber.
While most of the sensors evaluated had little or no obvious unwanted interaction with
the test atmosphere itself, it should be recognized that in some instances (Os), some interaction
(reactivity) between the test atmosphere and the casing surrounding a sensor was established
or hypothesized. In such cases, removal of the plastic casing often significantly improved
performance. We also had to remove casings from some of the sensors to accommodate their
overall size to fit within the chamber and therefore potential reactivity was not able to be directly
tested for every sensor evaluated. Even though the testing chamber was designed to deliver a
high output of test gas atmosphere (flow through rate of 10 l/min) to combat any potential for
sensor starvation, sensor proximity to reactive surfaces is an issue of potential performance
consequence that should be considered in their design. Higher or lower chamber flow rates
could have changed performance for any of the sensors. Since no testing was conducted under
true ambient conditions, it is not known if such an "excess" of gas interaction with a sensor
might have been sufficient to overcome any potential for starvation/equilibration with respect to
casing materials. Without question, it would be wise for sensor developers to encase sensors
with materials fully characterized for their potential for gas phase reactivity. Even proximity of
the sensor interface (position above the surface of the casing) might need to be considered to
ensure the adequacy of the sensor design. In addition, recommended stabilization time prior to
data collection would be highly desirable to ensure adequate performance of the sensor.
The sensors evaluated here often exhibited a high degree of linearity (typically R2 > 0.9)
over an extremely large test range. For both Os and NC>2, the upper end of the challenge
conditions (> 200 ppb) would represent very unique ambient events that such sensors might
never encounter in non-occupational environments. Likewise, most of the sensors would
appear to offer detection sensitivities in the low ppb range and with precision (repeatability) well
within typically acceptable values. Such findings are encouraging for their potential applicability
for citizen science and even professionally-performed research endeavors. Of significance here
is the often short rise and lag times observed with the sensors (e.g., ~1 min). The sharp stair-
step pattern of response in the graphical displays of performance under test conditions is a clear
indication of how quickly most of the sensor responded. We saw little evidence of hysteresis or
failure of the sensors by retaining "memory" of the previous challenge condition after test
atmospheres had changed to a new concentration. This would indicate that these low cost
sensors might have potential applicability for use in non-static situations (movement with
respect to spatial setting). Even so, the tests performed here were not of sufficient design to
33
-------
evaluate very short spatial or even temporal impacts that one might experience if for instance
the sensor was being used to investigate on-road pollutant concentrations.
Few if any of the sensors evaluated had built-in zero or response calibration features.
Such features on higher priced (often occupational) sensors would be advantageous for
technically trained end users. That being said, we only rarely observed some positive bias
(concentration response > 0 ppb) at the start of any testing sequence when pollutant-free air
was being supplied to the test chamber. This result linked to the often good linearity of
response is a possible indication that for the sensors tested, commonly used zero and span
procedures might be replaced with simple collocation (normalization) comparisons with
reference monitors in the users environmental setting. That being said, we sometimes
observed sensors provided for evaluation yield no initial usable test data and having to be
replaced by replicate units for the evaluations to proceed. This is an important finding and users
deploying low cost sensors need to ensure that their device has been either calibrated in the
more traditional manner or compared with ambient monitoring data from collocation trials before
being used in data collections. The reason for the aforementioned failures may be linked to
possible lot-to-lot and even intra-lot variability of the primary sensing element often being
incorporated into many of the low cost sensors. In other words, quality assurance procedures
associated with the primary manufactured sensing elements would appear to be one of
consequence.
This evaluation did not have the capability of examining long-term performance
response characteristics (e.g., drift of signal over extended time periods of constant challenge,
stability of response with respect to sensor lifetime). Even so, some sensors originally provided
for testing and being more than two years of age failed to respond, indicating some potential
lifespan issues. While we cannot define the useable lifespan for any of the sensors evaluated, a
good rule of thumb might be something on the order of 1-2 years, especially for MOS and
similar sensing elements. End users should perform at least one of the evaluation procedures
described above on a reoccurring basis to ensure the operation status of their device. It is not
known at this time if such sensors yield a gradual decay in response or sharp decline at the end
of their lifespan.
The evaluations performed here represent a first step in understanding how the low cost
sensor segment compares to recognized FRM/FEM specifications. The results were not only
encouraging but in many instances quite surprising with respect to how well the devices
performed for certain performance characteristics (e.g., detection limits, linearity, precision, rise
and lag times). It should be noted that ultimately resource limitations sometimes prevented
evaluation of every sensor under all of the test conditions.
Lastly, it should be recognized that the named sensor developers submitting their
devices for evaluations here are to be applauded for their willingness to share their technology
(and test results) with others, including the general public. It is apparent in our discussions with
not only these sensor developers, but the market segment as a whole, that there is a real desire
to provide sensors meeting a wide range of air quality monitoring needs. Additional testing,
including evaluation of sensors of this nature under true ambient conditions, would provide for
enhanced understanding of how well these sensors respond to changing environmental
conditions and their applicability for various data collection scenarios.
34
-------
Appendix A
Technical Aspects- FRM/FEM Performance Parameters
40 CFR Part 53 Table B-1: Performance Specifications for Automated Methods
Performance parameter
1 . Range
2. Noise
3. Lower detectable limit
4. Interference equivalent
Each interferent
Total, all interferents
5. Zero drift, 12 and 24 hour
6. Span drift, 24 hour
20% of upper range limit
80% of upper range limit
7. Lag time
8. Rise time
9. Fall time
10, Precision
20 % of upper range limit
80 % of upper range limit
Units1
ppm
ppm
ppm
ppm
uEP?
ppm
Percent
Percent
Minutes
Minutes
Minutes
ppm
Percent
ppm
Percent
C
Std.
range3
0-0.5
0.001
0.002
±0.005
±0.004
±3.0
2
2
2
2
2
>02
Lower
range23
<0.5
0.0005
0.001
4±0.005
±0.002
±3.0
2
2
2
2
2
03
(Std.
range)
0-0.5
0.005
0.010
±0.02
0.06
±0.02
±20.0
±5.0
20
15
15
0.010
0.010
NO2
(Std.
range)
0-0.5
0.005
0.010
±0.02
0.04
±0.02
±20.0
±5.0
20
15
15
0.020
0.030
35
-------
Appendix B
Photographs of Sensors
1. AGT Environmental Sensor
2. AirCasting App and AirCasting Air Monitor
AirCasting App
AirCasting Air Monitor
36
-------
3. CairClip
4. CitiSense
37
-------
5. Dynamo
6. Platypus
Platypus prototype
handheld NO2 monitor
The 5-mm diameter
replaceable circular
liquid crystal sensor
element that slots into
the Platypus prototype
handheld NO2 monitor
38
-------
7. U Pod
5V regulated
power-supply
39
-------
Appendix C
List of Research Operating Protocols and Quality Assurance Project Plans Used
in Support of This Research
SOP/ROP/QAPP No.
SOP-EHD-03-01
SOP-EHD-03-02
SOP-EHD-03-03
SOP-4425-03-08(1)
Air Sensor ROP-
PMRB-109.0
Air Sensor ROP-
PMRB-103.0
Air Sensor ROP-
PMRB-106.0
Air Sensor ROP-
PMRB-101.0
Air Sensor ROP-
PMRB-107.0
Air Sensor ROP-
PMRB-104.0
Air Sensor ROP-
PMRB-105.0
GAP P-AB-1 2-03
Title
Operation and Maintenance of the TECO Model
49C UV 03 Analyzer
Operation and Maintenance of the TECO Model
43A Pulsed Fluorescence Sulfur Dioxide Analyzer
Operation and Maintenance of the TECO Model
42C Chemiluminescence NOx Analyzer
Operation and Maintenance of the Omega RH 41 1
Digital Thermo-Hygrometer
Research Operating Procedure for UPOD Air
Sensor
Research Operating Procedure for Cairclip Os &
N02 Air Sensor
Research Operating Procedure for the ACT Mobile
Environmental Sensor
Research Operating Procedure for AirCasting Air
Monitor
Research Operating Procedure for Platypus
Technologies LC-NOx Air Sensor
Research Operating Procedure for CitiSense Air
Sensor
Research Operating Procedure for the Dynamo Air
Sensor
ORD Sensor/Application Test Bed Challenge:
Investigation of App/Sensor Response under
Controlled Laboratory Conditions
Version
0
0
0
0
0
0
0
0
0
0
0
0
40
-------
vxEPA
United States
Environmental Protection
Agency
Office of Research and Development (8101R)
Washington, DC 20460
Official Business
Penalty for Private Use
$300
PRESORTED STANDARD
POSTAGE & FEES PAID
EPA
PERMIT NO. G-35
------- |