EPA/600/R-16/032
The Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Quality Assurance Project Plan for
Verification of
Black Carbon Monitors
-------
QUALITY ASSURANCE PROJECT PLAN
for
Verification of
Black Carbon Monitors
Version: 1.0
April 12, 2013
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Black Carbon Monitors
QAPP
Page 1 of 42
April 12, 2013
EPA/600/R-16/032
SECTION A
PROJECT MANAGEMENT
Al VENDOR APPROVAL PAGE
ETV Advanced Monitoring Systems Center
Quality Assurance Project Plan for Verification of
Black Carbon Monitors
Version: 1.0
April 12, 2013
APPROVAL:
Name
Company
Date
-------
Black Carbon Monitors
QAPP
Page 2 of 42
April 12, 2013
EPA/600/R-16/032
A2 TABLE OF CONTENTS
Section Page
A PROJECT MANAGEMENT
Al Vendor Approval Page 1
A2 Table of Contents 2
A3 Distribution List 4
A4 Verification Test Organization 5
A5 Background 12
A6 Verification Test Description and Schedule 13
A7 Quality Objectives 18
A8 Special Training/Certification 20
A9 Documentation and Records 20
B MEASUREMENT AND DATA ACQUISITION
Bl Experimental Design 21
B2 Reference Sample Collection 25
B3 Sample Handling and Custody Requirements 26
B4 Laboratory Reference Methods 27
B5 Quality Control Audits and Requirements 27
B6 Instrument/Equipment Testing, Inspection, and Maintenance 28
B7 Instrument Calibration and Frequency 29
B8 Inspection/Acceptance of Supplies and Consumables 30
B9 Non-Direct Measurements 31
BIO Data Management 31
C ASSESSMENT AND OVERSIGHT
Cl Assessments and Response Actions 34
C2 Reports to Management 37
D DATA VALIDATION AND USABILITY
Dl Data Review, Validation, and Verification Requirements 39
D2 Validation and Verification Methods 39
D3 Reconciliation with User Requirements 40
E REFERENCES 42
-------
Black Carbon Monitors
QAPP
PageS of 42
April 12, 2013
EPA/600/R-16/032
ADQ Audit of data quality
AMS Advanced Monitoring Systems
BC Black carbon
DQI Data quality indicator
DQO Data quality objective
DRI Desert Research Institute
EC Elemental carbon
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
IMPROVE Interagency Monitoring of PROtected Visual Environments
LAC Light absorbing carbon
LRB Laboratory record book
OC Organic carbon
QA Quality assurance
QAO Quality assurance officer
QAPP Quality assurance project plan
QMP Quality management plan
RAAS Reference Ambient Air Sampler
RMO Records management office
RPD Relative percent difference
SI International standards
SOP Standard operating procedure
TOR Thermal optical reflectance
TSA Technical systems audit
VTC Verification test coordinator
-------
Black Carbon Monitors
QAPP
Page 4 of 42
April 12, 2013
EPA/600/R-16/032
A3 DISTRIBUTION LIST
Vendors
David Smith
Robert Gary
Sunset Laboratory Inc.
10180 SW Nimbus Avenue
Suite J/5
Tigard, OR 97223-4341
Tony Hansen
Magee Scientific Corporation
1916aM. L.King Jr. Way
Berkeley, CA 94704 USA
EPA
Doug Grosse
U.S. Environmental Protection Agency
26 West Martin Luther King Drive
Mail code: 489
Cincinnati, Ohio 45268
Battelle
Kenneth Cowen
Amy Dindal
Elizabeth Hanft
Thomas Kelly
Rosanna Buhl
Zachary Willenberg
Battelle
505 King Ave.
Columbus, OH 43201
-------
Black Carbon Monitors
QAPP
Page 5 of 42
April 12, 2013
EPA/600/R-16/032
A4 VERIFICATION TEST ORGANIZATION
The verification test described in this document will be conducted under the auspices of the U.S.
Environmental Protection Agency (EPA) through the Environmental Technology Verification
(ETV) Program. It will be performed by Battelle, which is managing the ETV Advanced
Monitoring Systems (AMS) Center through a cooperative agreement with EPA. The scope of
the AMS Center covers verification of monitoring technologies for contaminants and natural
species in air, water, and soil.
This verification test will be coordinated and directed by Battelle in cooperation with EPA. This
test will include a 30-day period of field testing that will be conducted at the Battelle Columbus
Operations Special Support Site (BCS3) located at 2555 International St., in Columbus, OH. The
field testing is planned to involve the evaluation of multiple commercial monitors for black
carbon (BC) in airborne particulate matter. The vendors of the BC monitors will install,
maintain, and if necessary repair their systems during the verification test. Reference method
sampling for BC will be performed by Battelle and analysis of the collected reference samples
will be performed by Desert Research Institute (DRI) under a purchase order from Battelle.
Quality assurance (QA) oversight will be provided by the Battelle AMS Center Quality Manager,
and the EPA at its discretion. This test is Quality Category III which requires a QA review of
10% of the test data (see section Cl). The organization chart in Figure 1 identifies the
responsibilities of the organizations and individuals associated with the verification test. Roles
and responsibilities are defined further below.
A4.1 Battelle
Dr. Kenneth Cowen is the AMS Center Verification Test Coordinator (VTC) for this test. In this
role, Dr. Cowen will have overall responsibility for ensuring that the technical, schedule, and
cost goals established for the verification test are met. Specifically, he will:
> Prepare and oversee review and approval of the QAPP.
-------
Black Carbon Monitors
QAPP
Paged of 42
April 12, 2013
EPA/600/R-16/032
> Establish a budget for the testing and manage staff to ensure the budget is not
exceeded.
Battelle
Management
Battelle Quality
Manager
Rosanna Buhl
Quality Assurance
Officer
Zachary Willenberg
Battelle AMS
Center Manaaer
Amy Dindal
Verification Test
Coordinator
Kenneth Cowen
Battelle Testing
Staff
EPA AMS Center Project
Officer
EPA Quality
Representative
Vendor Representatives
Figure 1. Organization chart for the black carbon monitor verification.
> Revise the draft QAPP in response to reviewers' comments.
> Assemble a team of qualified technical staff to conduct the verification test.
> Direct the team (Battelle, EPA, vendor(s)) in performing the verification test in
accordance with this QA project plan (QAPP).
> Hold a kick-off meeting approximately one (1) week prior to the start of the testing to
review the critical logistical, technical, and administrative aspects of the testing.
Responsibility for each aspect of the testing will be reviewed to ensure each
participant understands his/her role.
> Ensure that all quality procedures specified in the QAPP and in the AMS Center
Quality Management Plan1 (QMP) are followed.
> Prepare the draft and final QAPP, verification report(s), and verification statement(s).
> Revise the draft QAPP, verification report(s), and verification statement(s) in
response to reviewers' comments.
-------
Black Carbon Monitors
QAPP
Page 7 of 42
April 12, 2013
EPA/600/R-16/032
> Coordinate distribution of the final QAPP, verification report(s), and statement(s).
> Respond to QAPP deviations and any issues raised in assessment reports, audits, or
from test staff observations, and institute corrective action as necessary.
> Serve as the primary point of contact for vendor representatives and collaborators.
Ms. Amy Dindal is Battelle's manager for the AMS Center. Ms. Dindal will:
> Review the draft and approve the final QAPP.
> Attend the proj ect kick-off meeting.
> Review the draft and final verification report(s) and verification statement(s).
> Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test.
> Ensure that confidentiality of sensitive vendor information is maintained.
> Support Dr. Cowen in responding to any issues raised in assessment reports and
audits
> Maintain communication with EPA's AMS Center Project Officer.
> Facilitate a stop work order if Battelle or EPA QA staff discover adverse findings that
will compromise data quality or test results.
Battelle Field Testing Staff will oversee the testing of the BC monitors during the verification
test. Battelle staff will visit the testing site at the BCS3 facility daily during the verification test
to conduct the reference sampling, verify proper operation of the BC monitors being tested, and
communicate with the BC monitor vendors as needed. The responsibilities of the field testing
staff will be to:
> Assist in planning for training and testing as necessary;
> Attend the proj ect kick-off meeting
> Conduct testing and collect data and samples according to this QAPP.
-------
Black Carbon Monitors
QAPP
Page 8 of 42
April 12, 2013
EPA/600/R-16/032
> Collect reference samples and ship samples to the DRI laboratory for analysis.
> Report results of the reference sampling and analysis.
> Record qualitative observations about the maintenance and operation of the BC
monitors during testing.
> Assure that the data from the BC monitors are compiled, recorded, and transmitted to
the VTC on at least a weekly basis.
> Perform analysis of the collected data to carry out the statistical evaluations in
SectionB 1.1.
> Support Dr. Cowen in responding to any issues raised in assessment reports and
audits related to statistics and data reduction as needed;
> Immediately report deviations from this QAPP to the VTC
> Provide input on test procedures, technology operation and maintenance, and field
conditions for the draft verification reports.
Ms. Rosanna Buhl is Battelle's Quality Manager for the AMS Center. Ms. Buhl will:
> Review, or designate the review of the draft and final QAPP.
> Assign a Quality Assurance Officer (QAO) for this verification test.
> Delegate to other Battelle quality staff any QAO responsibilities assigned below as
needed to meet project schedules.
> Review any audit checklists prepared by the QAO for completeness and detail.
> Review draft and final audit reports prior to release to the VTC and/or EPA for clarity
and appropriate assessment of findings.
> Review audit responses for appropriateness.
> Review and approve QAPPs and deviations.
> Review draft and final verification report(s) and verification statement(s).
> Maintain real-time communication with the QAO on QA activities, audit results, and
concerns.
-------
Black Carbon Monitors
QAPP
Page 9 of 42
April 12, 2013
EPA/600/R-16/032
> Work with the QAO, VTC, and Battelle' s AMS Center Manager to resolve data
quality concerns and disputes.
> Recommend a stop work order if audits indicate that data quality or safety is being
compromised.
Mr. Zachary Willenberg is Battelle's QAO for this test. Mr. Willenberg will:
> Attend the verification test kick-off meeting and lead the discussion of the QA
elements of the kick-off meeting checklist.
> Prior to the start of verification testing, verify the presence of applicable training
records, including any vendor training on test equipment.
> Conduct a technical systems audit at least once near the beginning of the verification
test.
> Conduct audits to verify data quality.
> Prepare and distribute an audit report for each audit.
> Verify that audit responses for each audit finding and observation are appropriate and
that corrective action has been implemented effectively.
> Communicate to the VTC and/or technical staff the need for immediate corrective
action if an audit identifies QAPP deviations or practices that threaten data quality.
> Provide a summary of the QA/QC activities and results for the verification reports.
> Review the draft and final verification report(s) and verification statement(s).
> Maintain real-time communication with the Battelle Quality Manager on QA
activities, audit results, and concerns, including potential schedule and budget
problems.
> Communicate data quality concerns to the VTC and/or Battelle's AMS Center
Quality Manager; recommend the need for a stop work order if audits indicate that
data quality or safety is being compromised.
A4.2 Vendors
-------
Black Carbon Monitors
QAPP
Page 10 of 42
April 12, 2013
EPA/600/R-16/032
The responsibilities of the BC monitor vendors are as follows:
> Review and provide comments on the draft QAPP.
> Approve the final QAPP prior to test initiation.
> Provide duplicate units of their BC monitor for evaluation during the verification test.
> Provide all other equipment/supplies/reagents/consumables needed to operate their
monitors for the duration of the verification test.
> Supply a representative to install, operate, and maintain their monitors during the
verification test.
> Review the data from their duplicate monitors provided by the Battelle field testing
staff.
> Provide training to site operator(s) and others associated with supervising and/or
maintaining BC monitor operation including during the verification testing period.
> Provide written instructions for routine operation of their monitor, including a daily
checklist of diagnostic and/or maintenance activities.
> Review and provide comments on the draft verification report and verification
statement for their BC monitor.
A4.3 EPA
EPA's responsibilities are based on the requirements stated in the "Environmental Technology
Verification Program Quality Management Plan" (EPA QMP)2. The roles of specific EPA
testing staff are as follows:
Mr. Doug Grosse is EPA's Project Officer (PO) for the AMS Center. Mr. Grosse will:
> Review and approve the draft and final QAPP.
> Oversee the EPA review process for the QAPP and verification report.
-------
Black Carbon Monitors
QAPP
Page 11 of 42
April 12, 2013
EPA/600/R-16/032
> Be available during the testing to review and authorize any QAPP deviations by phone
and provide the name of a delegate to the Battelle AMS Center Manager should he not be
available during the testing period.
> Review and approve the draft and final verification report.
> Coordinate the submission of the verification report for final EPA approval.
> Post the QAPP and verification report on the ETV web site.
The EPA's AMS Center quality representative may:
> Review the draft and final QAPP.
> Perform at his/her option one external TSA during the testing.
> Prepare and distribute an assessment report summarizing results of the external audit.
> Perform audits of data quality.
> Notify the EPA AMS Center PO of the need for a stop or modify work order if the audit
of data quality indicates that data quality is being compromised.
> Review the draft and final verification report.
A4.4 Desert Research Institute (DRI)
DRI is responsible for preparing and analyzing the reference filter samples used for comparison
with the BC monitoring systems being tested. DRI will be responsible for shipment of the
reference sampling media to the field and for analysis of the exposed filters received from the
field.
Mr. Anthony Chen is the DRI Technical Lead for this verification test. In this role, Mr. Chen is
responsible for ensuring that the reference filter preparation, shipment, and analysis activities
meet the scheduled milestones agreed upon by DRI and Battelle. Mr. Chen will:
> Review the draft QAPP.
> Be the primary DRI contact for Battelle's VTC.
-------
Black Carbon Monitors
QAPP
Page 12 of 42
April 12, 2013
EPA/600/R-16/032
> Ensure that designated DRI staff are available for the verification test.
> Coordinate distribution of the QAPP to DRI staff.
> Coordinate the filter preparation, shipment, and analysis activities.
> Review and approve all data and records related to sampling and analysis activities.
> Deliver reference filter analysis results to Battelle's VTC within the agreed-upon
turnaround time.
A4.4 Verification Test Stakeholders
This QAPP and the verification report and verification statement based on testing described in
this document will be reviewed by experts in the fields related to BC monitoring. The
stakeholders for this verification test are Joann Rice from EPA's Office of Air Quality Planning
and Standards, and Andrea Polidori from the South Coast Air Quality Management District.
A5
The ETV Program's AMS Center conducts third-party performance testing of commercially
available technologies that detect or monitor natural species or contaminants in air, water, and
soil. Stakeholder committees of buyers and users of such technologies recommend technology
categories, and technologies within those categories, as priorities for testing. Among the
technology categories recommended for testing are "black carbon" monitors. Because of the
nature of "black carbon", this technology category includes monitors for both BC and elemental
carbon (EC). Although BC and EC are operationally defined based on different properties, for
convenience the term "black carbon" will be used in this QAPP to refer to both BC and EC.
Black carbon is a term that is commonly used to describe strongly light absorbing carbon (LAC),
which is thought to play a significant role in global climate change through direct absorption of
light, interaction with clouds, and by reducing the reflectivity of snow and ice. BC is formed
from the incomplete combustion of fossil fuels, biofuels, and biomass and can be emitted from
-------
Black Carbon Monitors
QAPP
Page 13 of 42
April 12, 2013
EPA/600/R-16/032
both anthropogenic and natural sources. It is a primary component of soot and has been linked to
adverse health effects and visibility reduction. Consequently, there is a great deal of interest in
monitoring BC in the atmosphere. However, differences in measurement techniques result in
measurements that are operationally defined and characterize the paniculate matter based on
either its light absorbing properties (leading to determination of BC) or its refractory properties
(leading to determination of EC), as illustrated in Figure 2. In this figure, the use of the subscript
a denotes that the measurements are technique specific and result in estimations of BC or EC that
are "apparent" based on the technique being used. The methods used to determine EC are
termed thermal-optical in Figure 2 because they involve conversion of particulate carbon to
gaseous form under varying temperatures and controlled atmospheres while the particulate
sample is monitored by either transmission or reflection of light.
Light-Absorption Classification
More
light-absorbing
Light-
absorbing
carbon
(LAC)
Less
light-absorbing
Black
carbon
Brown
carbon
(BrC)
Thermal-Optical Classification
More refractory
Elemental
carbon
Organic
carbon
Less refractory
* Measurement technique-specific split point
Figure 2. Illustration of measurements of carbonaceous particulate matter.
(Source: U.S. EPA)3
-------
Black Carbon Monitors
QAPP
Page 14 of 42
April 12, 2013
EPA/600/R-16/032
A6 VERIFICATION TEST DESCRIPTION AND SCHEDULE
The purpose of this verification test is to generate performance data on BC monitoring
technologies so organizations and users interested in installing and operating these systems can
understand the technology performance under a specified set of conditions. The test will be
conducted over a testing period of approximately 30 days and will involve the continuous
operation of BC monitors at an ambient air monitoring station installed at the BCS3 facility in
Columbus, OH. The results from the BC monitors will be evaluated relative to those of a
common "reference" method to assess comparability to that reference method. The reference
method selected for this comparison is the method used in the Interagency Monitoring of
PROtected Visual Environments (IMPROVE) network. However, it should be noted that this
determination of comparability relies on comparisons to a thermal optical method which may
result in biases in favor of similar thermal optical methods and against light absorption methods.
The unit-to-unit precision of the BC monitors will be determined from comparisons of paired
data from duplicate monitors. Operational performance parameters such as data completeness,
maintenance requirements, ease of use, consumables, and costs will be determined from
observations by the Battelle field testing staff. This test is not intended to simulate long-term
performance of these technologies at a monitoring site.
A6.1 Technology Description
The most common techniques used for the measurement of BC include light absorption and
thermal-optical methods. Near real-time monitors that provide high time resolution
measurements have been developed based on both types of methodology, and a single monitor
may perform both types of measurements. These monitors are the subject of this verification
test, and each monitor tested will be evaluated for each type of measurement that it makes.
Details of the specific instruments being tested will be described in the corresponding
verification reports that will be prepared for this verification test. Brief, generic descriptions of
these methods are provided below.
-------
Black Carbon Monitors
QAPP
Page 15 of 42
April 12, 2013
EPA/600/R-16/032
In general, light absorption techniques rely on the measurement of light absorption at one or
more specific wavelengths as that light passes through particle samples either collected on filters
or suspended in a volume of air. Since BC is highly light-absorbing the amount of light
absorption can be related to the amount of BC present through a wavelength dependent
absorption coefficient. Continuous BC monitors based on light absorption may use a filter tape
on which ambient particulate matter is collected as a sample spot. Advancement of the tape
places a collected sample spot in position for analysis, while particulate matter collection
continues on a previously unexposed spot on the tape. In this verification test, the Magee
Scientific/Aerosol Co. Aethalometer™ Model AE33-7, which is representative of light
absorption techniques, will be tested.
Thermal-optical techniques rely on the measurement of evolved gases that are released from
collected samples as the samples are heated under specific temperature profiles and in controlled
atmospheres. These techniques typically characterize both the organic carbon (OC) and EC
fractions of the collected particulate matter and include optical measurements to correct for
artifacts associated with charring of organic materials. Continuous monitors based on thermal-
optical methods function by automating the sequence of controlled temperatures and controlled
atmospheres needed to distinguish EC and OC fractions. In this verification test, the Sunset
Laboratory Model 4 OC-EC Field Analyzer, which is representative of thermal-optical methods,
will be tested.
A6.2 Verification Test Description and Schedule
This verification test will involve the evaluation of BC monitors under realistic operating
conditions at an existing ambient air monitoring station. The BC monitors will be operated
continuously sampling ambient air for approximately 30 days, during which time a series of
reference method samples will also be collected. Specifically, each day during the test period
duplicate integrated filter reference samples will be collected over successive 12-hour periods.
Thus, over the 30-day field period, a total of 120 reference filter samples will be collected (i.e.,
-------
Black Carbon Monitors
QAPP
Page 16 of 42
April 12, 2013
EPA/600/R-16/032
duplicate samples during each of the 60 12-hour sampling periods). The resulting samples will
be analyzed for BC by DRI using the IMPROVE thermal/optical reflectance (TOR) method4,
which monitors the filter sample by means of optical reflectance. Results from the BC monitors
will be compared to the reference results to assess the comparability of the BC monitor results to
the reference results.
Table 1 shows the planned schedule of testing and data analysis/reporting activities to be
conducted in this verification. The verification field test is planned to begin in April 2013 with
installation of the BC monitors, and will be completed in May 2013, with the removal of the BC
monitors from the field site. The period of operation of the BC monitors at the testing site will be
approximately 30 days, with routine operation expected to begin on April 5 and continue through
May 3, 2013. During testing, duplicate filter reference samples will be collected twice each day,
with each sampling period covering approximately 12 hours (e.g., 7 am to 7 pm and 7 pm to 7
am).
Table 1. Planned Verification Test Schedule
Date(s)
March 25- 29
April 5-May 3
May 6-7
May 1-31
June 28
July 31
August 31
September 30
Testing Activities
Equipment installation/shakedown
Routine operation
Reference sampling
Analysis of reference samples
Performance of ISA
Remove monitoring systems from test site
Complete analysis of reference samples
Perform ADQ #1 (1 0% of all data)
~
Perform ADQ #2
~
Data Analysis and Reporting
~
Prepare report template
Review and summarize field testing
staff observations
Compile data from BC monitors
Begin draft report(s)
-
Continue preparation of draft report(s)
Perform data analysis
Complete data analysis
Complete draft report(s)
Complete review of draft report(s)
Revise draft report(s)
Submit final report(s) for EPA approval
-------
Black Carbon Monitors
QAPP
Page 17 of 42
April 12, 2013
EPA/600/R-16/032
Subsequent to the verification test, a verification report will be drafted for each BC monitor
tested. This report will be reviewed by the vendor and by peer reviewers, and submitted to EPA
for final signature. In performing the verification test, Battelle will follow the technical and QA
procedures specified in this QAPP and will comply with the data quality requirements in the
AMS Center QMP.1
A6.3 Test Facility
The test will be conducted at the BCS3 facility located at 2555 International St., in Columbus,
OH. Figure 3 shows an aerial photograph of the test site (red marker "A") and the surrounding
area. The test site is located near a rail yard and in the vicinity of multiple industrial and
Figure 3. Aerial photograph of test site and surrounding area.
-------
Black Carbon Monitors
QAPP
Page 18 of 42
April 12, 2013
EPA/600/R-16/032
shipping facilities which result in frequent truck traffic past the site. The site also receives
regionally transported air pollution due to its location on the western side of the Columbus
metropolitan area. An environmentally controlled mobile laboratory will be installed at the site
to serve as a shelter for the BC monitors and as work space for the testing staff. Reference
samples will be collected using an Andersen Reference Ambient Air Sampler (RAAS) speciation
sampler located on a platform adjacent to the mobile laboratory.
A7 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT
DATA
The objective of this verification test is to evaluate the performance of the BC monitors under
realistic operating conditions. This evaluation will in part assess the capabilities of the monitors
for determining the ambient concentrations of BC through comparisons to collocated reference
samples collected during the verification test period. Additionally, this evaluation will assess
instrument precision based on comparisons of results from duplicate BC monitors, and will
assess data completeness based on the fraction of the data collected relative to the maximum
amount of data that could be collected during the testing period. The test will rely upon operator
observations to assess other operational performance characteristics of the monitoring systems
being tested such as ease of use, maintenance and repair needs, and consumables use.
The data quality objectives (DQOs) for this verification test were established to assess the
performance of the BC monitors relative to the IMPROVE_A_TOR reference method. In order
to provide a suitable benchmark for comparison, the reference samples must be of suitable
quality to allow for an accurate assessment of the BC monitors being evaluated. Thus, the DQOs
for this verification test include objectives for reference method accuracy and precision, as well
as data completeness for the reference method sample collection and analysis. The DQOs are
quantitatively defined in Table 2 in terms of specific data quality indicators (DQIs) and their
acceptance criteria. The quality of the reference method measurements will be assured by
-------
Black Carbon Monitors
QAPP
Page 19 of 42
April 12, 2013
EPA/600/R-16/032
adherence to these DQI criteria and the requirements of the reference methods including the
calibration and QA/QC requirements of those methods, which are discussed in detail in Sections
B2 to B7 of this QAPP. Calibration standards and QC samples must meet International
Standards (SI) traceability, when available. The quality of the reference method measurements
will be monitored by inclusion of blank samples and performance evaluation (PE) samples as
appropriate. Section Cl. 1 presents a description of the PE audit samples/ measurements to be
performed and the acceptance criteria for those measurements.
Table 2. DQIs and Criteria for Critical Measurements for Reference Methods.
Measurement
Reference method
flow rate
IMPROVE_A_TOR
analysis method
DQI
Accuracy
Accuracy
Precision
Laboratory Blank
Check
Method
Comparison to SI traceable flow transfer
standard
Comparison to SI traceable temperature
sensor
Comparison to SI traceable
pressure sensor
Analysis of standard reference material
Analysis of duplicate samples
Analysis of clean filter punch
Criteria
±5%
±2°C
±5 mmHg
±10% of actual
concentration
RPDa<15%
^0.2 ug
TCb/cm2
a RPD - Relative Percent Difference
b TC - total carbon.
Additionally, the verification test relies in part on observations of the Battelle field testing staff
for assessment of the performance of the monitoring systems being tested. The requirements for
these observations are described in the discussion of documentation requirements and data
review, verification, and validation requirements for this verification test.
The Battelle QAO signee will perform a TSA at least once during this verification test to
augment these QA/QC requirements. This TSA will be performed within the first week of the
verification test. The EPA quality representative also may conduct an independent TSA, at her
discretion.
-------
Black Carbon Monitors
QAPP
Page 20 of 42
April 12, 2013
EPA/600/R-16/032
-------
Black Carbon Monitors
QAPP
Page 21 of 42
April 12, 2013
EPA/600/R-16/032
A8 SPECIAL TRAINING/CERTIFICATION
Documentation of training related to technology testing, field testing, data analysis, and reporting
is maintained for all Battelle technical staff in training files at their respective locations. The
Battelle QAO may verify the presence of appropriate training records prior to the start of testing.
Battelle technical staff supporting this verification have a minimum of a bachelor's degree in
science/engineering. The VTC has a Ph.D. in Physical Chemistry and has approximately 14
years of experience performing ETV verifications.
A9 DOCUMENTATION AND RECORDS
The records for this verification test will include the QAPP, chain-of-custody forms, laboratory
record books (LRB), data collection forms, electronic files (both raw data and spreadsheets), and
the final verification report(s). All of these records will be maintained at the test facility or in the
VTC's office during the test and may be transferred to permanent storage at Battelle's Records
Management Office (RMO) at the conclusion of the verification test. All Battelle LRBs are
stored indefinitely by Battelle's RMO. EPA will be notified before disposal of any files. The
documentation and results of the reference method measurements made by DRI will be
submitted to Battelle within 10 days after completion of all sample analyses, review of the data,
and calculation of analyte concentrations in the ambient air. Section BIO further details the data
recording practices and responsibilities.
-------
Black Carbon Monitors
QAPP
Page 22 of 42
April 12, 2013
EPA/600/R-16/032
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
This test will specifically address verification of BC monitors by evaluating the comparability
and precision of the monitors, as well as data completeness, reliability, and maintenance needs of
each monitor. Comparability will be determined for the monitors by comparison of their results
to BC concentration measurements from the filter-based reference samples. Unit-to-unit
precision will be assessed through comparison of paired results from the duplicate monitors.
Data completeness will be assessed as the percentage of maximum data return that is achieved by
the monitoring systems over the test period. Reliability and maintenance needs will be evaluated
by means of observations by field testing staff, and records of needed maintenance, vendor
activities, and consumables used.
Bl.l Test Procedures
During testing, duplicate BC monitors will be installed inside an environmentally controlled
shelter at the BCS3 test site. The monitors will be installed by the vendors, and are intended to
operate continuously over the 30-day testing period. Routine maintenance of the BC monitors
will be performed by Battelle testing staff according to vendor instructions. Any repair or
replacement of the monitors will be performed by the vendor and will be documented by
Battelle. Data from the BC monitors will be retrieved by the Battelle testing staff and provided
to the respective vendors at least weekly. Battelle testing staff will conduct daily checks of the
monitors to confirm proper operation. The results of those checks will be documented on
checklists for each analyzer and any observed problems will be communicated promptly to the
respective vendor. Maintenance activities performed on the monitors will be documented by
Battelle and noted in any data analysis activities.
-------
Black Carbon Monitors
QAPP
Page 23 of 42
April 12, 2013
EPA/600/R-16/032
Integrated filter samples will be collected over 12-hour sampling periods throughout the testing
period, i.e., from 7:00 a.m. to 7:00 p.m. and from 7:00 p.m. to 7:00 a.m. daily. The filters used
for this test will consist of pre-fired 47 mm Pallflex quartz fiber filters. For sampling the filter
samples will be installed in a RAAS speciation sampler mounted on a platform placed at
approximately the same height as the roof of the trailer housing the BC monitors being tested.
Duplicate filter samples will be collected by simultaneous sampling in parallel channels of the
RAAS sampler at a flow rate of 16.7 L/min, after passage of the sample air through the sampler's
PM2.5 inlet. The filter samples will be retrieved after each sampling period and stored under
refrigeration until being shipped to the analytical laboratory for analysis by the
IMPROVE_A_TOR method. The collected samples will be shipped to the DRI laboratory for
analysis within 7 days of collection. Results from the reference method analysis will be divided
by the total volume of air sampled during the respective sampling periods and expressed as
average BC concentrations during the sampling periods.
Bl.1.1 Comparability
The BC monitors undergoing testing will be evaluated for comparability in two ways. Firstly,
comparability will be determined from a linear least squares regression analysis of the measured
BC concentrations from the continuous monitor against the corresponding BC results from the
reference method. For comparison to the reference results, average concentrations from each of
the monitors being tested will be determined separately for each of the 12-hour sampling periods,
by averaging the monitor's individual results over the corresponding sampling period. The
monitors being tested will be operated such that the measurement times can be synchronized
with the reference measurements to the extent possible. These averages will then be plotted
separately for each monitor being tested against the mean of the corresponding duplicate
reference method measurements. The slope and intercept of these plots will be determined from
a linear regression analysis and reported independently for each of the monitors.
-------
Black Carbon Monitors
QAPP
Page 24 of 42
April 12, 2013
EPA/600/R-16/032
Additionally, comparability will be determined in terms of the relative percent difference (RPD)
between the mean value of the reference measurements and the results from each BC monitor
being tested. The RPD will be calculated using Equation 1:
0)
C(re/),
where d is the average BC concentration measured by the BC monitor during the ith reference
sampling period, and ^re-> is the mean of the duplicate reference method BC
concentrations for the ith reference sampling period.
Both measures of comparability will be determined relative to the results from both reference
methods.
Bl.1.2 Correlation
The degree of correlation of each BC monitor's results to the reference results will be determined
based on the coefficient of determination (r2) value of the linear regression performed to assess
accuracy (Section B1.1.1). Correlation will be determined separately for each unit of each BC
monitor undergoing testing, and relative to the results from the reference method.
Bl.1.3 Precision
Precision (P) will be determined based on a comparison of paired measurements from the
duplicate BC monitors being tested. For this assessment of precision, the P between the paired
measurements from the duplicate BC monitors will be calculated using Equation 2:
where C(l), and C(2)z- are the BC concentrations measured by the first and second of the two
duplicate monitoring systems. Precision will be calculated for each set of duplicate BC monitors
-------
Black Carbon Monitors
QAPP
Page 25 of 42
April 12, 2013
EPA/600/R-16/032
for each reference sampling period, and the overall mean precision will also be reported. For this
calculation, measurement data below twice the vendor's stated instrumental detection limit will
be excluded from the analysis.
Bl. 1.4 Data Completeness
Data completeness will be assessed in two ways, based on the overall data return achieved by
each BC monitor during the testing period. First, for each of the BC monitors data completeness
will be calculated as the total hours of apparently valid data reported by the monitor divided by
the maximum total possible hours of monitoring data in the entire field period. Also, for each
BC monitor data completeness will be calculated as the percentage of 12-hour reference method
sampling periods in which the monitor provided at least 9 hours of valid data (75%). The causes
of any substantial incompleteness of data return will be established from operator observations or
vendor records, and noted in the discussion of data completeness results.
BL 1.5 Operational Factors
Operational factors such as maintenance needs, data output, consumables used, ease of use,
repair requirements, etc., will be evaluated based on observations recorded by Battelle testing
staff. Battelle staff will be at the monitoring site whenever the vendor is present and will record
all activities performed on the monitoring systems and will complete a daily checklist for each
analyzer to ensure proper operation. A laboratory record book will be maintained at the test site,
and will be used to enter daily observations on these factors. Examples of information to be
recorded in the record books include the daily status of diagnostic indicators for the monitoring
systems; use or replacement of any consumables; the effort or cost associated with maintenance
or repair; vendor effort (e.g., time on site) for repair or maintenance; the duration and causes of
any down time or data acquisition failure; and Battelle testing staff observations about ease of
use of the monitoring systems. These observations will be summarized in describing monitoring
system performance in the verification report. Explanatory information may be requested from
the vendor and included from the report as needed.
-------
Black Carbon Monitors
QAPP
Page 26 of 42
April 12, 2013
EPA/600/R-16/032
B1.3 Reporting
The statistical comparisons described above will be conducted separately for each of the BC
monitors being tested. Additionally, information on the operational parameters will be compiled
and reported. A verification report that presents the test procedures, test data, and results of the
statistical evaluation of those data will be prepared for each BC monitor tested.
Operational aspects of the monitoring systems will be recorded by Battelle testing staff at the
time of observation during the field test, and summarized in the verification report. For example,
descriptions of the data acquisition procedures, use of vendor-supplied proprietary software,
consumables used, repairs and maintenance needed, and the nature of any problems will be
presented in the report. The verification report will briefly describe the ETV program, the AMS
Center, and the procedures used in verification testing, and the results of the verification test will
be stated quantitatively. Each draft verification report will be subjected to review by the vendor,
EPA, and other peer reviewers. The review comments will be addressed in a subsequent revision
of the report, and the peer review comments and responses will be tabulated to document the
peer review process. The reporting and review process will be conducted according to the
requirements of the ETV/AMS Center QMP.2
B2 SAMPLING METHOD REQUIREMENTS
Filter sampling will be performed by Battelle testing staff as described in Section B 1.1 to provide
reference method measurements. The reference samples will be collected from a platform near
the roof of the same mobile laboratory at the monitoring site that houses the BC monitors being
tested. The standard sampling inlet used by the RAAS sampler will be used to prevent liquid
water from being drawn into the sample stream. A 2.5 |j,m cyclone inlet inside the sampler will
be used to limit the size of particles in the sample stream to less than an aerodynamic diameter of
2.5 [im.
-------
Black Carbon Monitors
QAPP
Page 2 7 of 4 2
April 12, 2013
EPA/600/R-16/032
Audits of filter sampling procedures will be carried out by Battelle testing staff as part of the
ISA procedure (Section C1.2) and the PE audit procedure (Section Cl.l).
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Sample handling procedures are designed to minimize handling of the reference filter cassettes
and limit the number of transfers of the filters. Upon receipt, DRI will inspect the filters and pre-
fire them according to DRI Standard Operating Procedure (SOP) #2-106.The filters will then be
loaded into separate, pre-labeled Petri dishes in DRI's laboratory facilities, sealed, and shipped to
Battelle for sampling. When not in use, the filters will remain in their sealed Petri dishes and
stored in an environmentally controlled environment (e.g., in the mobile laboratory), to prevent
contamination. The filters will be loaded into sampling cassettes inside the mobile laboratory by
Battelle testing staff using clean, lint-free gloves and will be loaded into and unloaded from the
reference sampler by Battelle testing staff. The filters will be installed in the sampler and
sampled continuously for 12 hours after which the exposed cassettes will be retrieved from the
sampler and clean, unexposed cassettes will be installed. Upon retrieval, the exposed filters will
be removed from the sampling cassettes, resealed in their respective Petri dishes and stored
refrigerated until shipment back to DRI for analysis. The exposed samples will be shipped to
DRI within 7 days of collection.
Sample custody will be documented throughout sample preparation, sample collection, sample
recovery, and sample analysis, using standard chain-of-custody forms provided by DRI. Each
chain-of-custody form will be signed by the person relinquishing samples once that person has
verified that the chain-of-custody form is accurate. Upon receipt at the laboratory, chain-of-
custody forms will be signed by the person receiving the samples once that person has verified
that all samples identified on the chain-of-custody forms are present in the shipping container.
Any discrepancies will be noted on the form and the sample receiver will immediately contact
the VTC to report missing or compromised samples. Copies of all chain-of-custody forms will
be delivered to the VTC upon request, and maintained with the test records.
-------
Black Carbon Monitors
QAPP
Page 28 of 42
April 12, 2013
EPA/600/R-16/032
B4 ANALYTICAL METHOD REQUIREMENTS
Analysis of the reference method samples will be performed by the IMPROVE_A_TOR method
as described in the DRI SOP #2-216r3, dated October 22, 2012,4 This method is "based on the
preferential oxidation of organic and elemental carbon (OC and EC) compounds at different
temperatures." In general, this method calls for a small punch from a quartz fiber filter to be
heated under prescribed temperature profiles in different oxidative atmospheres and for the
liberated carbonaceous species to be monitored with a flame ionization detector (FID) after
conversion first to CO2 and subsequently to CH4.
B5 QUALITY CONTROL REQUIREMENTS
As described in Section A7, reference method sampling will be carried out using a RAAS
speciation sampler. Analysis of the reference samples will be performed by DRI using the
IMPROVE_A_TOR method. A variety of quality control activities will be performed to ensure
the data quality of the reference method. Specific quality control activities include calibration
and verification of the reference sampling equipment, calibration and verification of the
analytical instrumentation, and analysis of duplicate samples and field blanks. The following
sections describe the quality control (QC) activities and acceptance criteria for the reference
method sampling and analysis. After initial installation of the BC analyzers being tested, no
additional QC activities are planned for those analyzers other than those performed automatically
by the analyzers as part of their routine operation.
B5.1 Reference Sample Collection
Quality control activities for the filter sampling include calibration of the sampler flow rate and
of the sampler temperature and pressure sensors. After the initial calibration, checks of the
sampler flow rate and the temperature/pressure sensors will performed after every third sampling
day. After each flow rate check, each sampling train will be checked for leaks to ensure proper
-------
Black Carbon Monitors
QAPP
Page 29 of 42
April 12, 2013
EPA/600/R-16/032
operation. Additionally, a field blank sample will be collected at least every fifth sampling day.
The field blanks will be collected by installing a clean, unexposed filter into a filter cassette and
installing the loaded cassette into one of the reference sample channels without drawing any air
through the filter. The cassette will then be recovered and the filter will be removed and handled
like a normal sample.
B5.2 Reference Sample Analysis
The analysis of the reference samples will be conducted by DRI according to the
IMPROVE_A_TOR method using a DRI Model 2001 carbon analyzer. In this method, a portion
of the filter sample is heated under various gaseous atmospheres (pure He and 98% He/2% 62) to
preset temperature conditions between 140 and 840 °C and the evolved gas is analyzed for the
presence of carbonaceous species. During analysis, the reflectance of the filter is monitored to
correct for artifacts introduced during the analysis procedure.
Table 3 summarizes the quality control requirements of the method. If the sampling or analytical
performance strays outside the required tolerances, the relevant QC checks will be conducted
again or the relevant QC samples will be prepared again and reanalyzed. If performance
problems persist, the reference instrument will be recalibrated, and/or affected samples will be
reanalyzed. Reference sample results not meeting the requirements will be excluded from
comparison to the continuous monitor results.
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND
MAINTENANCE
The equipment used for the reference method sampling and analysis will be tested, inspected,
and maintained so as to meet the data quality objectives of this verification test. System
preventive maintenance will be performed prior to the start of the verification test and for each
sampling period as needed. Each analyzer will be checked daily to ensure proper operation, and
-------
Black Carbon Monitors
QAPP
Page 30 of 42
April 12, 2013
EPA/600/R-16/032
vendors will be notified when repairs may be required. Laboratory equipment maintenance is
conducted according to the laboratory SOP.4
Table 3. QC Requirements for the Analytical Method
Measured
Parameter
Range
Accuracy
Detection limit
Accuracy
Accuracy
Accuracy
Accuracy
Accuracy
Accuracy
Precision
QC Check
Multipoint calibration
(daily when analysis is performed)
Multipoint calibration
( Every 6 months or after major
instrument repair)
System blank check
(daily when analysis is performed)
Leak check
(daily when analysis is performed)
Laser performance check
(daily when analysis is performed)
Calibration peak area check
(Every analysis)
Auto-calibration check
(daily when analysis is performed)
Manual injection calibration
(daily when analysis is performed)
Sucrose calibration check
(thrice per week)
Analysis of duplicate samples
(every 10th sample)
Required Performance
Concentrations must bracket range of all
sample concentrations
All slopes ±5% of average
<0.2 ug TC/cm2
Oven pressure drops less than 0.52 mm Hg/s
Transmittance >700 mV;
reflectance >1 ,500 mV
Counts >20,000 and 95-105% of average
calibration peak area of the day
95-105% recovery and calibration peak area
90-1 10% of weekly average
95-105% recovery and calibration peak area
90-1 10% of weekly average
95-105% recovery and calibration peak area
90-1 10% of weekly average
±10% when OC and TC >10 ug C/cm2;
±20% when EC >10 ug C/cm2, or
<±1 ug C/cm2 when OC and TC <10 ug
C/cm2; <±2 ug C/cm2 when EC <10 ug C/cm2
B7 INSTRUMENT CALIBRATION AND FREQUENCY
B7.1 Reference Method Sampling Calibration
The calibration for the reference method sampling trains includes a single point flow rate
calibration at the nominal flow rate using an Si-traceable flow transfer standard. This calibration
will be performed at the beginning of the verification test with flow checks performed no less
frequently than once every three days throughout the verification test. Flows will be adjusted if
-------
Black Carbon Monitors
QAPP
Page 31 of 42
April 12, 2013
EPA/600/R-16/032
measured flows are found to differ from the nominal flow rate by more than 5% (i.e. acceptable
flow range of 15.9 - 17.5 L/min).
Verification of the calibration of the temperature and pressure sensors in the reference sampler
will be performed by conducting single point calibration checks of the sensors at the beginning
of the verification test using Si-traceable standards. Agreement between the readings from the
reference sampler and the calibration devices must be within ± 2 °C for the temperature sensor
and ± 5 mmHg for the pressure sensor, or the sensors will be recalibrated.
B7.2 Analytical Instrumentation Calibration
Prior to sample analysis, a calibration of the DRI analysis equipment will be conducted
according to the method SOP.4 Also, calibration checks will be conducted as described in
Section B.5.2.
B7.3 Continuous BC Monitors
Upon installation at the field site the BC monitors being tested will be calibrated by the
respective vendors according to their recommended procedures. These activities will be
documented by Battelle. The Model 4 OC-EC analyzer performs an automatic calibration check
using a certified methane standard with every sample analysis. The Model AE33-7
Aethalometer automatically monitors internal operational parameters and provides an indication
if recalibration is needed. Recalibration is not likely to be needed within the 30-day period of the
field test, but if needed will be performed by the vendor.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
Upon receipt of any supplies or consumables used for the reference method, the DRI staff will
visually inspect the materials and ensure that the materials received are those that were ordered
and that there are no visual signs of damage that could compromise the suitability of the
materials. If damaged or inappropriate goods are received they will be returned or disposed of
-------
Black Carbon Monitors
QAPP
Page 32 of 42
April 12, 2013
EPA/600/R-16/032
and arrangements will be made to receive replacement materials. Certificates of analysis (COA)
or other documentation of analytical purity will be checked for all gases, reagents, and standards
to ensure suitability for this verification test. Unsuitable materials will be returned or disposed of
and arrangements for the receipt of replacement materials will be made.
B9 NON-DIRECT MEASUREMENTS
No non-direct measurements will be used during this verification test.
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by Battelle and
the instrument vendors during this verification test. All manually recorded data will be recorded
in permanent ink. Corrections to written records will be made by drawing a single line through
the entry to be corrected and providing a simple explanation for the correction, along with a date
and the initials of the person making the correction. Table 4 summarizes the types of data to be
recorded. All maintenance activities, repairs, calibrations, and operator observations relevant to
the operation of the monitoring systems being tested will be documented by Battelle or vendor
staff in the laboratory record book (LRB).
Results from the reference method will be compiled by DRI staff in electronic format, and
submitted to Battelle in the form of an analytical report at the conclusion of reference sample
analyses.
Records received by or generated by any Battelle or vendor staff during the verification test will
be reviewed by a Battelle staff member within one week of receipt or generation and before the
records are used to calculate, evaluate, or report verification results. If a Battelle staff member
generated the record, this review will be performed by a Battelle technical staff member involved
-------
Black Carbon Monitors
QAPP
Page 33 of 42
April 12, 2013
EPA/600/R-16/032
in the verification test, but not the staff member who originally received or generated the record.
The review will be documented by the person performing the review by adding his/her initials
Table 4. Summary of Data Recording Process
Data to Be Recorded
Dates, times, and
details of test events
BC monitor calibration
information,
maintenance, down
time, etc.
BC monitor readings
Reference method
procedures,
calibrations, QA, etc.
Reference method
analysis results
Where
Recorded
ETV LRBs, field
sampling records
ETV LRBs, or
electronically
Recorded
electronically by
each monitor and
then downloaded
to computer daily
ETV LRBs, or
data recording
forms
Electronically
from analytical
method
How Often
Recorded
Start/end of test
event
When
performed
Recorded
continuously by
each monitor
Throughout
sampling and
analysis
processes
Every sample
analysis
By Whom
Battelle
Vendor or
Battelle
Vendor for
transfer to
Battelle
Battelle
DRI
Disposition of Data
Used to
organize/check test
results; manually
incorporated in data
spreadsheets as
necessary
Incorporated in
verification report as
necessary
Converted to
spreadsheet for
statistical analysis
and comparisons
Retained as
documentation of
reference method
performance
Converted to
spreadsheets for
calculation of
ambient air
concentrations, and
statistical analysis
and comparisons
and date to the hard copy of the record being reviewed. In addition, any calculations performed
by Battelle will be spot-checked by Battelle technical staff to ensure that calculations are
performed correctly. Calculations to be checked include any statistical calculations described in
this QAPP.
Among the QA activities conducted by Battelle QA staff will be an audit of data quality. This
audit will consist of a review by the Battelle QAO of at least 10% of the test data. During the
-------
Black Carbon Monitors
QAPP
Page 34 of 42
April 12, 2013
EPA/600/R-16/032
course of any such audit, the Battelle Q AO will inform the technical staff of any findings and
any immediate corrective action that should be taken. If serious data quality problems exist, the
Battelle QAO will inform the AMS Center Manager who is authorized to stop work. Once the
assessment report has been prepared, the VTC will ensure that a response is provided for each
adverse finding or potential problem, and will implement any necessary follow-up corrective
action. The Battelle QAO will ensure that follow-up corrective action has been taken.
-------
Black Carbon Monitors
QAPP
Page 35 of 42
April 12, 2013
EPA/600/R-16/032
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
One of the major objectives of the QAPP is to establish mechanisms necessary to anticipate and
resolve potential problems before data quality is compromised. Internal QC measures described
in this QAPP will yield day-to-day information on data quality. The responsibility for
interpreting the results of these checks and resolving any potential problems resides with the
VTC. Technical staff has the responsibility to identify problems that could affect data quality or
usability. Any problems that are identified will be reported to the VTC, who will work with the
Battelle QAO to resolve any issues. Action will be taken to identify and appropriately address
the issue and minimize losses and correct data, where possible. Battelle will be responsible for
ensuring that the audits described in the following subsections are conducted as part of this
testing. See Table 5 for the proposed schedule of audits.
Any departures from the approved QAPP must be reported within 24 hours and documented in a
formal deviation submitted to the Battelle AMS Center Manager, QM, and EPA PO. If approval
by EPA or his designee is not received within 24 hours of notification testing will be halted until
a suitable resolution has been achieved.
Cl.l Performance Evaluation Audit
A PE audit will be conducted either prior to the beginning of the testing or within the first week
of testing to assess the quality of the critical measurements associated with the reference
sampling and analysis methods. Table 5 shows the critical measurements to be audited, with the
audit procedures and acceptance criteria for the audit comparisons. If the PE audit results do not
meet the acceptance criteria shown, they will be repeated. If the outlying results persist, a
change in reference method instrument and a repeat of the PE audit may be considered, and data
-------
Black Carbon Monitors
QAPP
Page 36 of 42
April 12, 2013
EPA/600/R-16/032
will be flagged until the PE audit results are acceptable. This audit will be performed once
during the verification test, and will be the responsibility of the VTC or designee.
Table 5. Methods and Acceptance Criteria for PE Audit Measurements
Critical Measurement
Reference method
sampling flow rate
Reference sampler
temperature sensor
Reference sampler
pressure sensor
PE Audit Method
Flow rate check with independent Si-
traceable flow rate standard
Check with independent Si-traceable
temperature sensor
Check with independent Si-traceable
pressure sensor
Acceptance Criteria
± 5% of nominal flow rate
±2°C
± 5 mm Hg
The PE audit of the filter sampling flow rate will be conducted using an independent Si-traceable
flow transfer standard. With an unused filter installed, the flow rate through the sampling train
will be measured and compared to the nominal flow rate. This filter will not be used as one of
the reference samples. The target criterion for this audit is agreement between the measured and
nominal flow rate within ±5%. If this criterion is not met, the cause of the problem will be
investigated and corrected if possible. Components of the sampling train will be replaced as
necessary until the flow rate criterion is met.
C1.2 Technical Systems Audits
The Battelle QAO will perform a TSA within the first week of testing. The purpose of this audit
is to ensure that the verification test is being performed in accordance with the AMS Center
QMP,1 this QAPP, published reference methods, and any SOPs used by the analytical laboratory.
In this audit, the Battelle QAO may review the reference methods used, compare actual test
procedures to those specified or referenced in this plan, and review data acquisition and handling
procedures. The Battelle QAO will prepare a project-specific checklist based on the QAPP
requirements to guide the TSA, which will include a review of the test location and general
testing conditions; observe the testing activities; and review laboratory record books. In the
TSA, the Battelle QAO will tour the test site; observe the reference method sampling and sample
recovery; inspect documentation of reference sample chain of custody; and review laboratory
-------
Black Carbon Monitors
QAPP
Page 3 7 of 42
April 12, 2013
EPA/600/R-16/032
record books. He will also check data acquisition procedures, and may confer with the vendor
and Battelle testing staff. A ISA report will be prepared as a memo to the VTC within 10
business days after completion of the audit; the completed checklist will be attached. The
Battelle AMS Center Manager, QM, and EPA PO will be copied on the memo. The VTC will
respond to the audit within 10 business days. The Battelle QAO or designate will verify that all
audit Findings and Observations have been addressed and that corrective actions are
appropriately implemented. A copy of the complete TSA report with corrective actions will be
provided to the EPA PO within 10 business days after receipt of the audit response. At EPA's
discretion, EPA QA staff may also conduct an independent on-site TSA during the verification
test. The TSA findings will be communicated to technical staff at the time of the audit and
documented in a TSA report..
C1.3 Data Quality Audit
The Battelle QAO will audit at least 10% of the sample results acquired in the verification test
and 100% of the calibration and QC data per the QAPP requirements. A checklist based on the
QAPP will guide the audit. An initial ADQ will be conducted on the first batch of test data
within 10 business days of when data were posted on the project SharePoint site to identify errors
early in the data reduction process. The first batch is defined as the testing and variable data
generated over the first week of testing. The remaining data will be audited at the completion of
the 30 day field test after all data has been posted on the project SharePoint site and once all
statistical analyses for that set of tests are complete. During this ADQ, the Battelle QAO will
trace the data from initial acquisition, through reduction and statistical comparisons, to final
presentation in the verification reports. It will also confirm reconciliation of the initial ADQ.
All formulae applied to the data will be verified, and 10% of the calculations will be checked.
Data for each set of tests will be reviewed for calculation and transcription errors and data
traceability. An audit report will be prepared as a memo to the VTC within 10 business days
after completion of each data audit; the completed checklist will be attached. The Battelle AMS
-------
Black Carbon Monitors
QAPP
Page 38 of 42
April 12, 2013
EPA/600/R-16/032
Center Manager, QM, and EPA PO will be copied on the memo. The TC will respond to the
audit within 10 business days. The Battelle QAO will verify that all audit Findings and
Observations have been addressed and that corrective actions are appropriately implemented. A
copy of the complete ADQ report with corrective actions will be provided to the EPA PO within
10 business days after receipt of the audit response. EPA QA staff may also conduct an
independent ADQ.
C1.4 QA/QC Reporting
Each assessment and audit will be documented in accordance with Section 10.5 of the AMS
Center QMP.1 The results of the TSA will be submitted to EPA. Assessment reports will include
the following:
• Identification of Findings and Observations
• Recommendations for resolving problems
• Response to adverse findings or potential problems
• Confirmation that solutions have been implemented and are effective
• Citation of any noteworthy practices that may be of use to others.
C2 TO
The Battelle QAO, during the course of any assessment or audit, will identify to the
technical staff performing experimental activities any immediate corrective action that should be
taken. If serious quality problems exist, the Battelle Quality Manager will notify the AMS Center
Manager, who is authorized to stop work. Once the assessment report has been prepared, the
VTC will ensure that a response is provided for each adverse finding or potential problem and
will implement any necessary follow-up corrective action. The Battelle Quality Manager will
ensure that follow-up corrective action has been taken. The QAPP and final report are reviewed
by EPA AMS Center quality assurance staff and the EPA AMS Center program management
staff. Upon final review and approval, both documents will then be posted on the ETV website
-------
Black Carbon Monitors
QAPP
Page 39 of 42
April 12, 2013
EPA/600/R-16/032
(www.epa.gov/etv). A summary of the required assessments and audits, including a listing of
responsibilities and reporting timeframes, is included in Table 6.
Table 6. Summary of Assessment Reports3
Assessment
TSA
ADQ1
(first batch)
ADQ2
(reduced data
and verification
report)
Prepared By
Battelle
Battelle
Battelle
Report Submission
Timeframe
TSA response is due to QAO
within 10 business days
TSA responses will be verified
by the QAO and provided to
EPA within 20 business days
ADQ will be completed within
10 business days after receipt
of first data set
ADQ will be completed within
10 business days after
completion of the verification
report review
Submitted To
EPA ETV AMS Center
EPA ETV AMS Center
EPA ETV AMS Center
a. Any QA checklists prepared to guide audits will be provided with the audit report.
-------
Black Carbon Monitors
QAPP
Page 40 of 42
April 12, 2013
EPA/600/R-16/032
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VERIFICATION, AND VALIDATION
REQUIREMENTS
The key data review and data verification requirements for this test are stated in Section A7 of
this QAPP. In general, the data review requirements specify that data generated during this test
will be reviewed by a Battelle technical staff member within one week of generation of the data.
The reviewer will be familiar with the technical aspects of the verification test but will not be the
person who generated the data. This process will serve both as the data review and the data
verification, and will ensure that the data have been recorded, transmitted and processed
properly. Furthermore, this process will ensure that the monitoring systems data and reference
method data were collected under appropriate testing conditions and that the reference sample
data meet the specifications of analytical methods. Time series plots will be generated to assess
potential anomalies in the data.
The data validation requirements for this test involve an assessment of the quality of the data
relative to the DQIs and audit acceptance criteria specified for this test. The DQIs listed in
Section A7 will be used to validate the quality of the data. The QA audits described within
Section C of this document, including the PE audit and the audit of data quality, are also
designed to validate the quality of the data.
D2 VERIFICATION AND VALIDATION METHODS
Data verification is conducted as part of the data review as described in Section A7 of this
QAPP. A visual inspection of handwritten data will be conducted to ensure that all entries were
-------
Black Carbon Monitors
QAPP
Page 41 of 42
April 12, 2013
EPA/600/R-16/032
properly recorded or transcribed, and that any erroneous entries were properly noted (i.e., single
line through the entry, with an error code and the initials of the recorder and date of entry).
Electronic data from the BC monitors and analytical equipment used during the test will be
inspected to ensure proper transfer from the datalogging system. All calculations used to
transform the data will be reviewed to ensure the accuracy and the appropriateness of the
calculations. Calculations performed manually will be reviewed and repeated using a handheld
calculator or commercial software (e.g., Excel). Calculations performed using standard
commercial office software (e.g., Excel) will be reviewed by inspection of the equations used for
the calculations and verification of selected calculations by handheld calculator. Calculations
performed using specialized commercial software (i.e., for analytical instrumentation) will be
reviewed by inspection and, when feasible, verified by handheld calculator, or standard
commercial office software. The Battelle staff member doing the verification will document
the activities.
To ensure that the data generated from this test meet the goals of the test, a number of data
validation procedures will be performed. Sections B and C of this QAPP provide a description
of the validation safeguards employed for this verification test. Data validation efforts include
the completion of QC activities, and the performance of ADQ and PE audits as described in
Section C. The data from this test will be evaluated relative to the measurement DQIs described
in Section A7, and the PE audit acceptance criteria given in Section Cl.l of this QAPP. Data
failing to meet these criteria will be flagged in the data set and not used for evaluation of the
monitoring systems, unless these deviations are accompanied by descriptions of their potential
impacts on the data quality.
An audit of data quality will be conducted by the Battelle QAO to ensure that data review,
verification, and validation procedures were completed, and to assure the overall quality of the
data.
-------
Black Carbon Monitors
QAPP
Page 42 of 42
April 12, 2013
EPA/600/R-16/032
D3 RECONCILIATION WITH USER REQUIREMENTS
The purpose of this verification test is to evaluate the performance of continuous monitors for
BC in ambient air. In part, this evaluation will include comparisons of results from the
monitoring systems to the results from reference method samples generated from the
IMPROVE_A_TOR method for sample collection and analysis. To meet the requirements of the
user community, the reference data collected during this verification test will meet the QA
requirements of the reference method. Additional performance data regarding operational
characteristics of the BC monitors will be collected by verification test personnel. To meet the
requirements of the user community, these data will include thorough documentation of the
performance of the monitoring systems during the verification test. The data review,
verification, and validation procedures described above will ensure that data meeting these
requirements is accurately presented in the verification reports generated from this test, and will
ensure that data not meeting these requirements will be appropriately flagged and discussed in
the verification reports.
This QAPP and the resulting ETV verification report(s) will be subjected to review by the
vendor, EPA, and expert peer reviewers. The reviews of this QAPP will assure that this
verification test and the resulting report(s) meet the needs of potential users of these monitoring
systems.
-------
Black Carbon Monitors
QAPP
Page 43 of 42
April 12, 2013
EPA/600/R-16/032
SECTION E
REFERENCES
1. Battelle, Quality Management Plan for the ETV Advanced Monitoring Systems Center,
Version 8.0, U.S. EPA Environmental Technology Verification Program, prepared by
Battelle, Columbus, Ohio, April 2011.
2. U.S. EPA, Environmental Technology Verification Program Quality Management Plan, EPA
Report No: EPA 600/R-08/009, U.S. Environmental Protection Agency, Cincinnati, Ohio,
January 2008.
3. Report to Congress on Black Carbon, EPA-450/R-12-001, U.S. Environmental Protection
Agency, March 2012, available at http://www.epa.gov/blackcarbon/.
4. Desert Research Institute, DRI Model 2001 Thermal/Optical Carbon Analysis (TOR/TOT) of
Aerosol Filter Samples - Method IMPROVE_A, DRI SOP#2-216r3, prepared by DRI, Reno,
NV, October 22, 2012.
------- |