U.S. EPA Technical Systems Audit Supplementary Checklist for PM2.5 Continuous Monitors Draft 5/12/2015

5/12/2015 - Draft - feedback is requested after TSA's are completed.

U.S. EPA
Technical Systems Audit
Supplementary Checklist for PM2.5 Continuous Monitors

Introduction and Overview:

This checklist is intended to support EPA Regional staff conducting Technical Systems Audits of PM2.5 continuous monitors
operated by State, local, or Tribal monitoring agencies. This checklist supplements Appendix H of the QA Handbook Volume II
(http://www.epa.gov/ttn/amtic/files/ambient/pm25/qa/QA-Handbook-Vol-ll.pdf). The checklist attempts to address the most
important aspects of ensuring a PM2.5 continuous monitor is set-up, operating, and reporting data to meet the intended monitoring
objectives (e.g., NAAQS and AQI). This checklist does not attempt to redundantly capture the activities already addressed in the
Appendix H checklist; however, there are a few places where an item already addressed in Appendix H is further detailed to ensure
the auditor has the key information to conduct the audit.

The audit is performed by looking at a number of documents and assessments in advance of travelling to a site and then by
inspecting how the monitor is set-up, operated, maintained, and reporting data at the site to ensure the method can appropriately
support its intended monitoring objectives.

While this checklist is fairly detailed, it attempts to balance calling attention to the best practices without having so much
detail that the auditor might not be able to successfully complete addressing all parts of the checklist. Additionally, and where
appropriate, illustrations and images are included to provide a reference for the auditor. If there is a need for a visual to illustrate a
setting or practice that is not included, please ask so that we can continue to improve this checklist.

This checklist is organized into three basic areas that cover the Technical Systems Audit (TSA):

1. Preparation and Planning: In advance of travelling to an agency, the auditor should review the agency's annual monitoring
network plan, assessments of data, and other documentation (e.g., list of designated reference and equivalent methods) to
determine if the monitor is meeting its stated objectives and producing data that meets the needs of the data users, (i.e.,
meeting DQOs). This preparation and planning should be very similar regardless of the specific method being run by the
monitoring agency.

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

2. On-Site Technical Systems Audit: While on site the auditor should spend time reviewing the set-up, operation, maintenance,
and reporting of data to ensure the method is functioning appropriately. If there are items observed during the audit that
can be addressed while on-site without compromising data quality (e.g., cleaning the VSCC during the period of time the
monitor is not pulling air), then these should be communicated immediately to the agency staff. The on-site audit activities
are organized in a method-specific checklist.

3. Follow-Up and Review: The auditor should ensure a written report is provided in a timely manner to the monitoring
agency. Audit findings should be grouped into either: a. Significant Findings, or b. Observations. Significant findings should
address those items that appear to affect data quality in an adverse way. Observations should include items that help ensure
the method is operating appropriately and efficiently. Additionally, for Significant Findings, the auditor should request that
the agency provide documentation to the EPA Regional office that the items are addressed. For example, zero test data
could be provided to document that a new zero test was conducted.

1. Preparation and Planning:

Regional staff should prepare for a TSA in advance of travelling to meet with an agency and their sites by reviewing several
documents, assessments, and reports of data. This preparation will help to ensure the auditor becomes familiar with the method
and use of the data as well as to help inform areas that should be focused on during the TSA.

Below find a table with recommended documents, assessments, and data-bases that should be reviewed prior to going into
the field.

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Table 1 - Preparation Table:

Document or
Assessment or To Do

Where or who to find

What to look for or Document or Data?

Comments

Annual Monitoring Network
Plan.

Agency's current and/or
planned annual Monitoring
Network Plan

Review and identify the sites you will visit that
have PM2.5 continuous monitors; whether they
are pre-FEM or FEM; and their stated
monitoring objectives.

Note any FEMs that are either already approved for
exclusion or requested for exclusion from the PM2.5
NAAQS due to not meeting performance
requirements.

Review latest copy of "List of
Reference and Equivalent
Methods" and compare to
annual plan.

http://www.epa.gOv/ttn/amtic/f
iles/ambient/criteria/reference-

equivalent-methods-list.pdf

Read method designation description for PM2.5
continuous FEMs that are at sites you will visit.
Document method designation number and
minimum firmware requirements in
descriptions.

Instrument company web
site

Look on line for the instrument
company

Look for latest version of instrument manual
and firmware for FEMs; however, you should
not need to download these.

Notes:

1. Some instrument company web sites may be
password protected.

2. In cases where a Pre-FEM monitor is being run,
but the latest manual is specific to the FEM
version, the agency may not need the latest
version of the manual.

Run AQS reports (e.g.,
AMP450NC) and ensure all
PM2.5 continuous monitors
identified in Annual Plan are
reporting to AQS.

Use AQS if you have an active user login and password.

Alternatively, use AirData (htto://www, epa.gov/airdata/) if you do not have ready
access to AQS. All ambient PM data in AirData originates from AQS via the
DataMart.

Monitors reporting to parameter code 88101 will be
used in NAAQS and AQI.

Monitors reporting to parameter code 88502 will be
used in the AQI.

PM2.5 continuous monitor
comparability assessment
tool

http://www.epa.gov/airquality/
airdata/ad rep frmvfem.html

For sites that will be visited, run assessment
and identify any performance issues, (primarily
focus on additive and multiplicative bias)

Note: a collocated FRM is required for the
assessment to run.

Run an hourly data report for
the PM2.5 continuous
monitor

AMP 350 or download data

Verify the following:

1. What is the lowest hourly reading
recorded to AQS? There should be
some data less than 0. If not, note and
review data farther back in time

2. What is the highest hourly value
recorded to AQS? There should be no
data at 985 (or close to it if using
analog)

An absence of any slightly negative (down to -10
ug/m3) data may be biasing the 24-hour averages
high. Among PM2.5 FEMs, only the GRIMM is known
to never have a negative hourly value.

Significantly high data at or near 985 ug/m3 indicate
that the instrument has gone to full scale. This is
usually associated with an error code and should not
have been reported.

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Document or
Assessment or To Do

Where or who to find

What to look for or Document or Data?

Comments

Also, the hourly data record is a good way to
check that the instrument was off line during
period of zero test.

Assessment of collocated
FRM data.

Where appropriate, this will
help to determine whether
the issue with the PM2.5 data
is the continuous FEM
monitor or FRM sampler.

AMP 256

Where there are comparability issues between
the FRM and continuous FEM, we would like to
verify that the FRM to FRM collocation meets
DQOs.

Look to see that the FRM to FRM CV UB is less
than or equal to the 10% CV?

In cases where the "Collocated Detail Report" is
providing the CV UB for the PM2.5 continuous
monitor collocated with an FRM, the calculation is
often relatively higher than the FRM to FRM
collocation, even in cases where the additive and
multiplicative bias statistic are met.

Reports of flow verifications
and audits

AMP 251

Are flow rate verifications and audits
acceptable?

Flow audits are required to be reported to AQS.

Flow verifications are not required to be reported to
AQS, but they can be.

Interview Data Users in
Region; find out what is
working well or any
concerns.

e.g., ESAT PEP field scientist

Ask if there are any known or potential issues in
the set-up, operation, maintenance, or
reporting of the PM2.5 continuous monitors?

Review AIRNow reporting
and ensure PM2.5 continuous
monitors identified in Annual
Plan are reporting to AIRNow
and available for public
reporting of data. Note:
AIRNow reporting is not
technically "required";
however, participating in
AIRNow fulfills the
requirements of Appendix G
for AQI reporting.

www.AirNowTech.org

If you do not have access, you
can request it or check with
colleagues who may already
have access.

Ensure monitors are being publically reported.

Public Maps (without needing to log in) are available

at: httpi//www,airnow.gov/index.cfm?action=airno
w.pointmaps

These maps, while national in scale, may be able to
provide a quick visual assessment if a site is
reporting PM2.5 continuous data publically.

SOP

Ask agency in advance

1. Adherence to method requirements
prescribed in instrument company
manual (e.g., settings in firmware)

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Document or
Assessment or To Do

Where or who to find

What to look for or Document or Data?

Comments





2. Incorporation of best practices (e.g.,
are more frequent zero tests
performed, where appropriate?)











5

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

2. On-site Technical Systems Audit of Met One BAM 1020

Conducting the on-site audit of the Met ONE BAM 1020 involves
evaluating several aspects of the set-up, operation, maintenance,
and reporting of the monitor. For this checklist we have grouped
the questions into four areas:

a. Items to review at the monitor

b. Items to review in the firmware and data logger

c. Items to review at the inlet.

d. Maintenance and QC records to review

BX-802
PMio Head

BX-808
BGI VSCC™
PM2.5 Cyclone

BX-596
AT/BP Sensor

8112
Inlet Tube

8' standard

This checklist can be applied to either pre-FEM or FEM Met One
BAM 1020 monitors. Pre-FEM Met One BAM 1020 monitors will be
set up and operated nearly identical to an FEM, except that they
may use a Sharp Cut Cyclone (SCC) rather than a VSCC and that the
version of firmware may be prior to version 3.2.4. There may be
other engineering differences between a pre-FEM and FEM Met
One BAM 1020; however, these will likely be unnoticeable to the
user or auditor. To the extent that any differences exist, it does not
necessarily mean that the pre-FEM version is not being run
appropriately.

Typical Met One BAM 1020 Set-up:

Note: Dimensions are typical and not necessarily part of a
requirement.

Inlet Support
Struts

BX-801
Roof Flange

Enclosure
Roof

BX-827 or 830
Smart Heater

with insulation

vacuum pump or
equivalent

Mounting

Bench or Rack

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Table 2 -Audit Questions for the Met One BAM 1020 PM2.5 Continuous Monitor:

Question #

Item

Response

Comments

Items to review at the Monitor:

Confirm the make and model of the
PM2.5 continuous monitor?

Does this make and model match
what is identified in the annual plan
and reported to AQS?

~ Yes

~ No

If expected, is there an FEM sticker
on the PM2.5 continuous Monitor?

~ Yes

~ No

Automated Equivalent Method: EQPM-0308-170

Are there any concerns about the
location of the monitor inside the
shelter?

~ Yes

~ No

This is largely professional judgement. Items of concern might
include: substantial vibration where monitor is set-up; AC
blowing directly on down tube; or poor access to monitor.

Is the latest SOP and Instrument
manual for the PM2.5 continuous
monitor available at the station?

~ Yes

~ No

The latest instrument manual version is Revision K.

Is the SOP current? Identify approval
date.

~ Yes

~ No

If practicable, look for approval before trip.

How far is the sampling pump placed
away from the BAM? Is the pump
isolated so as to minimize vibration
to the monitor?

~ Yes

~ No

Distance between the BAM and pump will help to minimize
vibration. Even placed a few feet away isolated on foam or
rubber mat will help.

Is the chassis of the monitor ground
to an earth ground? How is this
grounded?

~ Yes

~ No

This is in addition to the ground associated with the electrical

cord. Grounding of the pump is also

recommended. Photo at right

illustrates green/yellow ground line

leading from back of chassis.

Is the inlet down tube grounded with
two set screws at the receiving collar
of the monitor?

~ Yes

~ No

Ask operator if they have tested down tube grounding. While
not required, this can be confirmed by testing continuity
between the down tube and monitor chassis; however, a small

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Question #

Item

Response

Comments

This should be done by cranking hard
on the set screws to pierce the
aluminum anodizing, then back out
and gently tighten. Over-tightening
will cause leaks.

section of anodized aluminum would need to be scratched off
on the down tube to test this. A down tube that is not
grounded may be susceptible to electrical static build up,
which can lead to a noisier concentration output.

Describe any heat on or near the
smart heater? Does it appear the
smart heater is operating? Is the
downtube insulated? (although not
required, this may help the smart
heater perform better)

~ Yes

~ No

The smart heater has two power levels: a high level and a low
level. Even at the low level there should be a small amount of
noticeable heat at the down tube near the smart heater. A
smart heater that is not working may have a cold down tube.

Ask operator to open door of
monitor. Observe tape set-up; has
the tape run out or are there any
other visible issues?

~ Yes

~ No

The nozzle sits on the tape during
sampling right in the center of the unit.

w ¦

'51 n:

" ; *| • o

O ? o

Observe the tape that has already
sampled; if applicable, describe any
pin holes that may appear on the
tape?

~ Yes

~ No

Pinholes are a symptom of the nozzle pinching down on the
tape. Pinholes will result in excessively lower negative
readings in the data.

Observe the nozzle as it sites on the
filter tape. If applicable, describe
any visible debris on the bottom of
the nozzle?

~ Yes

~ No

The nozzle and vane are to be cleaned at regular intervals
(typically monthly) to ensure debris does not build up on
bottom of nozzle.

Is there build-up on the capstan
rollers?

~ Yes

~ No

M-'

• The capstan roller is located to the left of the nozzle.

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Question #

Item

Response

Comments

Inspect the inlet receiver where the
downtube meets the unit. Look for
any signs of water marks. Another
sign of water may be found on the
serial name plate at the bottom just
inside the door.

~ Yes

~ No

If water marks are found, this maybe an indication that the
shelter seal around the downtube is leaking and either needs
to be tightened or replaced.

Describe any other issues with the
monitor?

~ Yes

~ No

Items to review in the Firmware and data logger:

What version of software is
identified on the monitor?

Firmware is listed on
the main menu
screen.

3/28/2007 BAM 1020 16:08:29

LAST CONCENTRATION: 0.028 mg/m3
CURRENT FLOW: 16.7 LPM
STATUS: ON

FIRMWARE: 3236-02 3.2.4

SF.TTTP OPF.RATF. TF..ST TAPF, \l/

Is this version of software compliant
with firmware requirements for the
method identified in the "List of
Reference and Equivalent Methods"?

~ Yes

~ No

Approved firmware includes:

PM2.5- 3.2.4 or higher; PM10 - 2.0 or higher.

Latest versions of firmware as of Spring 2015 are:

Non-touch BAM - 3236 - 2 R3.7.1 (this is the unit on page 5)

Touch BAM - 3236-55 R2.3.1 (this has a touch screen)

Coarse BAM - 3236-6 R3.7.1

Note: it is not always necessary to upgrade to the latest
firmware; check manufacturer's web site for details.

Review the most recent zero test
data and verify that the zero value
entered in the firmware is the
opposite of the average of that zero
test data.

~ Yes

~ No

The zero setting is entered in the field "BKGD".

This is located in the "CALIBRATE SETUP" screen, which is
illustrated in the center screen shot in question 24. Note: it is
critical that the value entered in this field is the negative of the
average from the most recent zero test. For example, if the
zero test produces an average of 1.2 ug/m3, then the value
entered is "-0.0012".

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Question #

Item

Response

Comments

In reviewing the most recent zero
20 test, was the standard deviation of
the test data < 2.4 ug/m3?

~	Yes

~	No

21

Observe the last hour reported on
the LCD screen and compare this to
data reported at the station data
logger; are they the same?

~	Yes

~	No

Note: The last hour reported may not be available for a couple
minutes after the top of the hour; however, this data point
represents the previous hour at the start of the hour. For
example an updated value first posted at 10:02 am should
represent hour 09:00.

Is the clock on the BAM 1020 set to
22 run on local standard time (i.e., not
day light savings time)?

~	Yes

~	No

23

Compare time on BAM1020 to time
of data logger; is there any
difference in time and if so how
many minutes?

~ Yes
Min:

~ No

Note: in some cases Met One BAM 1020 Monitors are
purposefully set to have an offset in the time reading. This is
acceptable to ensure the latest BAM reading is interpreted by
the data system to represent the previous hour.

24

Verify firmware settings and note
any discrepancies. From main screen
select:

1.	>SETUP >SAMPLE

2.	>SETUP >CALIBRATE

3.	>SETUP >HEATER

Note 1: Sample will stop to verify
settings. An as found leak test is
recommended prior to stopping
flow.

Note 2: For PMio only (i.e., not
PM 10-2.5) measurements, CONCTYPE

1.	Items in "SAMPLE" screen:
Yes No

~	~ BAM SAMPLE: 042 MIN

~	~ OFFSET: -0.015 mg

~	~ COUNTTIME: 8 MIN

2.	Items in "CALIBRATE" screen:

~	~ CONCTYPE: ACTUAL

~	~ FLOW TYPE: ACTUAL

~	~ FLOW RATE: 16.7 Ipm

~	~ HEATER: AUTO

3.	Items in "HEATER" screen:

RS232 9600 8N1

SETUP SAMPLE
BAM SAMPLE 04 2 MIN

STATION # 01	MET SAMPLE 60 MIN

RANGE 1.000 rrq	OFFSET -0.015 rrq

CONG UNITS mg/rr.3	GOUNT TIME 8 MIN
SAVE EXIT

CALIBRATE SETUP

CONC TYPE
Cv
ABS
K

STD TEMP
SAVE

ACTUAL
1.047
0.822
1.005
25C

FLOW RATE
FLOW TYPE
Qo

]iSW

BKGD
HEATER

16.7
ACTUAL
0.000
0.306
-0.0030
AUTO
EXIT

Note: Cv, ABS, and K are set ot the factory and
will vary from monitor to monitor.

10

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Question #

Item

Response

Comments

is set to "STD" and STD TEMP is set
to "25C."

~	~ RH Control: YES%,

~	~ RH Setpoint 35%,

~	~ Datalog RH: YES (Chan 4)

~	~ Delta-T Control: NO

~	~ Delta-T Setpoint: 99 C,Datalog Delta-T log: This can be set to either
YES or NO. Since it's only the log, it does not affect operation of the monitor.

Heater Setup



RH Control

YES

RH Setpoint

35%

Datalog RH

YES (Chan 4)

Delta-T Control

NO

Delta-T Setpoint

99 C

Datalog Delta-T

NO (Chan 5)

SAVE

EXIT

25

If applicable, describe any other
issues in the firmware or data
logger?

Activities to review at monitors' inlet:

26

Is the PMio inlet a "pie pan" design?

~	Yes

~	No

Note: All PM2.5 continuous monitors nominally
operating at 16.7 Ipm should use the pie-pan inlet
design. Only the GRIMM uses an inlet design
different than the pie pan inlet for PM2.5. Some
PM10continuous FEM monitors and olderTEOMs
(pre-FEM) may use the flat head inlet design.

27

Open the inside of the PM10 inlet.
Describe the cleanliness of the inlet
well.

The PM10 inlet can be inspected near the top of the hour
without disrupting on-going operations as the flow runs for 42

11

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Question #

Item

Response

Comments







minutes. Flow is typically off from minute 50 to minute 8 each
hour.

28

If expected, is the PM2.5 second stage
separator an approved VSCC?

~	Yes

~	No



Notes:

1. VSCC's and SCC's look very similar in
|—J|fl| design.

L-flHJ 2. Also, there are two designs of the VSCC;
U^H3 however, all PM2.5 continuous monitors
should use the design on the left. The other VSCC
design is used with BGI samplers.

29

Open the VSCC. Describe the
cleanliness of the inside of the VSCC.



The VSCC can be inspected near the top of the hour without
disrupting on-going operations as the flow runs for 42 minutes.
Flow is typically off from minute 50 to minute 8 each hour.

30

Is there a gill screen or similar near
the inlet that is also connected to the
PM2.5 continuous monitor to provide
an ambient temperature reading?

~	Yes

~	No

t a

Jl

BAM-1020 FEM2.5 requires BX-596 (AT,BP)
BAM-1020 FEM10 requires BX-592 (AT)

\

1

31

Do the Inlet and down tube appear
perpendicular to the ground?

~	Yes

~	No

Best judgement is fine.

32

Describe any other issues at the
monitors' inlet?

~	Yes

~	No











Maintenance and QC Records to review:

33

Does the agency use an audit sheet
for regular maintenance and
verifications? Or logbook?

~	Yes

~	No

Review audit sheets and/or logbook while on site to ensure
maintenance and verifications are performed at expected
frequencies.

34

Monthly and quarterly checks:
Is there a record documenting that
the following checks are being

Yes No

~	~ Flow check

~	~ Temperature check

~	~ Barometric Pressure

12

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

Question #

Item

Response

Comments

performed at least monthly or
quarterly, where identified?

~ ~ Leak check

~ ~ Nozzle and vane cleaning

~ ~ PM io inlet is cleaned (at least quarterly)

~ ~ VSCC is cleaned

12 month checks:

Is there a record documenting that
the following checks are being
performed at least every 12 months?

Yes No

~ ~ Perform 72 hour zero test (note: more frequent zero tests [e.g.,
seasonally] are recommended, but not required). If an assessment indicates the
monitor is meeting the additive bias requirement, seasonal zero tests may not
be necessary. Nozzle cleaning/rebuild is recommended prior to zero test.

~ ~ Clean down tube

~ ~ Analog voltage audits, if applicable

While on site, also perform the following:

a. Interview the operator and have routine procedures described.

b. Ask for documentation providing evidence that the flow standards being utilized by the operator and the agency's own
auditor (these are required to be separate devices) are NIST traceable and within certification. The Agency office may
need to be contacted for these.

c. Communicate any items that can be addressed in the field without compromising data quality.

3. Follow-Up and Review:

These activities are suggestions to help ensure the TSA leads to actionable improvements in the operation of the method, where
appropriate. Regions that have their own polices for how TSA's are shared should follow those policies as directed by their
management.

a. Review your notes and checklist within 24 hours of leaving the site to ensure you completely document any activities of
note.

-------
5/12/2015 - Draft - Regions can begin working with this version; feedback is requested after TSA's are completed.

b.	Write a draft TSA report of the audit findings within 5 working days of completing travel to the agency. It is important to
get the TSA fully documented while it is fresh in your mind as well as to benefit the monitoring agency if there are issues
that need to be addressed quickly.

c.	In the draft report, group findings into two distinct categories as follows:

I.	Significant Findings - should address those items that appear to affect data quality in an adverse way. The TSA
report should request that the agency provide documentation that these items are addressed. For example, zero
test data could be provided to document that a new zero test was conducted.

II.	Observations - should address items that help to ensure the method is operating appropriately and efficiently.

d.	Share a draft electronic report with the appropriate manager(s) in the monitoring agency. Ask if there are any items that
appear unclear in how to address or which they may disagree with, including a rationale as to why they disagree. Also,
provide a recommended date for the monitoring agency to respond by (e.g., 4 weeks).

e.	With any comments received and/or actionable improvements already made by the agency, draft a final report for
distribution to the agency. Include a recommended timeline to address any remaining significant findings.

14

-------