1

Guidance on the Data Certification Process for Calendar Year 2020 AQS Data

Please see the Questions and Answers on Ambient Air Monitoring Data Certification for
CY2020 Data for information on which data needs certification, the certification process,
the certification reviews, and the certification flag meanings.

OAQPS realizes that the CY2020 data, including data completeness and QA/QC
completeness, for many organizations may have been impacted due to COVID-19. Data
should be appropriately validated, flagged, and submitted to AQS. If a monitoring
organization requires assistance in determining appropriate flags or actions pertaining
to their data (including QA/QC) please work with your Regional Office to determine the
best course of action. Checks of analyzers that produced a measurement to test
technical functionality (e.g., precision check, flow check, etc) of an analyzer or sampler
and are specified as critical criteria in Appendix D, Measurement Quality Objectives and
Validation Templates, of the Quality Assurance Handbook for Air Pollution
Measurement Systems Volume II or in an approved QAPP, were not subject to change.
However, QA/QC checks and data validity are subject to 40 CFR Part 58, Appendix A,
Section 1.2.3, including failure to conduct QA/QC, "weight of evidence", and use of data
submitted.

Certifying Agencies vs. PQAOs

It is recommended that wherever technically feasible, PQAOs be set up as "Certifying
Agencies". A State Agency may choose to be the certifying agency for several PQAOs
within the state. Certifying agencies do not necessarily equate to PQAOs and yet
several summary parameters use data aggregated at the PQAO level, for example:

•	NPAP Data (valid audits and NPAP bias)

•	Collocation Data (PM10, Pb and PM2.5 completeness and CV)

•	PEP Data (PM2.5 and Pb completeness and bias)

•	Pb Analysis Audit Data (completeness, bias)

For the data which are aggregated and assessed at the PQAO level, monitoring
organizations that are part of a larger PQAO but decide to certify the sites/data within
their "certifying agency" will see the same results for the parameters listed above as
other monitoring organizations within the same PQAO. Therefore, AQS recommended
flags for these parameters will be consistently applied to every monitoring organization
within the PQAO. For example, if there are three distinct monitoring organizations within
a PQAO and organization #1 has 4 PM10 sites, organization #2 has 3 PM10 sites, and
organization #3 has 7 PM10 sites, the collocation summary for each organization (if
each organization decides to certify their own data) will identify a total of 14 sites
requiring 2 collocated monitors for the PQAO (14*0.15=2.1). Like the AMP256 QA Data
Quality Indicator Report, the AMP600 will then determine the percent complete and the
precision estimate for the PQAO.

Evaluation of PEP and NPAP Data Suspended for CY20 Certification

The AMP600 will report completeness and bias data of any PEP values reported to
AQS but will not perform any automated evaluations (flagging) of that information.


-------
2

Routine Data Completeness

Data completeness for routine monitoring data for the AMP600 is based on the sample
period start date and end date of the monitor and is not based on a calendar year. For
example, if a monitor is started on July 1, 2020 and monitored successfully at the
required sampling frequency through the remainder of the year (sample period end date
was after December 31, 2020) then the completeness would be calculated as 100%.
From a NAAQS standpoint this monitor would be incomplete, but for the AMP600 the
monitor would be determined to be 100% complete (based on the sample period start
date).

For ozone data completeness determinations, the ozone season is used. For non-
NCore monitors that report data outside the ozone season, this data will not be used in
completeness calculations. NCore ozone monitors are required to operate all year, so
the AMP600 completeness evaluation for these monitors is based on the entire year.

For Continuous PM Monitors

There may be a difference between the estimate of routine data completeness between
the AMP430 Data Completeness Report and the AMP600 report for continuous
monitors. The AMP430 report evaluates completeness by hourly values while the
AMP600 evaluates completeness by the number of valid days compared to the number
of scheduled days for the monitor. Therefore, while a valid day for a continuous monitor
is 18 hours or greater, the AMP430 report estimates completeness based on the
number of valid hours sampled in that day divided by 24. For example, a day where only
18 valid hours were sampled the AMP430 completeness would be reported as 75%
(18/24). The AMP600 report would consider this day to be valid but would report data
completeness as 100%. Since the AMP600 report evaluates data completeness over a
complete year for a site (from sampler begin date to end date as entered into AQS), the
discrepancy between the two reports should be small.

QC data

Any valid QC check (for gaseous, PM, and Pb) must be reported to AQS. For 1-
pt QC checks, please refer to January 30, 2018 technical memo posted on
AMTIC1. The changes in the memo have not been completed in the AMP600. If
a 1-pt QC check is determined to be invalid but the routine data is still
considered valid, the 1-pt QC can be coded "1C" which means the 1-pt QC will
count towards QC completeness but will not affect the QC bias calculation. At
this time, there is not yet a field on PM QA/QC entries for null codes.

Comparing the AMP256 to AMP600

In previous certification periods there were a few discrepancies between the AMP256
report and the AMP600 report. The following fixes have been made to ensure that both
reports provide the same information:

1 Steps to Qualify Data after an Exceedance of Critical Criteria Checks


-------
3

Flow rate criteria - For semi-annual flow rates the AMP256 acceptance criteria
requires two audits that are within 5-7 months from each other. The "Criteria Met" field
in the AMP256 is based on the two audits being within this time period; however,
completeness will still show 100% on the AMP256 even if the criteria are not met. The
AMP600 uses the same criteria for the completeness estimate but will code the field as
yellow and report 70% if there are a least two audits in two quarters of the year but the
5-7month rule was not met, and red (recommended "N" flag) if only 1 or no audit was
performed during the year (<50% completeness).

Flow Rate Verifications - The March 2016 Revision to 40 CFR Part 58 Appendix A
included the reporting of flow rate verification data for all PM parameters (PM10, PM2.5
and Pb). Prior to the new rule, the flow rate verification data were only required for
PM10 continuous monitors. The certification reports for 2020 will not evaluate flow rate
verification data for the PM parameters other than PM10 continuous samplers. This
feature is planned to be applied to all PM parameters in the future.

How Does Data in the Summary Section of the AMP600 Reports Get Used at the
Site Level?

There has been some question on how the certification flags are generated for the
"PQAO Criteria Met" fields. Below provides some explanation and examples.

QAPP Approval -The QAPP Approval Field is based on QAPP approval dates supplied
from the monitoring organizations to the EPA Regions. Figure 1 provides an explanation
on how flags are set at the site pollutant level. The QAPP approval date (if one exists)
will be displayed above the site details but then transferred down to the site level. The
QAPP approval field is implemented in the same manner for all pollutants.

QAPP Approval

QAPP Approval - If a QAPP approval date is in AQS it will appear on the PQAO Pollutant Page

•	If no approval date or date > 5 years old, all sites will have AQS red "N" flag.

•	If date is < 5 years all sites will have AQS green "Y" flag, unless impacted by other parameters.

QAPP Approval Date
NPAP Audit Summary:

06/21/2017 _

Number of Valid Audits

Criteria Mot

Y

AOS
Site ID

a



Routine Data (ppm)







One Point Quality Check

Annual PE

NPAP



Certification

Monitor
Typ«

Mean

Min

Max

Exceed.
Count

Outlier
Count

Percent
Complete

!
1

a.

Bias Complete

Bias

Complete

Bias PQAO Level
Criteria

QAPP
Appr.

Aqs Rec Submit
Flag Req Flag

SPECIAL 3.6

0.0

21.8



0

77

3.35

~3.17

100



0

Y

Y

N



INVALID

22

¦ 0.6

18.4



0

97

1.94

~2.46

100

7.72

100

Y

Y

N



SLAMS

05

- 02

5.0



0

96

2.27

+M 95

100

4 42

100

Y

Y

Y



SLAMS

10

- 02

79,4



0

97

1.93

¦2 48

100

2 24

100

Y

Y

Y



NOTE: Any QAPPs whose approval date is greater than 5 years old will
have all sites flagged with Red "N" in the QAPP approval column and the
AQS recommended flag column. This was described in a July 11, 2017
technical memo posted on AMTIC2.

2 EPA Review of Monitoring Organizations QAPP's for Critical Criteria Conformance


-------
4

Gaseous Pollutants

1-point QC Check Completeness

The 1-point QC completeness data will be evaluated in the following manner:

1.	Count the number of checks in each 14-day interval starting with the Jan
1-14 interval. For each 14-day interval, multiple checks will only count as
one.

2.	Divide the total number of checks in #1 by 26

3.	Must be within the ranges identified in 40 CFR Part 58 Appendix A Section
3.1.1. If a 1-Point QC transaction is submitted with the assessment
concentration outside the valid ranges, it will be accepted with a warning,
but will not be used in regulatory precision and bias statistics or count
towards meeting the required frequency3.

For certification, a green Y is > 75% completeness. That means a monitoring
organization could miss six 14-day intervals (meaning no checks performed during a 14-
day interval) and still get a green "Y" flag. For a yellow flag, they could miss nine 14-day
intervals and get a yellow "Y" warning. Missing ten or more 14-day intervals will prompt
a red "N" flag to be displayed. In the event that ample valid checks were performed and
reported to AQS, but the dates of the checks do not align with the AQS programmed
spacing, the certifying agency can opt to include a comment in the comment field
explaining the discrepancy. The EPA Regional Office can then work with the certifying
agency to determine the appropriate EPA evaluation flag.

PM2.5 Pollutant PQAO Level Criteria

PM2.5 Collocation - 40 CFR Part 58 Appendix A requires that a PQAO collocate 15%
of the monitors for each method designation. The AMP256 has been revised to assess
whether there is 15% collocation for each method designation of only the primary
monitor and therefore matches the results in the AMP600 report. However, there may
be cases where more than one method designation was used at a site for the primary
monitor. Any method designation used as a primary monitor at any time during the year
will be counted towards the collocation evaluation. For example, if a method 118
sampler runs as the primary sampler for 6 months and a method 143 sampler runs as
the primary sampler for the other 6 months at the same site, the AMP600 will expect to
see collocation for each method designation within the PQAO.

Several interactions occur with collocation data. Figure 2 provides an example PM2.5
AMP600 report where these interactions are highlighted for discussion. First, each
method designation that was reported as a primary monitor for a site will be listed in the
collocation summary. Data from this summary should be the same information one
would see on the AMP256 report, at least for the collocations that occurred. As
mentioned earlier, the AMP256 now only counts those monitors that are considered the
primary monitor at each site, so both the AMP600 and the AMP256 results should be

3 This information was Question #10 during the AQS "Ask the Experts" webinar hosted
in November 2018.


-------
5

similar. However, there will be cases where more than one method designation is
reported for a site and both method designations will be identified for collocation (see
Fig. 2 116/117). "PQAO Criteria Met" for collocation is based on the completeness
summary statistic and the precision estimate (CV-UB). In the Figure 2 example, the
method 116 sampler shows 100% completeness and a PQAO precision estimate of
15.93 which is in the warning range. Therefore, all sites using 116 as the primary
method code are color coded yellow. Sites that had a primary method designation of
117 did not have collocation data available (completeness is 0%), so these sites do not
meet criteria and are flagged as "N". Also note that any individual collocated site/monitor
where the CV is greater than 25% will be flagged with an AQS recommended "N" flag
even if the PQAO level CV estimate is less than 25% (see method 170 examples in
Figure 2).

PQAO Level "Criteria Met" Flags

For Collocation and PEP , AQS Recommended flags are generated at the PQAO level and then "transfered" back
to each site

Collocation

based on CV, and completeness
and is also associated with
method designation

Bias

based on bias
estimate not
completeness.

PM2.5 - Local Conditions (88101)

3arameter:

PQAO Name:

Quality Assurance Project Plan Approval Date:
Collocation Summary

03/31/2011

PEP Summary

# Sites

# Sites

%

CV



Criteria

# # Audited

#PEP # PEP %



Criteria

Method # Sites Req

Collocated

Collocated

Est

CV UB

Met?

Methods Methods

Required Submitted Complete

Bias

Met?

116 2 1

1

100

13.87

15.93

Y

3 3

8 4 50

+5.02

Y

117 2 1

0

0 A





N









170 6 1

2

100 N

21.44

24.69

Y\









Monitors Summaries

Exceed.

AQS Site ID POCMethod Type Mean Min Max Count Countypmplete

1

116

SLAMS

9.91

1.8

27.2

1

116

SLAMS

6.79

2.0

14.4

3

170

SLAMS

4.28

-3.0

28.3

1

116

SLAMS

8.59

1.9

27.1

2

116

SLAMS

9.51

1.9

25.9

1

116

SLAMS

7.73

2.4

24.3

3

170

SLAMS

7.75

-3.0

47.3

3

170

SLAMS

6.37

-3.0

40.1

1

116/
117

SLAMS

6.68

1.3

19.3

3

170

SLAMS

8.70

-3.0

79.3

3

170

SLAMS

6.26

-3.2

53.2

1

117

SLAMS

8.19

1.8

24.0

Fig.2

PM2.5 Bias- Bias data is derived from the PEP and is aggregated at the PQAO level.
However, for CY2020 data the AMP600 will report the information but will not flag
this data in the report.


-------
6

PM10 Pollutant PQAO Level Criteria

PM10 Collocation - PM10 collocation is only required for manual (intermittent)
samplers. The AMP256 and the AMP600 only count sites where a manual sampler is
the primary sampler at a site. However, there may be times when a site had a manual
sampler as the primary for a period of time and then switched to a continuous monitor.
Sites where the manual sampler operated as the primary for any time during the year
will be included in the manual count.

In addition, CFR does not distinguish method designations for PM10, so all primary
intermittent samplers are aggregated at the PQAO level and 15% of the sites where
intermittent monitors are listed as primary monitors are required to be collocated.
Therefore, "Method" code information is not identified in the summary line of Figure 3. In
the example below, like PM2.5, both collocation completeness at the summary level and
the CV_UB are used for the Collocation "PQAO Criteria Met" column at the site/monitor
level. Data from this summary should be the same information in the AMP256 report. In
Figure 3 both the collocation and CV_UB were acceptable.

PM10 Total 0-1 Oum STP (81102) INTERMITTENT

Parameter:

PQAO Name:

Quality Assurance Project Plan Approval Date:

04/01/2007

Collocation Summary

# Sites # Sites	%	CV	Criteria

# sites Reg Collocated	Collocated	Est	CV UB Met?

13 2 2	100	5.55	6.11 Y

Monitors Summaries

Monitor
Type

Mean

Min

Max

SLAMS

20.47

7.0

46.0

SLAMS

20.18

7.0

44.0

SLAMS

15.70

6.0

32.0

SLAMS

13.07

4.0

23.0

SLAMS

16.04

6.0

36.0

SLAMS

17.37

2.0

36.0

SLAMS

19.58

2.0

33.0

SLAMS

15.24

6.0

30.0

SLAMS

15.58

2.0

28.0

SLAMS

16.20

2.0

41.0

SLAMS

15.48

2.0

68.0

SLAMS

15.28

2.0

36.0

SLAMS

16.18

2.0

31.0

Routine Data (ug/m3)

Exceed.Outlier

QAPP AOS Rec Req EPA
Appr. Value ValueValue

97
90

92
95

93
93

98
95
87
82
98
93
90

+0.63
-1.11
+0.09
+0.21
+0.55
+1.51
+0.34
-1.84
-0.59
+1.53
+1.23
+1.93
+1.15

100
100
100
100
100
100
100
100
100
100
100
100
100

5.15 100

Fig. 3

Lead Parameters

There are currently two Pb parameters; Pb-TSP and Pb-PM10. They will be discussed
separately.

Pb-TSP - Pb-TSP (Fig. 4) is a more established program. Like the other PM
parameters, both the collocation completeness and the precision estimate (CV-UB) will
be used in the "Collocation PQAO Criteria Met" column. The analysis audits are the
audits described in 40 CFR Part 58 App A section 3.3.4.2. Both the completeness and


-------
7

the bias estimate will be used in the "Lead Analysis Criteria Met" column at the site
monitor level, EPA has improved it's reporting of Pb-PEP data but will not use this
information in the certification evaluations for this year.

Lead (TSP) LC (14129)

Name: "	1

Quality Assurance Project Plan Approval Date:	06/01/2012

Collocation Summary: Number of Number of

Number CoHoc Sites Actual	Percent

of Sites Required Colloc Sites Collocated

PEP Data Not Used this Year

PEP Summary	Number of Number of Number of

Criteria	Number of Methods PEP Audits Audits Percent

Met	Methods Audited Required Submitted Complet

Criteria
Met

Analysis Audit Summary:

Number
Required

Number Percent
Submitted Complete Bias

24	22

Monitors Recommended for Certification

Monitor
AQS Site-ID poc Type



1 SLAMS



1 SLAMS



2 SLAMS



1 SLAMS



1 SLAMS



1 SLAMS



1 SLAMS



1 SLAMS



1 SLAMS



2 SLAMS



1 SLAMS



1 CI AUC

Routine Data (ug/mA3)

Exceed. Outlier
Max Count Count

0.024

0.001

0.202

0

0

100

0 198

0.004

2.135

3

0

100

0.207

o.ooo

2.100

3

0

95

0.216

0.005

2.220

3

0

100

0.075

0.004

0.577

0

0

98

0.175

0.004

1.884

1

0

100

0041

0.000

0.518

0

0

97

0.042

0.003

0.476

0

0

95

0480

0.005

7.008

4

0

100

0.512

0.005

3.594

4

0

100

0.027

0.001

0.284

0

0

98

n n-w

ftftft-S

n to*

ft

ft

oa

PEP

Lead



Certification

PQAO

Analysis

QAPP

AQS Req EPA

Crit. Met

Crit. Mel

Appr.

Rec. V Value Value

Fig. 4

Pb-PM10 - Since there are different implementation requirements for sampling Pb-
PM10 at source and non-source-oriented sites, collocation and PEP are not required at
every PQAO implementing this parameter4. Due to complications with programming
these requirements, collocation and PEP evaluations will not be used for certification on
a site/monitor level for CY2020 data. However, if values (as seen in Fig. 5) are
available, they will be reported. Lead analysis audit data will be used for certification.

Certification Report for Lead

Certification Year 2012	,	. r.	.	. ,

Certifying Agency Name:	¦ Collocation and PEP not used in Certification This Year

Number Number Percent	Criteria

Required Submitted Complete Bias	Met

24	18	75 -1.81 Y

Fig 5

4 A March 2016 revisions to 40 CFR Part 58 Appendix A discontinued Pb-PEP at non-
source NCore sites


-------
8

Attachment 1

Criteria That Will Generate Green (Acceptable) Warning (Yellow) and "N" Flags

(Red)

Notes:

1.	Blue shaded rows are evaluations that will be reported (when data is
available) but not used in certification flag settings

2.	Green shaded rows are rules promulgated in March 2016 but will not be
evaluated in 2020

3.	One Red for any monitor will elicit an AQS recommended "N" flag

4.	Three warnings (yellow) for any monitor will elicit an AQS recommended
"N" flag

NOTE: For the 2020 data certification process (due May 1, 2021), any sites for
PQAOs whose QAPP approval date is greater than 5 years old will be given a
Red "N" flag.


-------
9

Assessment

Current CFR
Requirement
or Guidance

Green
(Acceptable)

Yellow
(Warning)

Red

(Recommend N
Flag)

Comments

Technical
Systems Audit

PQAO every 3
years

TSA within 3
years

TSA within 4
years

TSA > 5 years

Not a monitoring Org
responsibility. Will be
reported on summary
page not by pollutant

Gaseous Criteria Pollutants

Routine Data
Completeness

75%

> 80%

80-70%

< 70%

Based on CFR criteria
for data use 100*
Number of hourly
obs/number of hours
in monitor sample
period1

QAPP Approval

Approval date
within 5 years
of current date

Approval date
within 5 years
of current
date

N/A

Not approved

and/or
approval date
greater than 5
years

Could be sole reason
for "N" flag if QAPP not
approved.

1-Point QC
Completeness

75%

> 75%

65-75%

< 65%

Based on 26,1-point

QC for a year.
Calculated based on
the number of days
the monitor operated.

1-Point QC
Precision

<±7.1% 03,

<	±10.1% CO,

S02

<	±15.1% N02

<±7.1% 03,

<	±10.1% CO,

S02

<	±15.1% N02

±8-20% 03
±11-25%
CO, S02
±16-25%
N02

> ±20% 03
> ±25% others

Based on all valid 1-
point QC checks in AQS

for the year. Value
should reflect AMP256
value

1-Point QC Bias

<±7.1% 03,

<	±10.1% CO,

S02

<	±15.1% N02

<±7.1% 03,

<	±10.1% CO,

S02

<	±15.1% N02

±8-20% 03
±11-25%
CO, S02
±16-25%
N02

> ±20% 03
> ±25% others

Based on all valid 1-
point QC checks in
AQS.

Value should reflect
AMP256 value

AnnualPE
Completeness

1 PE/year
3 audit levels

1 PE/year
3 audit levels

1 PE/year
2 audit
levels

No PE or
1 audit level

Will not count more
than one actual value
in an audit level. For
example, two audit in
one level count as 1
audit level.

Annual PE Bias

03, S02, N02

< ±1.5 ppb /
< ±15.1%

< ±1.5 ppb /
< ±15%

< ±1.6-3.0
ppb/
±16-25%

> ±3.0 ppb/
±25%

Average PD of all PE
values for the monitor

CO

< ±0.031 ppm/
< ±15.1%

< ±0.031 ppm/
< ±15.1%

< ±0.04-
0.06 ppm/
±16-25%

> ±0.06 ppm/
±25%



NPAP Audit
Completeness -
PQAO

20% of sites in
PQAO

20% of sites in
PQAO

10-19% of
sites in
PQAO

<10% of sites in
PQAO

Not a monitoring Org
responsibility. Will be
marked as "Y"


-------
10

Assessment

Current CFR

Green

Yellow

Red

Comments



Requirement

(Acceptable)

(Warning)

(Recommend N





or Guidance





Flag)



NPAP Bias

<±10.1% 03

<±10.1% 03

±10.1-20%

> ±20% 03

median PD for all



< ±15.1%

< ±15.1%

03

> ±25% others

values at a site and



others

others

±15.1-25%
others



median PD for PQAO
level estimate

NPAP Audit

4 levels

4 levels

2-3 levels

< 1 level

Not a monitoring Org

Completeness -









responsibility

Site











PM2.5 Criteria

Routine Data

75%

> 80%

80-70%

< 70%

Based on CFR criteria

Completeness









for data use
100 * number of
creditable
samples/number of
scheduled samples in
monitor sample
period1

QAPP Approval

Approval date

Approval date

N/A

Not approved

Could be sole reason



within 5 years

within 5 years



and/or

for "N" flag if QAPP not



of current date

of current
date



approval date
greater than 5
years

approved.

Flow Rate

every 30 days

Every 30 days

Every 45

> 45 days

Not implemented in

Verification

(12/year)

(11-12/year)

Days

(< 8/year)

2020

Completeness





(8-11/year)





Flow Rate

< ±4.1% of

< ±4.1% of

±4.1-6% of

> ±6% of

design =design flow

Verification Bias

transfer

transfer

transfer

transfer

rate



standard

standard

standard

standard

Average PD for audits



< ±5.1% from

< ±5.1% from

±5.1-7%

> ±7% from

at monitor level



design

design

from design

design

Value should reflect

AMP256 value
Not implemented in
2020

Flow Rate Audit

2 /year every 6

2/year every

2 across 2

1 audit

Semi-annual flow rate

Completeness

months

5-7 months or

3 or 4 with
one audit in 3
or 4 quarters

quarters



audits.

Based on how long
sampler operated. If
sampler operates <9
months at least 1 is
expected. If operated
>9 months two audits
expected.

Flow Rate Audit

< ±4% of

< ±4% of

±5-6% of

> ±6% of

design =design flow

Bias

transfer

transfer

transfer

transfer

rate



standard

standard

standard

standard

Average PD for audits



< ±5% from

< ±5% from

±6-7% from

> ±7% from

at monitor level



design

design

design

design

Value should reflect
AMP256 value


-------
11

Assessment

Current CFR

Green

Yellow

Red

Comments



Requirement

(Acceptable)

(Warning)

(Recommend N





or Guidance





Flag)



Collocation

75%

> 75%

65-74%

< 65%

By method designation

Completeness









Summary level=
average of
completeness of site

level values
Site level = number of
reported observations
/30 Based on how long
sampler operated

Collocation

< 10.1%

< 10.1%

10.1-25%

> 25%

By method designation

Precision









Same statistics as
AMP256 for summary
level and site level.
Value should reflect
AMP256 value

PM2.5 PEP

5 or 8

5 or 8

3-4 or 6-7

< 3 or 6

Not a monitoring Org

Completeness









responsibility

PEP Bias

< ±10.1%

< ±10.1%

±10.1-30%

> ±30%

Value should reflect
AMP256 value

PM10 Continuous Methods

Routine Data

75%

> 80%

80-70%

< 70%

Based on CFR criteria

Completeness









for data use
100 * number of
valued strata (days per
collection frequency) /
total number of strata

QAPP Approval

Approval date

Approval date

N/A

Not approved

Could be sole reason



within 5 years

within 5 years



and/or

for "N" flag if QAPP not



of current date

of current
date



approval date
greater than 5
years

approved.

Flow Rate

75%

> 75%

65-74%

< 65%

12 per year, based on

Verification









how long sampler

Completeness









operated
Not implemented in
2020

Flow Rate

< ±7.1% of

< ±7.1% of

±7.1- 9% of

> ±9% of

Average of percent

Verification Bias

transfer

transfer

transfer

transfer

differences.



standard

standard

standard

standard

Value should reflect

AMP256 value
Not implemented in
2020


-------
12

Assessment

Current CFR

Green

Yellow

Red

Comments



Requirement

(Acceptable)

(Warning)

(Recommend N





or Guidance





Flag)



Flow Rate Audit

2 /year every 6

2/year every

2 across 2

1 audit

Semi-annual flow rate

Completeness

months

5-7 months or

3 or 4 with
one audit in 3
or 4 quarters

quarters



audits
Based on how long
sampler operated. If
sampler operates <9
months at least 1 is
expected. If operated
>9 months two audits
expected.

Flow Rate Audit

<±10.1% of

<±10.1% of

±10.1-12%

> ±12 % of

Semi-annual flow rate

Bias

audit standard

audit standard

of audit
standard

audit standard

audits.

Value should reflect

AMP256 value
Average of percent
differences

Collocation

75%

> 75%

65-74%

< 65%

Summary level=

Completeness









average of
completeness of site

level values
Site level = number of
reported observations
/30 Based on how long
sampler operated

Collocation

10%

> 10%

11-20%

> 20%

Same statistics as

Precision









AMP256 for summary
and site level. Value
should reflect AMP256
value

PM10 Manual Methods

Routine Data

75%

> 80%

80-70%

< 70%

Based on CFR criteria

Completeness









for data use
100 * number of
valued strata (days per
collection frequency) /
total number of strata

QAPP Approval

Approval date

Approval date

N/A

Not approved

Could be sole reason



within 5 years

within 5 years



and/or

for "N" flag if QAPP not



of current date

of current
date



approval date
greater than 5
years

approved.

Flow Rate

every 30 days

Every 30 days

Every 45

>45 days

Not implemented in

Verification

(12/year)

(11-12/year)

Days

(<8/year)

2020

Completeness





(8-11/year)





Flow Rate

< ±7.1% of

< ±7.1% of

±7.1-9% of

> ±9% of

Semi-annual flow rate

Verification Bias

transfer

transfer

transfer

transfer

audits.



standard

standard

standard

standard

Value should reflect

AMP256 value
Not implemented in
2020


-------
13

Assessment

Current CFR

Green

Yellow

Red

Comments



Requirement

(Acceptable)

(Warning)

(Recommend N





or Guidance





Flag)



Flow Rate Audit

2 /year every 6

2/year every

2 across 2

1 audit

Semi-annual flow rate

Completeness

months

5-7 months or

3 or 4 with
one audit in 3
or 4 quarters

quarters



audits
Based on how long
sampler operated. If
sampler operates <9
months at least 1 is
expected. If operated
>9 months two audits
expected.

Flow Rate Audit

<±10.1% of

<±10.1% of

±10.1-12%

> ±12 % of

Semi-annual flow rate

Bias

transfer

transfer

of transfer

transfer

audits.



standard

standard

standard

standard

Value should reflect
AMP256 value

Collocation

75%

> 75%

65-74%

< 65%

Summary level=

Completeness









average of
completeness of site

level values
Site level = number of
reported observations
/30 Based on how long
sampler operated

Collocation

10%

> 10%

11-20%

> 20%

Same statistics as

Precision









AMP256 for summary
and site level. Value
should reflect AMP256
value

Pb-TSP

Routine Data

75%

> 80%

80-70%

< 70%

Based on CFR criteria

Completeness









for data use
100 * number of
creditable
samples/numbers of
scheduled samples in
monitor sample
period1

QAPP Approval

Approval date

Approval date

N/A

Not approved

Could be sole reason



within 5 years

within 5 years



and/or

for "N" flag if QAPP not



of current date

of current
date



approval date
greater than 5
years

approved.

Flow Rate

every 90 days

every 90 days

every 120

> every 120

Not implemented in

Verification

and 4 times a

and 4 times a

days and 3

days and < 3

2020

Completeness

calendar year

calendar year

times a
calendar
year

times a
calendar year



Flow Rate

< ±7.1% from

< ±7.1% from

±8-9% of

> ±9% of

Not implemented in

Verification Bias

transfer

transfer

transfer

transfer

2020



standard

standard

standard

standard




-------
14

Assessment

Current CFR

Green

Yellow

Red

Comments



Requirement

(Acceptable)

(Warning)

(Recommend N





or Guidance





Flag)



Flow Rate Audit

2 /year every 6

2/year every

2 across 2

1 audit

Semi-annual flow rate

Completeness

months

5-7 months or

3 or 4 with
one audit in 3
or 4 quarters

quarters



audits
Based on how long
sampler operated. If
sampler operates <9
months at least 1 is
expected. If operated
>9 months two audits
expected.

Flow Rate Audit

< ±7.1% of

< ±7.1% of

±8-9% of

> ±9% of

Semi-annual flow rate

Bias

transfer

transfer

transfer

transfer

audits.



standard

standard

standard

standard

Value should reflect
AMP256 value

Collocation

75%

> 75%

65-74%

< 65%

Summary level=

Completeness









average of
completeness of site

level values
Site level = number of
reported observations
/30 Based on how long
sampler operated

Collocation

< 20.1%

< 20.1%

21-30%

> 30%

Same statistics as

Precision









AMP256 for summary
and site level

Pb PEP

5 or 8

4 or 7

3 or 6

< 3 or 6

Not a monitoring Org

Completeness









responsibility

Pb PEP Bias

< ±15.1%

< ±15.1%

±15.1-25%

> ±25%

Average PD

Analysis Audit

75%

> 75%

65-74%

< 65%

Average completeness

Completeness









by quarter than take
average of all 4
quarters

Analysis Audit

< ±10.1%

< ±10.1%

±10.1-18%

> ±18%

Average PD

Bias











Pb-PMIO

Routine Data

75%

> 80%

80-70%

< 70%

Based on CFR criteria

Completeness









for data use
100 * number of
creditable
samples/numbers of
scheduled samples in
monitor sample
period1

QAPP Approval

Approval date

Approval date

N/A

Not approved

Could be sole reason



within 5 years

within 5 years



and/or

for "N" flag if QAPP not



of current date

of current
date



approval date
greater than 5
years

approved.


-------
15

Assessment

Current CFR
Requirement
or Guidance

Green
(Acceptable)

Yellow
(Warning)

Red

(Recommend N
Flag)

Comments

Flow Rate Audit
Completeness

2/year every 6
months

2/year every
5-7 months or

3 or 4 with
one audit in 3
or 4 quarters

2 across 2
quarters

1 audit

Semi-annual flow rate
audits
Based on how long
sampler operated. If
sampler operates <9
months at least 1 is
expected. If operated
>9 months two audits
expected.

Flow Rate Audit
Bias

< ±4% of
transfer
standard

< ±4% of
transfer
standard

±5-6% of
transfer
standard

> ±6% of
transfer
standard

Semi-annual flow rate.
Value should reflect
AMP256 value

Collocation
Completeness

75%

> 75%

65-74%

< 65%

Summary level=
average of
completeness of site

level values
Site level = number of
reported observations
/30 Based on how long
sampler operated

Collocation
Precision

20%

< 20%

21-30%

> 30%

Value should reflect
AMP256 value

Pb PEP

Completeness

5 or 8

5 or 8

3 or 6

< 3 or 6

Not a monitoring Org
responsibility

Pb PEP Bias

±15%

±15%

±16-25%

> ±25%



Analysis Audit
Completeness

75%

> 75%

65-74%

< 65%

Based on 24 audits per
year

Analysis Audit
Bias

10%

10%

< 18%

> 18%

Average of percent

differences.
Value should reflect
AMP256 value

1 Sample period is the time interval between the sample period start date and the
sample period end date.


-------