United States
Environmental Protection
Agency
Environmental Monitoring
Systems Laboratory
Research Triangle Park NC 27711
Research and Development
EPA-600/S4-84-032 June 1984
4MEPA Project Summary
Summary of Precision and
Accuracy Assessments for the
State and Local Air Monitoring
Networks—1981
E. Gardner Evans and Raymond C. Rhodes
This report covers the first year of
collecting Precision and Accuracy data
from state and local monitoring agen-
ies. The data are summarized by state
and pollutant, and then by region and
the nation.
A brief comparison of the precision
and accuracy from the Precision and
Accuracy Reporting System and the
independent performance audit
program conducted by the Environ-
mental Monitoring Systems Laboratory
is presented.
This Project Summary was developed
by EPA's Environmental Monitoring
Systems Laboratory, Research Triangle
Park, NC, to announce key findings of
the research project that is fully docu-
mented in a separate report of the same
title (see Project Report ordering infor-
mation at back).
Introduction
The purpose of this document is to
report the first year of data from the
Precision and Accuracy Reporting
System (PARS). Federal regulations
promulgated on May 10, 1979, required
quality assurance precision and accuracy
(P&A)* data to be collected. Collection
started January 1, 1981, according to
guidelines set forth in 40 CFR Part 58
*When one speaks of precision and accuracy of meas-
urement data,1 one really means the precision and
accuracy of the measurement process from which
the measurement data are obtained. Precis/on is a
measure of the "repeatability of the measurement
process under specified conditions." Accuracy is a
measure of "closeness to the truth."
(44FR27558-27604). These guidelines
provide for more uniform Quality
Assurance programs and reporting
requirements across all state and local air
monitoring stations.'
The bulk of the material presented in
this report results from the considerable
efforts of the state and local agencies to
produce and report the data quality
assessments for P&A.
Comparisons have been made of the
accuracy data collected for PARS and the
results of the Performance Audit (PA)
Program, which has been an ongoing pro-
gram conducted by the Environmental
Monitoring Systems Laboratory (EMSL)
since the early 1970's.
Background
Ambient air quality data, collected by
states and local agencies, have been
stored in the National Aerometric Data
Bank (NADB) since 1957. These data are
used in planning the nation's air pollution
control strategy, in determining if the
National Air Quality Standards are being
achieved, and in determining long-term
trends of air quality. Prior to the air
monitoring regulations of May 10, 1979,
the procedures used in selecting
monitoring sites, in operating and
controlling the equipment, and in calcu-
lating, validating and reporting the data
varied considerably among agencies.
Frequently the procedures being used
were not well-documented. This made it
difficult to compare data from different
sites and agencies.
-------
To help alleviate this problem, the May
10, 1979, regulation mentioned in the
Introduction imposed uniformity on data
collected after December 31, 1980, and
intended to be submitted to the NADB.
For example, only EPA reference,
equivalent, or other EPA-approved air
monitoring methods were to be used.
Also, calibration standards were to be
traceable to National Bureau of
Standards (NBS) or other authoritative
standards. Further, the quality assurance
systems of the states were required to be
documented and approved by the EPA
Regional Offices. Finally, the reporting
organizations must also follow specific
procedures when assessing the P&A of
their measurement systems and were
required to report the P&A data quarterly
to EPA.
The precision assessments were
generally determined either by
performing repeated measurements on
approximately ambient-level "calibration"
gases at two-week intervals, or—by
obtaining duplicate results from
collocated samplers. The accuracy
assessments were generally determined
by analyzing blind audit materials
traceable to NBS. During each calendar
year, each site or instrument must be
audited at least once. Details concerning
the specific procedures and computa-
tions used to assess P&A are contained in
the regulations.
When a request is madeforambientair
quality monitoring data from the NADB,
the requestor receives the P&A data
along with the routine monitoring data.
The requestor, or user, of the data can
feel more confident that the data are of
the quality indicated by the assessments
and that the data have been obtained
from an agency having a planned and
documented quality assurance system.
The EPA can also rely on the data in
producing its control strategies and
determining whether standards have
been met.
Results
The first year of data collected for P&A
has been compiled and summarized. All
141 reporting organizations reported
P&A data for particulates, using the
method that is used throughout the
nation. Other methods were not reported
from all organizations for several
reasons. First, reporting organizations in
some areas no longer use the manual
methods for SO2 and NO2, having
installed only continuous analyzers.
Since the cut-off date for approval as a
State and Local Air Monitoring (SLAM)
site was January 1, 1983, some sites had
not yet been approved for some other
methods. Of course, some problems did
arise simply because this was the first
year of data collection. Typical start-up
problems were: following instructions
correctly, using the proper form, coding
the proper numbers into the correct
fields, and mailing the results to the right
place. Through the efforts of EMSL's
Quality Assurance Division (QAD), the
regional quality assurance offices, and
the state and local agencies, the P&A
data have been reviewed and verified for
completeness and accuracy.
All data summaries presented in this
report were based on all of the data
received from the state and local
agencies; no outliers were removed from
the data set. An evaluation of removing
outliers from the PARS data set did not
indicate significant changes in the data
summary average values.
The P&A numbers for particulate data
from the high volume sampler are
impressive. From 13,248 valid collocated
data pairs, the national average of lower
and upper probability limits for precision
are -12% and +13%. The accuracy
probabilities calculatedfrom 5,560audits
are even more impressive with lower and
upper and lower probabilities of -06% and
+07%.
Results from the PARS Program
Table 1 exhibits the national averages
for each of the manual pollutant methods
in the PARS system.
The precision limits reflect the
repeatability of the methodology used in
the field to collect and analyze the
samples at ambient levels. The spread of
the limits may be somewhat inflated
because of measurements at relatively
low concentration levels.
The accuracy of the manual methods
indicates the limits at predetermined
concentration levels for the chemical
analysis performed in the samples for
lead, sulfur dioxide, and nitrogen dioxide.
For the particulate, the accuracy
measurement is for the flow rate only.
The probability limits for manual
accuracy are extremely good and reflect
the quality of work done in the chemical
laboratories for lead,sulfur dioxide, and
nitrogen dioxide analyses, and in the field
for flow rate measurement for particu-
lates.
The P&A limits for automated methods
are in general good for the first year of
data collection. Table 2 indicates the
amount of effort involved, with over
10,000 precision checks each being
performed for SO2 and O3 instruments.
Although the results are good for the
first .year, further improvement is
believed possible. Details of the
individual states' results are presented in
the body of the detailed report. The
regions for which further improvement
should be possible are those showing the
larger of variability in the results.
Comparison of Results from the
PARS and the PA Program
A general comparison between the
accuracy data of the PARS program and
the PA data is included in this report. The
audit data are the results of an
independent check conducted by the
QAD of the EMSL. Blind samplesaresent
to the laboratories that perform the state
and local agencies' analyses. The samples
are analyzed and results are sent to QAD
where they are evaluated.
Since precision assessments are not
made in the PA program, only accuracy
can be compared across the PARS and
the PA programs. For the purpose of this
report, the results from PARS and the PA
system are compared at approximately
the same levels. Also, the results are
compared on a state basis, which means
that for states with more than one labora-
tory involved, the results are averaged.
Since the PARS data are presented with
outliers, the same approach was taken
with the audit data. Knowledge of the
historical audit data reports, however,
indicates that the presence of outliers
does make a significant difference in the
average audit results forthe performance
audits.
Future reports which compare PA and
PARS are planned. Those reports will
address the comparison in more detail,
making a laboratory-to-reporting
organization comparison with and
without the inclusion of outliers.
Comparisons of the national averages
of the probability limits exhibit
correlations between the results of the
two programs. However, in general
(except for the manual N02), the spread of
the limits is wider at level 1 than at levels
2, 3, and 4. Lack of better agreement can
be attributed to a number of factors. First,
the inclusion of outlier values in the PA
data appears to have introduced some
excessive distortion of general trends.
Second, for both data sets, variations due
to many sources of error are averagad
together to obtain the national averages,
thereby masking any correlations which
may have existed for the results of
individual agencies. Third, the concentra-
tion levels for the two systems do not
-------
Table 1. National Precision and Accuracy Probability Limit Averages for Manual Methods
Precision
Accuracy
Pollutant
TSP
Lead
Sulfur Dioxide
Nitrogen Dioxide
Number of Valid
Collocated
Data Pairs
13.248
416
965
752
Probability
Limits (%)
Lower Upper
-12
-25
-51
-41
+ 13
+30
+57
+44
Probability Limits (%)
No. of
Audits
5.560
577
711
769
Level 1
Lower Upper
...
-11
-16
-07
...
+08
+09
+ 10
Level 2
Lower Upper
-06
-09
-09
-04
+07
+02
+07
+06
Level 3
Lower Upper
...
...
-08
-04
...
...
+06
+05
Table 2. National Precision and Accuracy Probability Limit Averages for Automated Analyzers
Precision
Accuracy
SO2
03
CO
/VO2
No. of
Precision
Checks
10.821
10,399
7,996
2.487
Probability
Limits (%)
Lower
-15
-12
-10
-14
Upper
+09
+ 10
+08
+ 12
No. of
Audits
1,062
1.336
821
329
Level 1
Lower
-17
-14
-13
-21
Upper
+ 13
+ 12
+ 13
+26
Probability Limits f%)
Level 2
Lower
-13
-10
-07
-12
Upper
+ 11
+09
+08
+ 11
Level 3
Lower
-13
-10
-07
-10
Upper
+ 11
+09
+06
+08
coincide exactly at each of the audit
levels. Fourth, the PA data are the results
of independent external audits while the
PARS accuracy data are based on the
results of independent internal audits.
The expected effects of the first and last-
mentioned factors would be to cause the
spread of the limits for the performance
audits to be wider than that for the PARS.
Examination of the results, shown in
Table 3, confirm this expectation.
Table 3. Summary Report of QAD Performance Audits (PA) vs PARS Accuracy Audit Data
for Year 1981
National Averages
Probability Limits (%)
Pollutant and
Method Code
C42101 CO
PA
PARS
142602 N02
PA
PARS
142401 SO2
PA
PARS
112128 Lead
PA
PARS
1111101 HIV
PA
PARS
Audits
1118
(666)
234
(7161
157
(640)
84
(535)
2114
(3073)
Level 1
Lower Upper
-14
(-18)
-29
(-9)
-44
(-22)
-25
(-12)
+ 10
(+19)
+23
(+12)
+28
(+15)
+ 19
(+8)
Level 2
Lower Upper
-8
(-8)
-12
(-6)
-30
(-17)
-15
(-7)
-18
(-10)
+8
(+9)
+8
(+7)
+20
(+13)
+ 11
(+5)
+ 19
(+10)
Level3 Level 4
Lower Upper Lower Upper
-7 +7
(-8) (+8) (-30) (+15)
-13 +8 -11 +7
(-5) (+6)
-16 +10 -20 +12
(-14) (+12)
-15 +10
-------
The EPA authors E. Gardner Evans and Raymond C. Rhodes are with the
Environmental Monitoring Systems Laboratory, Research Triangle Park, NC
27711.
The complete report, entitled "Summary of Precision and Accuracy Assessments
for the State and Local Air Monitoring Networks—1981. "(Order No. PB 84-189
968; Cost: $ 11.50, subject to change) will be available only from:
National Technical Information Service
5285 Port Royal Road
Springfield, VA 22161
Telephone: 703-487-4650
The EPA authors can be contacted at:
Environmental Monitoring Systems Laboratory
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711
•it US GOVERNMENT PRINTING OFFICE; 1984 — 759-01 5/7726
United States Center for Environmental Research
Environmental Protection Information
Agency Cincinnati OH 45268
Official Business
Penalty for Private Use $300
------- |