United States
Environmental Protection
Agency
Environmental Monitoring
Systems Laboratory
Research Triangle Park NC 27711
EPA/600/4-88-007
January 1988
Research and Development
vvEPA
Precision and Accuracy
Assessments for
State and Local Air
Monitoring Networks,
1986
-------
EPA/600/4-88/007
January 1988
PRECISION AND ACCURACY ASSESSMENTS
FOR STATE AND LOCAL AIR MONITORING NETWORKS
1986
Raymond C. Rhodes
Quality Assurance Division
Environmental Monitoring Systems Laboratory
E. Gardner Evans
Monitoring and Assessment Division
Environmental Monitoring Systems Laboratory
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF RESEARCH AND DEVELOPMENT
ENVIRONMENTAL MONITORING SYSTEMS LABORATORY
RESEARCH TRIANGLE PARK, NORTH CAROLINA 27711
_ ,-tection
>PI
36^
60604
-------
NOTICE
The information in this document has been subjected to the Agency's
peer and administrative review and it has been approved for publication.
Mention of trade names or commercial products does not constitute endorse-
ment or recommendation for use.
11
-------
FOREWORD
Measurement and monitoring research efforts are designed to anticipate
potential environmental problems, to support regulatory actions by develop-
ing an in-depth understanding of the nature and processes that impact health
and the ecology, to provide innovative means of monitoring compliance with
regulations, and to evaluate the effectiveness of health and environmental
protection efforts through the monitoring of long-term trends. The Environ-
mental Monitoring Systems Laboratory, Research Triangle Park, North Caro-
lina, has the responsibility for assessment of environmental monitoring
technology and systems; implementation of agency-wide quality assurance
programs for air pollution measurement systems; and supplying technical
support to other groups in the Agency including the Office of Air and Radi-
ation, the Office of Toxic Substances, and the Office of Enforcement.
Ambient air quality data collected by States and local agencies are
used in planning the nation's air pollution control strategy, in deter-
mining if National Ambient Air Quality Standards are being attained, and in
determining long-term trends of air quality. Prior to the regulations of
May 10, 1979, the procedures used in site selection, controlling equipment,
and calculating and validating data varied considerably among agencies.
These regulations serve to improve and make more uniform the quality assur-
ance programs of the state and local agencies and to require the assessment
and reporting of data quality estimates for precision and accuracy. Re-
porting of precision and accuracy data was first required for calendar year
1981. Previous reports summarized the results for 1981, 1982, 1983, 1984
and 1985. This report summarizes and evaluates the results for 1986.
John C. Puzak
Deputy Director
Environmental Monitoring Systems Laboratory
Research Triangle Park, North Carolina
-------
ABSTRACT
Precision and accuracy data obtained from State and local agencies
during 1986 are summarized and evaluated. Some comparisons are made with
the results reported for previous years to determine the indication of any
trends. Some trends indicate continued improvement in the completeness of
reporting of precision and accuracy data. The national summaries indicate
a further improvement in the precision and accuracy assessments of the
pollutant monitoring data collected. The annual results from each reporting
organization are given so that comparisons may be made from year-to-year
and with other reporting organizations.
A comparison of the precision and accuracy data from the Precision and
Accuracy Reporting System (PARS) with those from the independent National
Performance Audit Program (NPAP) conducted by the Environmental Monitoring
Systems Laboratory is made.
-------
CONTENTS
Page
Foreword ill
Abstract iv
Figures vi
Tables vii
Acknowledgment * ix
1. Introduction 1
2. National Results 4
National Data Reporting 4
National Activity in Performing Precision
Checks and Accuracy Audits 4
1986 Results from the PARS program 9
National Precision Results Comparison 11
National Accuracy Results Comparison 12
National Frequencies . . . . 13
3. Regional Results 16
Regional Data Reporting 16
Regional Comparisons 20
4. Results by Reporting Organizations 31
5. Further Evaluation of PARS Data . . 33
Manual Methods 34
Comparison of National Limit Values and 50
Percentile Values . 36
6. Comparison of Results from the PARS and the Performance
Audit Program 38
7. Conclusions and Recommendations 49
References 50
Appendix A - Glossary A-l
Appendix B - Formulas for Combining Probability Limits B-l
Appendix C - Listing of Reporting Organizations C-l
Appendix D - Precision and Accuracy Data by Reporting Organization . D-l
Appendix E - Problems Involved in the Comparison of Performance
Audit (PA) Data and Precision and Accuracy
(PARS) Data E-l
Appendix F - Comparisons of PARS and Performance Audit Data .... F-l
-------
FIGURES f >
Number Page
1. National Precision Probability Limits for 1983 through 1986 . . 12
2. National Accuracy Probability Limits for 1983 through 1986 . . 13
3. CO Precision and Accuracy by Region for 1983 through 1986 ... 21
4. Continuous S02 Precision and Accuracy by Region for 1983
through 1986 . 22
5. Continuous N0£ Precision and Accuracy by Region for 1983
through 1986 23
6. Ozone Precision and Accuracy by Region for 1983 through 1986 . 24
7. TSP Precision and Accuracy by Region for 1983 through 1986 . . 26
8. Lead Precision and Accuracy by Region for 1983 through 1986 . . 27
9a. Comparison of PA and PARS for CO (Level 2) 41
9b. Comparison of PA and PARS for TSP (Level 2) . 41
9c. Comparison of PA and PARS for Pb (Level 2) . 42
9d. Comparison of PA and PARS for Continuous S02 (Level 3) .... 43
10. Comparison of PA and PARS, National Values, 1986 45
E-l. Concentration Levels for Comparing PARS and PA
Data, Continuous Methods E-6
E-2. Concentration or Flow Levels for Comparing PARS
and PA Data, Manual Methods E-7
VI
-------
TABLES
Number Page
1. Requirements for Performing Precision Checks for
SLAMS Network 3
2. Concentration Levels for Conducting Accuracy
Audits of SLAMS Network 3
3. Percent of Reporting Organizations Reporting Precision
and Accuracy Data 4
4. Year-to-Year Activity of Precision and Accuracy
Assessments for the Manual Methods 5
5. Year-to-Year Activity of Precision and Accuracy
Assessments for the Continuous Methods 6
6. National Precision and Accuracy Probability Limit Values
for Manual Methods 10
7. National Precision and Accuracy Probability Limit Values
for Automated Analyzers 11
8. Percentiles of Quarterly Probability Limits for All
Reporting Organizations (1986) .... 14
9. Total Number of Reporting Organizations Required to Report
for the Year 1986, by Pollutant 16
10. Percentage of SLAMS Sites with Complete Data in PARS
for the Year 1986 . 17
11. Number of Reporting Organizations Having Data in the PARS
Master File for the Year 1986 31
12. Comparison of the 50-Percentile Frequency Distribution
Values with the National Limit Values for 1986 33
13. Values of Quarterly Probability Limits Considered as
Excessive Based on 1986 Data 37
14. Summary Comparison of EMSL Performance Audits (PA) vs.
PARS Accuracy Audit Data for Year 1986 39
vn
-------
TABLES (continued)
Number Pa9e
E-l. Concentration Levels for PARS and PA Audits for
1986 for the Continuous Methods E-2
E-2. Concentration (or Flow) Levels for PARS and PA
Audits for 1986 for the Manual Methods E-3
E-3. Concentration Ranges for Comparison of PARS and
PA Data - Continuous Methods E-8
E-4. Ranges for Comparison of PARS and PA Data -
Manual Methods . . ...... E-9
vi i i
-------
ACKNOWLEDGMENTS
The authors express appreciation to the following persons and organi-
zations who assisted in the preparation of this report: the States and
local agencies, for performing and reporting the results of the precision
checks and accuracy audits; the Regional Office persons responsible for
reviewing and coordinating the reporting of the precision and accuracy data
to EMSL/RTP; Robert L. Lampe, EMSL/RTP, for reviewing and processing the
precision and accuracy reports received from the Regional organizations;
Robert Lyon, Computer Sciences Corporation, for the computer programing,
processing, and summarization of the precision and accuracy data; John Hol-
land, Northrop Services, Incorporated, for assistance in preparing the figures
and tables; Edward Barrows, Northrop Services, Incorporated, for programming
and reporting of the comparisons of the results of the EMSL performance audit
program with the precision and accuracy data; Les Sizemore Northrop Services,
Incorporated, for the programming and preparation of the charts showing the
results from each reporting organization for the years 1981 through 1986; and
to Elizabeth Hunike and Ceci Ellington, EMSL, for typing this report.
-------
-------
SECTION 1
INTRODUCTION
The purpose of this document is to report the sixth year of data from
the Precision and Accuracy Reporting System (PARS). Federal regulations
promulgated on May 10, 1979, require quality assurance precision and accu-
racy (P and A)* data to be collected. Collection started January 1, 1981,
according to requirements set forth in 40 CFR Part 58 Appendix A.1 These
requirements provide for more uniform Quality Assurance programs and speci-
fic precision and accuracy assessment and reporting requirements across all
State and local air monitoring agencies.
The major portion of this report consists of summarizations and evalua-
tions of the P and A data obtained by the efforts of the States and local
agencies. In addition, comparisons have been made of the accuracy data
collected for PARS with the results of the National Performance Audit
Program (NPAP) which has been an ongoing program conducted by the Environ-
mental Monitoring Systems Laboratory (EMSL) since the early 1970's.
These summarizations and evaluations of precision and accuracy data
serve the following purposes:
1. Provides quantitative estimates of the precision and accuracy of
their ambient air monitoring data to State and local agencies.
2. Indicates the need to improve quality assurance systems in specific
reporting organizations if a comparison of the data from all the
agencies shows excessive variability or bias.
3. Indicates the need for improvement in monitoring methodology if
precision and accuracy estimates are excessive or erratic.
4. Provides users of data from the State and Local Air Monitoring
Stations (SLAMS) network a quantitative estimate of the precision
and accuracy of the ambient air quality data.
*When one speaks of precision and accuracy of measurement data,2 one really
means the precision and accuracy of the measurement process from which the
measurement data are obtained. Precision is a measure of the "repeatability
of the measurement process under specified conditions." Accuracy is a meas-
ure of "closeness to the truth."
-------
Ambient air quality data, collected by States and local agencies since
1957, have been stored in the National Aerometric Data Bank (NADB). These
data are used in (1) planning the nation's air pollution control strategy,
(2) determining if the National Ambient Air Quality Standards are being
achieved, and (3) determining long-term trends of air quality. Prior to the
EPA air monitoring regulations of May 10, 1979, the procedures used in se-
lecting monitoring sites, operating and controlling the equipment, and calcu-
lating, validating and report-ing the data varied considerably among agencies.
Frequently the procedures being used were not well-documented. These condi-
tions made it difficult to compare data from different sites and agencies.
Furthermore, little information was available on the reliability of the moni-
toring data.
To help alleviate these problems, EPA's air monitoring regulations
imposed uniform requirements on network design, siting, quality assurance,
monitoring methods, and data reporting after December 31, 1980. For example,
only EPA reference, equivalent, or other EPA-approved air monitoring methods
are to be used. Also, calibration standards are to be traceable to the
National Bureau of Standards (NBS) or other authoritative standards. Further,
the quality assurance systems of the States are required to be documented and
approved by the EPA Regional Offices. Finally, the reporting organizations
must also follow specific procedures when assessing the P&A of their measure-
ment systems and must report the P&A data to EPA quarterly. Starting January
1, 1981, these regulations became effective for National Air Monitoring Sites
(NAMS), and beginning January 1, 1983, for all State and Local Air Monitoring
Stations (SLAMS). These regulations have remained in effect for data obtained
through 1986. Revised regulations, March 19, 1986, require the reporting of
all the raw data to EMSL/RTP effective for all data obtained after December
31, 1986.
The precision assessments were determined by performing repeated meas-
urements of ambient-level "calibration" gases at two-week intervals for
continuous methods, or by obtaining duplicate results from collocated sam-
plers for manual methods. Table 1 summarizes the requirements for perform-
ing precision checks. The accuracy assessments were generally determined
by analyzing blind audit materials traceable to NBS. Table 2 shows the
concentration levels. During each calendar year, each site or instrument
must be audited at least once. Details concerning the specific procedures
and computations used to assess P&A are contained in the regulations.
-------
TABLE 1. REQUIREMENTS FOR PERFORMING PRECISION CHECKS
FOR SLAMS NETWORK
Precision check
Frequency"
Parameter
CO (continuous analyzer)
S02, N02, and 03
(continuous analyzer)
TSP, S02, and N02
(manual)
Pb
8-10 ppm
0.08 - 0.10 ppm
Collocated sampler
(Ambient concentration)
Duplicate strips
(Ambient concentration)
Once each 2 weeks
Once each 2 weeks
Once each 6 days
Once each 6 days
TABLE 2. CONCENTRATION LEVELS FOR CONDUCTING
ACCURACY AUDITS OF SLAMS NETWORK
Parameter
S02, N02, 03
(continuous)
CO
TSP (flow only)
S02 (manual)*
N02 (manual)*
Pb**
Level 1
0.03-0.08 ppm
3-8 ppm
0.013-0.020 ppm
0.018-0.028 ppm
0.6-1.8 yg/m3
Level 2
0.15-0.20 ppm
15-20 ppm
1.13-1.70 m3/min
0.033-0.040 ppm
0.046-0.055 ppm
3.5-5.9 yg/m3
Level 3
0.35-0.45
ppm
35-45 ppm
0.053-0.059
ppm
0.074-0.083
ppm
Level 4
0.80-0.90
ppm
80-90 ppm
Concentration levels corresponding to flow rates of .2 L/min,
**Concentration levels corresponding to flow rates of 50 cfm.
When a request is made to the NADB for ambient air quality monitoring
data, the requestor receives the P and A data along with the routine moni-
toring data. The requestor, or user, of the data can feel more confident
that the data are of the quality indicated by the assessments and that the
data have been obtained from an agency having a planned and documented
quality assurance system. The EPA can also rely on the data in producing
its control strategies and determining whether standards have been met.
-------
SECTION 2
NATIONAL RESULTS
NATIONAL DATA REPORTING
A measure of the completeness of the precision and accuracy data reporting
is the percentage of reporting organizations which were required to report data
for a particular pollutant and which have reported results for at least one
calendar quarter for that pollutant. Table 3 shows the progress in data re-
porting over the years 1981 through 1986. Reporting for the manual methods
for Pb, S02, and N02 was required by the regulations beginning January 1, 1983.
The reporting of 1986 Pb data has remained the same as for 1985. Only two
reporting organizations, Guam and Hawaii, continue to use the manual S02 method.
The bO percent reporting indicates that no reports were received from one of
the two. The manual N02 method is no longer used at any SLAMS sites. The
percentages of reporting organizations reporting some data for TSP and the
continuous methods for 1986 have remained essentially the same as for 1985.
The reporting organizations which should have
but did not are listed in Section 3.
reported data for 1986
TABLE 3. PERCENT OF REPORTING ORGANIZATIONS
REPORTING PRECISION AND ACCURACY DATA
Pollutant
measurement
CO
S02
N02
°3
TSP
Pb
S02 (manual )
N02 (manual)
1981
77
82
56
83
94
__
1982
89
93
72
89
97
--
* W
1983
99
96
88
99
99
93
75
86
1984
99
97
94
99
99
92
80
100
1985
96
97
96
95
99
96
75
100
1986
97
96
94
96
98
96
50
NATIONAL ACTIVITY IN PERFORMING PRECISION CHECKS AND ACCURACY AUDITS
A review of Tables 4 and 5 clearly indicates the considerable increase
in the total number of precision and accuracy checks from the beginning of
-------
TABLE 4. YEAR-TO-YEAR ACTIVITY OF PRECISION AND ACCURACY ASSESSMENTS FOR THE MANUAL METHODS
Pollutant
TSP
Pb
S02
N02
Year
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
Avg. no. of
samplers
2,334
2,538
2,662
2,650
2,455
2,128
73
164
452
492
486
413
172
63
46
36
20
8
185
83
77
50
36
12
Precision
Avg. no. of
collocated
sites
317
338
342
338
331
316
13
32
76
92
86
61
34
21
15
10
6
2
38
25
25
13
10
5
No. of valid
collocated
data pairs
13,335
16,281
16,816
17,152
16,462
15,744
473
1,704
3,885
3,937
3,508
2,749
965
706
389
297
185
62
1,422
1,168
1,324
691
469
174
No. of
data pairs
per site
42.1
48.2
49.2
50.8
49.7
49.8
36.4
53.2
51.1
42.8
40.8
45.1
28.4
33.6
25.9
28.3
30.8
31.0
37.4
46.7
53.0
53.2
46.9
34.8
Accuracy
No. of
audits
x levels
5,840
6,461
6,989
7,436
6,820
6,292
581
655
1,389
1,657
1,616
1,612
711
551
301
203
174
155
769
583
348
175
161
92
No. of
audits per
sampler
2.5
2.6
2.6
2.8
2.8
3.0
4.0
2.0
1.5
1.7
1.7
2.0
1.4
2.9
1.1
1.9
2.9
6.5
4.2
2.3
1.5
1.2
1.5
2.6
-------
TABLE 5. YEAR-TO-YEAR ACTIVITY OF PRECISION AND ACCURACY ASSESSMENTS FOR THE CONTINUOUS METHODS
CTi
Pollutant
CO
S02
N02
Oi
uo
Year
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
Avg. no. of
analyzers
282
354
447
424
426
391
420
566
633
630
571
560
127
193
235
240
232
206
404
514
598
579
574
529
Precision
No. of
precision
checks
8,248
13,089
15,714
14,692
14,465
13,225
10,851
23,144
36,887
38,312
22,863
30,609
2,498
6,876
9,299
8,653
7,695
6,686
10,536
18,964
21,342
20,031
18,822
17,438
Precision
checks
per analyzer
29.2
37.0
35.2
34.7
34.0
33.8
25.8
36.6
58.3
60.8
40.0
54.1
19.7
35.6
39.6
36.0
33.2
32.5
26.1
36.9
35.7
34.6
32.8
33.0
Accur
No. of
accuracy
audits x levels*
856
1,180
1,501
1,265
1,143
1,052
1,016
1,248
1,625
1,500
1,397
1,272
320
442
635
589
550
510
1,162
1,328
1,705
1,629
1,499
1,328
-acy
No. of
audits
per analyzer
1.01
1.11
1.12
0.99
0.89
0.90
0.81
0.73
0.86
0.79 i
0.82
0.75
0.84
0.76
0.90
0.82
0.79
0.83
0.96
0.86
0.95
0.94
0.87
0.84
*Levels 1, 2, and 3 only.
-------
the PARS system through 1984 for all pollutant methods except the manual
S02 and N02 methods. The increase in effort resulted because of the effec-
tivity of the regulation requirements for P and A data for the NAMS* sites on
January 1, 1981 and for the SLAMS on January 1, 1983. The reduction in the
manual N02 and ' SO2 methods has resulted from the replacement of the manual
methods with continuous analyzers. The average number of analyzers/samplers
and the total number of precision checks have decreased from 1983 and 1984
for all measurements. However, the numbers of precision checks per site
have decreased in some cases (manual and continuous NO;?), increased in other
cases (Pb, manual and continuous S02, and 03), and remained the same for TSP.
The reduction of the number of analyzers/samplers since 1983 has reresulted
in corresponding decreases in the number of precision checks and accuracy
audits. However, the numbers of accuracy audits per analyzer/sampler have
increased for all methods except CO, continuous S02 and 03.
For the manual methods, Table 4 shows the average number of data pairs
per collocated site for precision checks and the average number of accuracy
audits per sampler. If the collocated samplers are operated every sixth day,
there should be 365/6 = 61 data pairs per year, assuming that all the results
are above the detection limit. This level of precision checks is being ap-
proached for the TSP and Pb. The regulations require that each TSP sampler/
site be audited for accuracy at least once each year, and that the laboratory
for the other manual methods be audited at least twice per quarter. The
computed average number of audits per TSP sampler is well above the required
frequency.
For the continuous methods, the minimum frequency for precision checks
is once every two weeks or 26 per year. Table 5 shows that CO, N02, and 03
analyzers are being checked somewhat more frequently, about 33, and that the
S02 analyzers are being checked at almost twice a week. Perhaps experience
has indicated that the S02 analyzers drift at a higher rate than the other
instruments. The regulations require at least one accuracy audit per ana-
lyzer/site per year. The average number of audits per analyzer for the
continuous methods indicates that from 10 to 25 percent of the analyzers are
not being audited as required by the regulations. (Note: The tabulated
values consider only the audits at the three lower concentration levels.
Analyzers requiring level four audits, e.g., episode monitors, are not
considered.)
A comparison can be made between the average number of samplers for
which PARS data are reported and the number of SLAMS/NAMS sites in the
nation:
*See Glossary, Appendix A, for definitions,
-------
No. SLAMS/NAMS
sites
Avg. no. samplers
reporting PARS
data
1984
1985
1986
1984
1985
1986
Continuous methods
S02
540
538
534
630
571
566
N02
252
232
231
240
232
206
03
600
617
622
579
574
529
CO
439
440
450
424
426
391
Manual methods
TSP
2477
2424
2363
2650
2455
2128
Pb
382
403
414
492
486
413
S02
14
6
6
36
9
8
N02
15
14
0
50
36
12
It appears that for all of the manual methods and for continuous S02
in 1984 and 1985, P and A data from more samplers were received than existed
as SLAMS/NAMS sites. Presumably, these extra or additional samplers are
being used for special purpose monitoring and/or both samplers at collocated
sites (manual methods) are being counted. However, in 1986 neglecting the
manual S02 and manual N02 methods for which none or a few SLAMS sites exist,
the average number of samplers reporting PARS data is less than the number of
SLAMS/NAMS sites for all methods except continuous S02- This would indicate
less than the required P and A reporting for most of the methods (except for
continuous S02).
-------
1986 RESULTS FROM THE PARS PROGRAM
Estimates of precision and accuracy are required to be computed and
reported for each calendar quarter by each Reporting Organization (a State
or local agency) as percentage deviation values. For precision, the re-
peatability for each check is measured as the deviation from the expected
value as a percentage of the expected value. For accuracy, the deviation
of the audit value from the true value is measured as a percentage of the
true value. For both precision and accuracy, 95 percent probability limits
are computed for the percentage values from the average and standard devia-
tions of the individual percentage values:
"U ± 1.96 S
where D~ = the average of the individual percent differences;
S = the standard deviation of the individual percent differences;*
1.96 = the multiplication factor corresponding to 95% probability.
It is these upper and lower 95% probability limits which are reported and
discussed in this report.
Moreover, it should be noted that the data and the evaluations present-
ed in this report include any outlier values which may have been reported
by the States and local agencies. It is possible that the presence of
outliers might influence such comparisons by having undue impact on average
values for individual reporting organizations.
The probability limits presented throughout this report for states, re-
gions, and the nation have been calculated using the formulas shown in Ap-
pendix B and thereby most appropriately reflect the total variability within
the entity involved. (Note: Probability limit values in this report and the
19845 and 19856 reports in Tables 6, 7, 12, and 14 and Figures 1 through 10
cannot be validly compared with corresponding tables and figures of previous
reports.3»4 The limits given in this report are generally wider than corre-
sponding limits of previous reports for the reasons discussed in Appendix
B.)
Table 6 exhibits the national probability limits for each of the man-
ual pollutants.
*For the precision of manual methods obtained from paired observations, the
standard deviation, S, is divided by /2, to obtain variability estimates
that apply to individual reported values.
-------
TABLE 6. NATIONAL PRECISION AND ACCURACY PROBABILITY
LIMIT VALUES FOR MANUAL METHODS FOR 1986
Pollutant
TSP
Lead
Sulfur
dioxide
Nitrogen
dioxide
Precision
Probability
limits (%)
Lower Upper
-12 +13
-20 +20
-19 +22
-48 +45
Accuracy
Probability limits (%)
Level
Lower
-14
-43
-8
1
Upper
+11
+21
+12
Level 2
Lower Upper
-8 +8
-11 +9
-18 +14
- 6 +8
Level
Lower
__
-17
-4
3
Upper
__
+17
+5
The precision limits reflect the
in the field to collect and analyze
spread of the limits may be somewhat
tively low concentration levels.
repeatability of the methodology used
the samples at ambient levels. The
inflated due to measurements at rela-
The accuracy of the manual methods indicates the limits at predetermined
concentration levels for the chemical analysis performed in the samples for
lead, sulfur dioxide, and nitrogen dioxide. For the TSP method, the accuracy
measurement is for the flow rate only. The probability limits for manual ac-
curacy are very good and reflect the quality of work done in the chemical
laboratories for lead, sulfur dioxide, and nitrogen dioxide analyses, and in
the field for flow rate measurement for the TSP method. Because of the
continual replacement of the manual SOg and N02 methods with continuous meth-
ods, very little data are reported for these methods and further discussion
of these manual methods is limited. However, the detailed results, if any,
are tabulated in Appendix D.
The precision and accuracy limits for automated methods are presented in
Table 7. The results are nearly the same as reported for 1985.
10
-------
TABLE 7. NATIONAL PRECISION AND ACCURACY PROBABILITY
LIMIT VALUES FOR AUTOMATED ANALYZERS FOR 1986
CO
S02
N02
03
Precision
Probability
limits (%)
Lower Upper
-9 +9
-10 +10
-11 +11
-10 +8
Accuracy
Probability limits (%)
Level 1
Lower Upper
-16 +15
-17 +16
-21 +20
-14 +13
Level 2
Lower Upper
-8 +8
-13 +12
-13 +11
-17 +16
Level 3
Lower Upper
-8 +7
-12 +11
-13 +11
-10 +9
Level 4
Lower Upper
-15 +15
-14 +13
-14 +6
-8 +6
NATIONAL PRECISION RESULTS COMPARISON
Figure 1 shows the national probability limits for precision for the
various methods. With data from the four most recent years, some minor
trends are evident. Some slight improvement, as measured by a continued
reduction in the spread of the limits, is noted for the manual methods
TSP and S02, and the continuous methods ~ 03, CO and N02. Increases in
precision for the manual methods, Pb and N02, and for the continuous method
S02 may be due to small sample (N02) or statistical variations. The slight
but persistent negative bias for the continuous S02 method indicates that on
the average there is some negative instrument drift from the most recent cal-
ibration or instrument adjustment to the time of the biweekly precision
check.
Although the manual methods for Pb, S02, and N02 were not required to
be reported until 1983, a number of agencies began reporting in 1981. The
manual S02 and N02 methods are much more variable than the continuous methods.
11
-------
88
78-
68-
58-
48-
30-
28-
10-
0
-10-
L -20-
M ~30~
; -40-
T -50-H
S -60-
% -78-
-88
NATIONAL VALUES FOR PRECISION
1983-1986
[ 63
84
85
I 86
Figure 1. National precision probability limits for 1983 through 1986.
NATIONAL ACCURACY RESULTS COMPARISON
Figures 2a and 2b show the national values for accuracy audits for the
continuous and manual methods, respectively, for the four most recent years,
1983-1986. Improvement for the manual methods is not evident except perhaps
for Pb and N02. The variability for the TSP method remains the same and the
S02 method has shown a definite increase. The results for the manual methods
for S02 and N02 vary considerably from year to year because the methods are
used in only 2 or 3 Regions and are being replaced by the continuous methods.
Slight improvement was evident for all the continuous methods over past years,
but has not continued for 1986. The continuous methods for S02 and N02 show
more inaccuracy than all other methods. However, it is pointed out that the
accuracy audits for the manual methods check only a portion of the measure-
ment method.
12
-------
NATIONAL VALUES FOR ACCURACY
1983-1986
MANUAL METHODS
NATIONAL VALUES FOR ACCURACY
1983-1986
CONTINUOUS METHODS
Pollutant and Level
a
Pollutant and Level
b
Figure 2. National accuracy probability limits for 1983 through 1986.
Although the continuous N02 method is more variable than the other
methods, it has shown the greatest improvement, particularly for the level
1 concentration.
The general, and expected, pattern of variability across levels is very
evident, with the greatest percentage variability at the lowest concentration
levels. The slight negative biases for the continuous S02 method is consis-
tent across all three levels. A possible cause is that, on the average, a
negative drift occurs with these analyzers from the time of last calibration
or instrument adjustment until the time of the accuracy audit.
NATIONAL FREQUENCIES
Table 8 contains the 1986 percentiles for precision probability limits
and accuracy probability limits at levels 1, 2, 3, and 4. The percentiles
are based on the total number of reporting-organization quarters of data.
The individual quarter of data consists of an upper and lower probability
limit for precision, and upper and lower probability limits for accuracy for
each of the levels. The narrower the distribution, the better the data
quality. For example, for precision for CO, the upper 5 percentile value for
the upper limit is +15%, and the lower 5 percentile value for the lower limit
is -15%. It can be seen from both Figure 2 and Table 8 that CO shows the
tightest range of the pollutants presented. The variabilities shown in Table
8 are consistent with those shown in Figures 1 and 2. The 95th percentiles
provide criteria beyond which a reported probability limit may be considered
excessive and for which the computation should be rechecked or the measurement
system investigated and corrected, if so indicated.
13
-------
TABLE 8. PERCENTILES OF QUARTERLY PROBABILITY LIMITS FOR
ALL REPORTING ORGANIZATIONS (1986)
ENVIRONMENTAL PROTECTION AGENCY
DATE 10/16/87
PROGRAM PA250
MANUAL METHODS
EMSL PRECISION/ACCURACY REPORTING SYSTEM
FREQUENCY DISTRIBUTION OF PROBABILITY LIMITS
DATA SELECTED FOR THE YEAR 1986
NATIONAL FREQUENCY SUMMARY
PAGE NO. 147
REPORT PA250R01
POLLUTANT
LEVEL
MANUAL METHODS
OF LOMER PROBABILITY LIMIT STD UPPER PROBABILITY LIMIT STD
REP.ORS.-QTR MIN 01X 05X 10X 25X SOX 75X 90X 95X 99X MAX MEAN DEV MIN 01X 05X 10X 25X SOX 75X 90X 95X 99X MAX MEAN DEV
111101 - TSP
PRECISION
ACC-LVL 2
112128 - PB
PRECISION
ACC-LVL 1
ACC-LVL 2
546
511
219
273
273
-34 -29 -21 -17 -13 -09 -06 -03 -02 -00 +20 -10
-25 -21 -14 -11 -08 -06 -03 -01 -00 +02 +09 -06
5.9 -03 +01 +02 +03 +06 +09 +14 +19 +24 +33 +46 +11 6.9
4.5 -10 -04 -01 +01 +03 +05 +08 +12 +15 +25 +46 +06 5.4
-51 -45 -37 -28 -19 -10 -07 -03 -01 -00 -00 -14 10.5
-59 -48 -18 -15 -10 -06 -03 -01 +01 +05 +13 -08 7.8
-37 -28 -19 -12 -09 -05 -02 -01 +01 +03 +08 -06 6.0
-20 -09 -00 +02 +05 +10 +17 +30 +38 +65 +99 +13 13.8
-30 -12 -03 -02 +01 +04 +07 +13 +17 +38 +45 +05 7.4
-15 -08 -03 -01 +01 +03 +07 +10 +15 +18 +69 +04 6.5
181102 -
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
142401 - S02
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
142602 - N02
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
46
12
25
1
6
11
14
10
13
12
11
10
_39 .39 -32 -Z6 -17 -09 -04 -00 +05 +11 +11 -11 10.9
-31 -31 -31 -17 -09 -06 -01 -01 -00 -00 -00 -08 8.9
-27 -27 -24 -17 -10 -06 -03 -01 -00 +04 +04 -08 7.3
-13 -13 -13 -13 -13 -13 -13 -13 -13 -13 -13 -13 Q.r
-32 -32 -32 -32 -16 -08 -08 -06 -06 -06 -06 -14 9.7
-75 -75 -75 -51 -48 -30 -11 -08 -07-07 -07 -31 22.6
-37 -37 -37 -32 -14 -10 -04 -01 +02 +02 +02 -12 11.1
-40 -40 -40 -14 -09 -08 -04 -01 -01 -01 -01 -10 11.1
-09 -09 -01 +02 +04 +06 +17 +22 +24 +33 +33 +10 8.8
+01 +01 +01 +01 +03 +08 +12 +16 +42 +42 +42 +09 11.4
-02 -02 -01 -01 +02 +06 +15 +27 +29 +43 +43 +10 11.2
+17 +17 +17 +17 +17 +17 +17 +17 +17 +17 +17 +17 0.0
+04 +04 +04 +04 +11 +26 +33 +35 +35 +35 +35 +21 12.3
+05+05 +05 +05 +05 +10 +20 +23 +40 +40 +40 +13 10.8
-03 -03 -03 +02 +04 +06 +13 +20 +25 +25 +25 +08 7.6
-03 -03 -03 +01 +01 +06 +17 +22 +22 +22 +22 +08 9.1
.47 .47 .47 -38 -34 -28 -11 -05 +11 +11 +11 -22 16.6 -41 -41 -41 -16 +05 +07 +55 +65 +89 +89 +89 +20 36.0
-18 -18 -18 -17 -11 -07 -01 -00 -00 -00 -00 -08 6.2 -01 -01 -01 +02 +05 +12 +14 +16 +16 +16 +16 +10 5.8
-10 -10 -10 -08 -06 -03 -01 -01 -00 -00 -00 -04 3.1 -02 -02 -02 -01 +02 +04 +08 +12 +12 +12 +12 +05 4.7
-06 -06 -06 -04 -04 -01 -00 +03 +03 +03 +03 -02 2.6 -01 -01 -01 +01 +01 +04 +04 +06 +06 +06 +06 +03 2.3
-------
(Continued)
ENVIRONMENTAL PROTECTION AGENCY
DATE 10/16/87
PROGRAM PA250
TABLE 8. PERCENTILES OF QUARTERLY PROBABILITY LIMITS FOR
ALL REPORTING ORGANIZATIONS (1986)
EMSL PRECISION/ACCURACY REPORTING SYSTEM
FREQUENCY DISTRIBUTION OF PROBABILITY LIMITS
DATA SELECTED FOR THE YEAR 1986
NATIONAL FREQUENCY SUMMARY
PAGE NO. 146
REPORT PA250R01
POLLUTANT
NUMBER I
LEVEL REP.ORG.-
C42101 - CO
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C42401 - S02
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C42602 - N02
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C44201 - 03
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C42601 - NO
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
362
281
274
268
9
413
332
328
322
44
236
168
162
160
9
394
305
298
291
22
4
2
2
2
AUTOMATED ANALYZERS
LOWER PROBABILITY LIMIT STD UPPER PROBABILITY LIMIT STD
REP.ORG.-QTR MIN 01X 05X 10X 25X SOX 75X 90X 95X 99X MAX MEAN DEV MIN 01X 05X 10X 25X 50X 75X 90X 95X 99X MAX MEAN DEV
-27 -24 -15 -12 -09 -06 -04 -02 -01 +01 +05 -07
-65 -59 -23 -19 -14 -08 -02 -00 +01 +06 +16 -09
-41 -21 -14 -12 -08 -05 -02 -00 +02 +06 +06 -05
-61 -20 -15 -11 -07 -04 -02 -00 +02 +06 +07 -05
4.5 -05 -03 -00 +01 +03 +06 +09 +13 +15 +22 +48 +07 5.4
9.7 -14 -11 -04 -01 +03 +07 +14 +20 +26 +52 +99 +09 11.2
5.3 -05 -05 -02 -00 +02 +04 +08 +12 +14+24 +26 +05 5.1
5.9 -05 -05 -01 -00 +02 +04 +07 +11 +13 +23 +34 +05 4.9
-25 -25 -25 -25 -13 -10 -01 -00 -00 -00 -00 -09 8.4 -00 -00 -00 -00 +03 +05 +13 +33 +33 +33+33 +09 10.6
-42 -33 -22 -19 -13 -09 -07 -05 -03 +01 +19 -11 6.3
-76 -
-------
SECTION 3
REGIONAL RESULTS
REGIONAL DATA REPORTING
All reporting organizations having SLAMS/NAMS sites for the criteria
pollutants are required to report P and A data. The numbers of such re-
porting organizations are listed in Table 9. Note that only two reporting
organizations use the manual SOe method at SLAMS sites and none uses the
manual N0£ method.
TABLE 9. TOTAL NUMBER OF REPORTING ORGANIZATIONS REQUIRED
TO REPORT FOR THE YEAR 1986, BY POLLUTANT
Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation
Automated
CO S02
C42101 C42401
6
3
10
21
20
10
11
2
11
4
98
6
5
12
22
25
10
9
4
9
3
105
methods
N02
C42602
5
2
11
10
12
9
5
3
9
2
68
03
C44201
6
3
12
26
24
10
11
5
11
2
110
TSP
111101
6
4
14
34
30
14
12
9
12
4
139
Manual
methods
Pb S02
112128 142401
5
3
9
11
15
11
9
3
8
4
78
0
0
0
0
0
0
0
0
2
0
2
N0£
141602
0
0
0
0
0
0
0
0
0
0
0
The breakdown of data completeness (defined as the percentage of re-
porting organizations which reported P&A data to EPA relative to the number
required to report each quarter) is given in Table 10.
16
-------
TABLE 10. PERCENTAGE OF REPORTING ORGANIZATIONS WITH COMPLETE DATA
IN PARS FOR THE YEARS 1983, 1984, 1985 and 1986
Manual Methods
Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation
Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation
TSP
111101
83 84
98 100
72 97
99 100
97 99
99 97
95 98
97 95
96 100
82 95
100 92
95 98
85
100
88
98
96
94
97
99
100
73
84
94
86
74
93
98
95
95
96
98
88
56
93
91
83
95
75
88
78
89
83
66
75
59
59
79
CO
C42101
83 84
85 91
92 88
100 1UO
83 84
78 85
91 97
78 78
68 98
77 89
88 94
83 90
85
90
88
99
85
83
90
80
90
63
84
84
86
71
91
97
82
85
89
84
80
72
85
83
83
92
66
100
79
77
82
69
100
60
88
80
Pb
112128
84 85
99 90
100 100
95 92
80 92
85 93
88 95
83 76
83 75
74 53
72 72
85 85
Automated
SO 2
C42401
84 85
98 96
83 63
100 100
87 79
92 86
93 84
77 72
98 91
93 64
97 11
92 82
S02
142401
86
59
96
88
85
81
85
68
77
46
84
73
83
_
94
--
--
50
72
84 85
__
__
86
__
100 100
0
43
64
0
19
34
~
--
19
19
N02
141602
83 84 85 86
100
75 100 ~
100 -
63 100 100
o
73 100 100
Methods
N02
C42602
86
71
80
97
90
92
93
85
82
73
80
86
83
56
100
96
51
65
70
68
92
58
81
69
84
80
100
98
63
79
85
75
92
94
100
88
85
50
88
89
65
66
83
69
83
71
81
73
86
49
100
94
71
75
85
84
80
74
18
78
03
C44201
83 84 85 86
79 79 73 57
96 99 67 91
99 100 99 95
81 79 81 82
76 88 78 82
96 98 96 94
80 73 68 75
96 100 88 69
75 95 59 74
94 100 100 100
84 91 80 82
NOTE Means no data was required, there being no SLAMS sites for these
pollutants.
17
-------
From 1985 to 1986, the percentages of reporting on a national level
decreased for all manual methods and increased for all automated methods,
except CO.
A substantial lack of reporting of 1986 data occurred for the fourth
quarter, during the time when plans and preparations were being made for the
reporting of all raw data, beginning January 1, 1987. The regulations per-
mitted the reporting organizations to begin using the raw data reporting
system beginning as early as for the third quarter data of 1986. Start-up
problems with the raw data reporting system were no doubt responsible for
some loss of data for the third and fourth quarters.
A number of reporting organizations having SLAMS/NAMS sites for certain
pollutants have reported r\o_ precision or accuracy data for 1986 for these
pollutants:
Regi on
State
Reporting organization
Number
Name
Pollutant
I
II
IV
VII
VIII
IX
IX
X
NH 30001 New Hampshire***
VI 55001 Virgin Islands
FL 10018 Dade County
TN 44005 Chattanooga-Hamilton Co.,*
Air Pollution Control
MO 26003 St. Louis City**
MT 27003 Great Falls City-County
MT 27004 Missoula City-County
CA 05036 San Diego***
HI 12120 Hawaii
NV 29100 State of Nevada***
NV 29100 State of Nevada***
NV 29200 Washoe County
NV 29300 Clark County**
GU 54100 Guam***
GU 54100 Guam***
WA 49001 Washington
NO 2
TSP, S02
S02
CO
Pb
03
03
Pb
S02, N02
CO
03
CO, N02,
03, TSP
Pb, CO
TSP, Pb
SO 2
(manual)
N02
*Repeats from 1985.
**Repeats from 1984 and 1985.
***Repeats from 1983, 1984 and 1985.
18
-------
Precision and accuracy reporting for 1986 was complete only for the
following Region and pollutant combinations:
Region Pollutant
II
X 03
Considering the reporting for all pollutants (omitting the manual S02
and N02 methods) and all reporting organizations, the reporting organizations
of Region III were most complete for 1986 (95%). Region III was also the
most complete in 1983, 1984 and 1985. Region I data was the least complete
(59%).
Percentage of
reports complete
Region 83 84 85 86
I
II
III
IV
V
VI
VII
VIII
IX
X
When considering the various pollutant methods across all Regions, re-
porting was most complete for the TSP and S02 methods and least complete for
the manual S02 method, the same as for 1984 and 1985.
84
84
97
80
83
74
65
88
66
85
91
95
99
86
88
82
80
95
83
93
83
82
96
85
83
81
77
88
57
82
64
92
95
84
85
90
82
79
66
77
19
-------
Pollutant
TSP
03
CO
S02
Pb
N02 (manual)
S02 (manual)
NO 2
Percentage of
reports complete
83
86
95
84
83
80
79
73
72
69
98
91
90
92
85
100
64
88
94
80
84
82
85
100
34
73
91
82
83
86
73
--
19
78
REGIONAL COMPARISONS
Figures 3 through 10 compare the precision and accuracy probability
limits for 1983, 1984, 1985 and 1986. These comparisons are presented for
each pollutant on a Region by Region basis.
C0_ (Figure 3)
Only Regions VI, VIII, and X showed a noticeable improvement from 1985
for precision. Regions II, IV and V were worse in 1986 for all accuracy
levels than in 1985. Regions I, IV and VI, showed consistent improvement at
all accuracy levels.
SO? (Figure 4)
Regions II, V and VI were consistent in improvements for all three
levels of accuracy; however, Regions I, VIII and X were worse in precision
and all levels of accuracy.
N02 (Figure 5)
More regions showed improvement than not. Regions IV, and VII were
better at all accuracy levels in 1986 than in 1985 ~ Regions II and VIII
were worse.
(Figure 6)
For 1985 more regions showed improvement in precision and accuracy for
ozone than for any other measurement. These significant improvements were
possibly attributed to the use of the standard reference photometers (SRP's)
developed by the National Bureau of Standards for EPA and located at:
20
-------
40-
P
R
0
B
A
B
I
L
I
T
Y
30-
20-
10-
L -10-|
M
I -20-
T
S
-30-
-40-
CO PRECISION
1983-I 986
p
R 30-
0
; 20-
B
1 10-
. IkJ-
I
T 0
V
L -10-
I
M
I -20-
T
S
-30-
%
4d-
t
1
\
-i
- m
- m
- E
]w
1
^
CO ACCURACY LEVEL 1
1983-1986
1
a
_ E -J- = :
: : :
: : :
: : :
: E : '
: = :
: : : [ [
HJ : . : . : .
m n
1 p
= " i ' ! ' IH
- - Hr
: : : B=
: : : H=
: ' : : JF
j - It
1 it ^ -^
i
-
.
~:
u
"^ D
II 1 ! I 1 I 1
|83
Q64
|es
i 86
*J fc^fcl %QtQ
40
30-
20-
10-
0
-10-^
-20-
-30-
-40
CO ACCURACY LEVEL 2
1983-1986
~r
-------
S02 PRECISION
1983-1986
40
R 38-
0
B
T
. 10-
L
I
T 0
Y B
L IB-
M
I -20-
T
S
-30-
'/,
4PI-
- "1
^ fc B Ll - ' H
: 1 : 1 : :Jj : I -
1 "'-m :m 'm ' * '
: 1 L f -f ~\£- I '
IF LJ LU g_B
w- \ 1 111
\ 1 -3 fc «,
:
.
_j
B
^
: :
; -
: '.
= -
| t
3
B
. I
: 1
- 1
:J1
LJ
i i i
6 1 % 9
50-
P 40-
R
n
B 30-
A
B 20-
I
L 10-
I
T o,
T 0_
, -10-
I
M -20-
I
T -30-
s
x -^
!
.
| -3KJ
\
: 9g
: m
' m
f
^
S02 ACCURACY LEVEL
1 983- 1 986
D
".
.
"
-
r :'._,' ' .
! . : . : .
: ! : ! : i :
I I '. \
: ' i I '
' 1 "-f : a :
LJ ^ ' !
'
3 : a n
i ' - i :
i = - | Q,
: r :
i : = _
! " i ! = "
1 . ' E : '
. "~^ -
n
083
184
85
86
1 1 l 1 1 1 1
P
R
0
B
A
B
I
L
I
T
Y
L
I
M
I
T
S
50
40-
30-
28-
10-
0
-10-
-20-
-30-
-40-
-50
S02 ACCURACY LEVEL 2
1983-1986
l
1
\
6
50
40-
30-
20-
10-
0
-10-
-20-
-30
-40
-50
S02 ACCURACY LEVEL 3
1983-1986
~T
e
83
84
85
§86
Figure 4. Continuous S02 precision
1983 through 1986.
and accuracy by region for
22
-------
60-
40-
20-
L
I -20
M
I
T
S -40
-60
N02 PRECISION
1983-1986
60
40-
20-
P
R
0
B
A
B
I
L
I
T
Y
L
I -20
M
I
T
S -40
-60
N02 ACCURACY LEVEL 1
1983-1986
ri
-
-
L
9
: -
E F i T
-. ' *M_ ' E " = '
: . f . . .
I i - 1
Fn i i !
i
E - !
: - : -
).
:
;
|J :
i
:
=
E
j
E.! B»
85
86
N02 ACCURACY LEVEL 2
1983-1986
-60
60-
40-
20-
-20-
-40-
-60-
N02 ACCURACY LEVEL 3
1983-1986
\ ^
\ i i i i r
t, <3 6 1
-------
40-
30-
20-
10-
P
R
0
B
A
B
I
L
I
T
Y
L -10-
M
I -20-
T
-30-
-40-
03 PRECISION
1983-1986
p
R
0
B
A
B
I
L
I
T
Y
L-,e
M
I -20
T
S
-30
A
-40
40
30-
20-
10- r
6 1
03 ACCURACY LEVEL 1
1 983- 1 986
I :
S
1
a
»
Vi "
i -
m -
P "
I
i:
f -
f :
1
7, *J
r^B n
= : 1 :
= - !
l|p :
». * 6
- = - |c
5 \
_ \ ; [ ;
. B D-^ :
-i %
\ \
I :
a
=
1
|
^
83
84
85
86
P
R
0
B
A
B
I
L
I
T
Y
40-
30-
20-
10-
L -10-
M
I -20-
T
S
-30-
-40
03 ACCURACY LEVEL 2
1983-1986
P
R
0
B
A
B
I
L
I
T
Y
L
I
M
I
T
S
I
-i
40
30-
28-
10-
0
-10-
-20-
-30-
-40
03 ACCURACY LEVEL 3
1983-1986
I 83
I 8<*
85
lee
Figure 6. Ozone precision and accuracy by region
for 1983 through 1986.
24
-------
EPA, EMSL, Research Triangle Park, NC
EPA, Region II, Edison, NJ
EPA, Region V, Chicago, IL
EPA, Region VI, Houston, TX
EPA, Region VIII, Denver, CO
California Air Resources Board, Sacremento, CA
and which are being used as calibration reference sources throughout the
nation. In November 1987, a seventh SRP was added at EPA, Region IV, Athens,
GA. And, in 1988, an eighth SRP will be added at EPA, Region I, Lexington,
MA.
However, comparison of 1986 results with those of 1985 does not indicate
a continuing improvement, except for Regions II and III. Regions I and IX
show more variability at all accuracy levels in 1986 than 1985.
TSP (Figure 7)
All regions except III did better in precision in 1986 than in 1985.
Most Regions, except II, III and IV, were better or the same in 1986 compared
to 1985.
Pb_ (Figure 8)
Only four regions II, III, V and VI showed improvement in precision.
And Regions II, III, IV, VI and IX were worse in accuracy in 1986 than in 1985.
Ranking Comparisons of Regions
Ranking comparisons were made to determine the regions and pollutant-
measurement methods which improved most from 1985 to 1986. Improvement was
indicated by a reduction in the spread of the probability limits from 1985 to
1986. Considering all pollutant-measurement methods (except manual SO? and
manual NOg) and precision and accuracy results, the following table lists the
regions in order of improvement.
For comparison, these measures of improvement from 1984 to 1985 are also
shown. Interestingly, there were more indications of improvement from 1984
to 1985 than from 1985 to 1986. Also, Region IV averaged at "no change" for
both comparisons and Region X was least improved for both comparisons.
25
-------
p
R
0
B
A
B
I
L
I
T
Y
40-
30-
20-
10-
L -10-
I
M
I -20-
T
S
-30-
y.
-40-
TSP PRECISION
1983- 1 986
40-
p
R
0
B
A
B
I
L
I
T
Y
L
I
M
I
T
S
30-
20-
10-
0-
-10-
-20-
-30
-40
TSP ACCURACY LEVEL 2
1983-1986
83
I e4
I 85
I 86
Figure 7. TSP precision and accuracy by region for
1983 through 1986.
26
-------
p
R
0
B
A
B
I
L
I
T
Y
L
I
M
I
T
S
tee
80-
60-
40-
20-
0
-20-
-40-
-60
-80
-tee
PB PRECISION
1983-1986
~r
e
83
84
85
86
P
R
0
B
A
B
I
L
I
T
Y
30-
20-
10-
L -|0.
M
I -20-
T
S
-30'
-40
PB ACCURACY LEVEL I
1983-1986
T~
e
40-
0
5 »
B
I
L
I
T
Y
10-
L -10-
M
I -28
T
-40'
PB ACCURACY LEVEL 2
1983-1986
| 83
I 84
[ 85
80
Figure 8. Lead precision and accuracy by region for
1983 through 1986.
27
-------
Relative
Score*
9
8
7
6
b
4
3
2
1
0
-1
-2
-3
-4
-5
-6
-7
-8
Regions
1984 to 1985 1985 to 1986
Ill, IX
VII
I, VI, VIII
V
IV
II
VI
V
VII
I, II
IV
III
IX
VIII
X
Most
improved
No change
Least
improved
*The maximum possible score is +12, i.e., if improvement is indicated for
precision and accuracy for all 6 methods. Similarly, the most negative
possible score is -12.
The most improved measurement method was determined by combining the rank-
ings across regions and across precision and accuracy.
Relative
Score
25
20
15
10
5
0
"O
-10
-15
Pollutants
1984 to 1985 1985 to 1986
25fc 03
8 » CO
7 S02
4 « N02
-3 TSP
9 TSP
7
4
-1
-6
-12
NO 2
CO
03
Pb
SO 2
General comparisons among regions can be made on several different
bases. One basis is that of improvement, as shown by the above analysis.
However, comparisons of improvement may not be fair to those regions which
already demonstrate a history of good precision and accuracy they have
little further room for improvement and may be approaching the inherent
limitations or capabilities of the measurement methods. On the contrary,
28
-------
the regions that have
improvement.
shown poorer precision and accuracy have more room for
A better measure for comparison may be the magnitude of the accuracy
assessments. Not considering any significant biases reflected by the mean of
the upper and lower probability limits, the spread of the limits would be a
good measure of how well the precision and accuracy of measurement systems
are being controlled. The following analysis using the spread of the limits
provide this additional and perhaps better way of making general comparisons
across regions.
Ranking comparisons were also made to determine the regions and pollu-
tant-measurememt methods which were best based on the widths of the probabil-
ity limits for 1986. These comparisons were made separately for the con-
tinuous methods and manual methods and also separately for precision and
accuracy. The rankings were:
Ranking of Regions for Achievement
Continuous Methods
Precision
Rank 1985
1 (best)
2
3
4
5
6
7
8
9
10 (worst)
VI
III, IV
I
II
V, X
IX
VIII
VII
1986
VI
III
I
VIII
II
V
IX
IV
VII
X
Accuracy
Rank 1985
1 (best)
2
3
4
5
6
7
8
9
10 (worst)
III
II
VIII, IX
I
X
VII
V, VI
IV
1986
I
III
VI
IX
V
II
VII
IV
VIII
X
Manual Methods (TSP and Pb Only)
Rank
Precision
1985
1986
1 (best)
2
3
4
5
6
7
8
9
10 (worst)
I, III
X
IX
II
IV, V
VI
VIII
VII
I, II
V
III
X
VI
IX
IV
VIII
VII
Accuracy
Rank 1985
1
2
3
4
5
6
7
8
9
10
(best) V
II
IV
VI, VIII
X
III
I
IX
(worst) VII
1986
X
I, V
IX
IX
VIII
II
VII
IV
III
29
-------
The above rankings are similar to those for previous years. The application
of Spearman's Rank Correlation tests to the above four sets of data indicate
significant correlations (approximate 0.05 significance level) for the pre-
cision rankings, but not significant correlations for the accuracy data.
It could be said that the comparisons of improvement relate to measures
of progress, whereas the comparisons of variabilities, i.e., the* spread of
the limits, relate to measures of achievement.
General
Taking into account the minor trends of improvement, the general con-
sistency from year to year of the differences of results among pollutants and
among levels of the same pollutants on a national basis, and among regions
for given pollutants, is truly surprising. These appreciable differences
which persist from year to year strongly indicate that whatever forces or
causal factors are in action in each region and in each pollutant measurement
system are persistent over the years. These significant differences between
regions should be investigated to identify the major causal factors, since
some regions consistently produce more precise and accurate data than other
regions.
Further, each region should evaluate the differences among the states
and reporting organizations in a similar graphical manner as shown by Figures
3 through 10 and the ranking comparisons of improvement and accomplishment as
shown above. Then investigations should be conducted to determine why some
states or reporting organizations produce better precision and accuracy than
others. Appropriate corrective actions should then be taken to improve
the precision and accuracy of the reporting organizations having the worst
results.
30
-------
SECTION 4
RESULTS BY REPORTING ORGANIZATIONS
Table 11 shows the total number of Reporting Organizations reporting
data to EMSL in 1986. By comparing the numbers between Tables 9 and 11, one
can see the extra effort exerted by some of the State and local agencies to
provide quality assurance information in cases where they have no SLAMS or
NAMS sites. There are an additional 4 reporting organizations for CO, 14 for
continuous S02, 5 for continuous N02, 10 for 03, 17 for TSP, 8 for Pb, 5 for
manual S02 and 6 for manual N02. Apparently, these additional sites are
special purpose monitoring sites or additional local sites not in the SLAMS/
NAMS network.
TABLE 11. NUMBER OF REPORTING ORGANIZATIONS HAVING DATA
IN THE PARS MASTER FILE FOR THE YEAR 1986
Automated
Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation
CO
C42101
6
3
11
20
20
10
11
5
9
4
99
SO 2
C42401
10
4
13
29
26
11
9
4
7
3
116
pollutants
NO 2
C42602
4
2
11
13
12
10
7
3
7
1
70
03
C44201
7
3
13
33
25
10
11
3
9
2
116
TSP
111101
16
3
15
38
30
14
13
9
10
4
152
Manual pol
Pb
112128
5
3
9
14
19
11
8
3
7
4
83
lutants
SO 2
142401
0
0
0
3
1
0
0
1
1
0
6
NO 2
141602
0
0
0
1
1
1
2
1
0
0
6
Appendix D shows the annual combined upper and lower probability limits
for each reporting organization. Each reporting organization can compare
their values with those of other reporting organizations and with the regional
and national values. Also given for each reporting organization are the
following informational items:
31
-------
Continuous methods
Manual methods
No. of SLAMS and NAMS sites
No. of analyzers
No. of precision checks
No. of accuracy audits
No. of SLAMS and NAMS sites
No. of samplers
No. of collocated sites
No. of accuracy audits
Any user of monitoring data from some specific site and time period
should obtain, from the local air monitoring agency, the precision and accu-
racy data for the specific sites and time periods involved.
A graphical summarization of the precision and accuracy probability
limits for each reporting organization for the years 1981 through 1986 will
be issued as a supplement to this report. A review of these charts will
showtime trends and other relationships for the data from each reporting
organization. In addition, some discussion will be presented on control
charts which should be plotted by the reporting organizations for the results
from each monitoring site. Also, some examples of precision and analysis
data presented in graphical form in some of the periodic state reports will
be included.
32
-------
SECTION 5
FURTHER EVALUATION OF PARS DATA
Some Interesting comparisons can be made by considering the correspond-
ing national averages of Tables 6 and 7 and the 50-percentile values of the
probability limits of Table 8. Table 12 compares these limits by consider-
ing the spread, or range, of the limits.
TABLE 12. COMPARISON OF THE 50-PERCENTILE FREQUENCY DISTRIBU-
TION VALUES WITH THE NATIONAL LIMIT VALUES FOR 1986
Manual methods
TSP Precision
Accuracy*
Pb Precision
Accuracy
Continuous methods
CO Precision
Accuracy
03 Precision
Accuracy
N02 Precision
Accuracy
S02 Precision
Accuracy
Nati
Lower
limit
-12
- 8
-20
-11
- 9
- 8
onal values
Upper
limit Range
13 25
8 16
20 40
9 20
9 18
8 16
-10 8 18
-17 16 30
(-10)** (9) (19)
-11
-13
-10
-13
11 22
11 24
10 20
12 25
50-percentile
Lower Upper
limit limit
- 9 9
- 6 5
-10 10
- 5 3
- 6 6
- 5 4
- 7 7
- 7 6
- 9 9
- 7 5
- 9 7
- 9 7
values
Range
18
11
20
8
12
9
14
13
18
12
16
16
*A11 accuracy values for all pollutants are for Level 2.
**Values in parentheses were calculated omitting the 4th quarter of New York
State results when limits were -99 and +99.
33
-------
MANUAL METHODS
For the manual methods, in all cases the spreads (ranges) of the prob-
ability limits are considerably greater for precision than for accuracy.
These differences are consistent for both the National averages and the
50-percentile values. These same relationships have existed for all previ-
ous years. This means that the short-term within-sampler variability (pre-
cision) is larger than the variability of accuracy which would normally in-
clude variations between, or among, samplers as well as imprecision within
samplers. This may seem contradictory at first, but giving consideration
to exactly how the results are obtained and what the results represent will
provide a rational explanation.
TSP. In the case of TSP, the precision results are obtained from col-
located sampler data. They include variability from the sample collection
process, the analytical filter weighing process, the filter handling and
conditioning process, and also the flow rate measurement process; whereas
the accuracy audit is a check only on the flow rate measurement. Further,
the collocated sampler results are obtained at all ambient concentrations
above 1 wg/m^, the detection limit for the method. At low concentration
levels the relative variability is greater than at higher concentrations.
The combined effects of these two causes explain the wider limits for
precision.
Manual SO? and NO?. Similar to the TSP data, the precision results
are obtainedfrom collocated sampler data. They include variability from
the flow measurement, absorbing solutions, sampling, sample handling, and
storage effects (stability) of the samples as well as the laboratory ana-
lytical portion of the method; whereas the accuracy audit is a check only
on the laboratory analytical portion of the method. Further, the collo-
cated sampler results are obtained at all ambient concentrations above the
detection limits of the methods. Many of these concentrations are below
the concentrations of the accuracy audits. At lower concentrations, the
relative variability is greater than at higher concentrations.
As noted from Table 12, these differences are considerable, indicating
that only a small portion of the variability results from the laboratory
analytical part of the method. A very considerable amount of variability of
the method is attributed to other portions of the measurement process. The
very wide limits of uncertainty attributed only to the imprecision of these
methods strongly emphasizes that the manual methods should be replaced by the
continuous analyzers. Alternatively, if any reliance is to be placed on
individual daily data from the manual methods, all of the various portions of
the measurement processes must be much more closely controlled, if possible.
Pb. The precision estimates for Pb are obtained from the analysis of
duplicate strips from the same hi-vol filter. Consequently, actual varia-
bility of Pb content across the length of the filter, filter handling (with
possible loss of particulate), variation in cutting filter strips, and the
34
-------
extraction of real-world participate are involved in addition to the chemi-
cal analytical portion of the method. The accuracy audit data are obtained
from the chemical analysis of strips to which known amounts of water-soluble
Pb salts have been added and thus do not involve the other portions of the
measurement process, nor do they involve real-world particulates.
Further, similar to the other manual methods (TSP, N02, and S02), the
precision estimates are obtained at all concentrations above the detection
limit. Many of these concentrations are less than those of the accuracy
audits. At lower concentrations, the relative variability is expected to be
greater than at higher concentrations.
Beginning January 1, 1987, the precision for Pb samplers must be esti-
mated by the use of collocated samplers similar to the TSP measurement. It
is expected that more variability will be exhibited by the differences in
the results between collocated samplers then between duplicate strips of the
same filter. In anticipation of the effectivity of the regulation, some
agencies may have begun using collocated samplers for Pb during the latter
part of 1986.
Manual Methods (General). To make valid comparisons of the precision and
accuracy data, such comparisons should be made at the same concentration
levels. Only then will it be possible to determine whether the larger var-
iabilities of the precision estimates are due to differences in concentra-
tion level or to the larger scope of the measurement system involved.
Such comparison studies can be accomplished when the raw concentration
data are obtained from the State and local agencies for each precision and
accuracy check beginning January 1, 1987, as specified by the proposed reg-
ulation revisions to Appendix A of 40 CFR, Part 58 promulgated March 19,
1986. Heretofore, only the reporting organizations could perform such stud-
ies, since only they had the raw data available.
The estimation of the magnitude of the contributions of the various
sources of variability to the total measurement processes could also be
systematically studied in specially designed experiments.
CO, S02, N02, 03 (Continuous Methods). The national values for precision
for the continuous methods are nearly the same as the accuracy values at
level 2. For these continuous measurement methods, the precision assess-
ments reflect the within-instrument variability obtained from bi-weekly
checks at relatively low concentrations, namely
and
8-10 ppm for CO
.08 -.10 ppm for S02, N02, and 03.
In comparison, the accuracy audits include between-1nstrument variability as
well as imprecision, but are conducted at somewhat higher (level 2) concen-
trations.
35
-------
15 - 20 ppm for CO
.15 - .20 ppm for S02, !%, and 03.
Thus, the added between-instrument variability for the level 2 accuracy audit
is almost exactly offset by the improved percentage within-instrument varia-
bility for the precision.
Level 1 accuracy audits are conducted at concentrations of
.03 - .08 ppm for CO
3-8 ppm for S02, N02, and 03.
At Level 1, concentrations less than those for the precision checks, the
probability limits for accuracy are, as expected, wider than for precision.
(See Table 7.)
COMPARISON OF NATIONAL LIMIT VALUES AND 50-PERCENTILE VALUES
With reference again to Table 12, in all cases the spreads (ranges) of
the national values for both precision and accuracy are greater than for the
corresponding 50-percentile values. For the continuous S02 method, the
ranges for the national values were wider than for the 50-percentile values.
There are two reasons why the spreads of the national values are much wider
than the 50-percentile values. First, the presence of significant differences
between quarters within reporting organizations, between reporting organiza-
tions within States, between States within regions, and between regions cause
some increase in the total variability over and above that which would be
obtained from only random variability. Second, the national values are unduly
influenced by extreme or outlier values. If there were no significant dif-
ferences and no outlier values, the 50-percentile values should closely agree
with the national values.
An evaluation of the shape of the distributions does in fact show that
the distributions are not normal due to an excessive number of extreme values
(i.e., values in the tails of the distribution).
All of the distributions of the upper and lower probability limits are
generally symmetric about zero. The only exception is for the S02 method.
For prior years the accuracy audits for the manual method and the precision
and accuracy audits for the continuous methods were biased negatively. For
1986, the limits for the 50 percentile values for the continuous S02 method
continue to indicate a slight negative bias for both the precision and the
accuracy data. A possible explanation for the negative bias for precision is
that the relatively low concentrations of S02 (0.08 - 0.10 ppm) in cylinders
specially prepared for precision checks may degrade after preparation. These
biases for S02 were observed in prior years seem to be consistent in magnitude
and direction. These consistent biases should be investigated and corrected,
if possible.
36
-------
Based on the percentiles of Table 8, quarterly probability limit values
which exceed those listed in Table 13 should be considered excessive or
outlier values and should initiate immediate investigation to determine and,
hopefully, correct the cause of such excessive values. The values given in
Table 13 are slightly tighter in some cases than the corresponding values
given in the report for the 1985 data.
TABLE 13. VALUES OF QUARTERLY PROBABILITY LIMITS CONSIDERED
AS EXCESSIVE BASED ON 1986 DATA
Manual methods
TSP
Pb
Continuous methods
CO
03
N02
S02
Precision limits
± 23
± 35
± 15
± 17
± 26
± 20
Level
± 18
± 25
± 24
± 38
± 28
Accuracy
1 Level
± 14
± 15
± 15
± 18
± 21
± 22
limits
2 Level 3
±14
± 17
± 19
± 21
37
-------
SECTION 6
COMPARISON OF RESULTS FROM THE PARS AND THE PA AUDIT PROGRAM
A general comparison between the accuracy data of the PARS program and
the Performance Audit (PA) data is included in this report. The Performance
Audit data are the results of an independent -check conducted by the Quality
Assurance Division (QAD) of the EMSL under the National Performance Audit
Program (NPAP).
In the NPAP, specially prepared audit samples or devices are sent from
QAD to the participating ambient air monitoring agencies. The samples or
devices are carefully and accurately assessed by EMSL utilizing NBS Stan-
dard Reference Materials (SRM's) or standards. The monitoring agencies
analyze or measure the samples or devices as unknowns or blinds and report
their results to QAD for evaluation. Audit programs are conducted for the
following pollutant measurements, using the materials indicated:
Portion of measure-
Measurement Audit materials ment system audited
(manual) Freeze-dried sodium sulfite Chemical analysis
N02 (manual) Aqueous sodium nitrite Chemical analysis
Pb Filter strip with lead nitrate Chemical analysis
TSP Reference flow device Flow
CO Cylinders containing CO gas Sampling and analysis
S02 Cylinder containing S02 gas Sampling and analysis
The audit materials or devices are prepared at three to six different
concentrations or flow levels. Separate reports on the evaluation of the
PA data are published by EMSL.7'11 Also, other reports12*13 have dealt
with the use of PA and PARS data.
As indicated above, the NPAP does not yet include an audit for the
ozone or continuous N02 methods. Therefore, no comparisons of the NPAP or
PA data with the PARS data are possible for those pollutants.
Since precision assessments are not made in the PA program, only
accuracy can be compared across the PARS and the PA program?. For the pur-
pose of this report, the results from PARS and the PA system are compared
at approximately the same levels by matching laboratories and reporting
organizations. (See Appendix E for a more detailed discussion of the prob-
lems involved in comparing the PARS and PA data.) Since the PARS data are
presented with outliers, if any, the same approach was taken with the audit
data. Knowledge of the past audit data reports, however, indicates that the
presence of outliers may make a significant difference in the audit results
for some agencies.
38
-------
Comparisons of the national values of the probability limits (Table 14)
exhibit fairly good agreement between the results of the two programs.
Variations due to many sources of error for both data sets are averaged to-
gether to obtain ttienatio'nal values, thereby masking any correlations which
may have existed for the (results of individual agencies. There is consid-
erable variation between the results of the two programs when comparisons are
made on Regional and reporting organization bases. Lack of better agreement
results from several factors. First, the inclusion of outlier values in the
PA and PARS data appears to have introduced some excessive distortion of
general trends. Second, the concentration levels for the two systems do not
coincide Exactly at each of the audit levels. Third, the PA data are the
results of independent external audits^ while the PARS accuracy data are
based on the results of independent internal audits. The expected effects of
the last-mentioned factor would cause the spread of the limits for the PA to
be wider than that for the PARS. Examination of the results (see Table 14)
confirm these expectations. The PA data for 1986 are generally better than
the corresponding data for 1985.
TABLE 14. SUMMARY COMPARISON OF EMSL PERFORMANCE AUDITS
(PA) vs. PARS ACCURACY AUDIT DATA FOR 1986
Pollutant
CO
PA
PARS
S02
PA
PARS
TSP
PA
PARS
Pb
PA
PARS
Audits
501
695
704
961
3350
4357
592
901
National values
95% probability limits (%).
Level 1
Lower
-13
-14
-10
-15
-16
-14
Upper
11
14
16
14
13
10
Level 2
Lower Upper
- 6 6
- 8' 8
- 9 14
-13 13
- 7 9
- 7 7
-18 13
-13 10
Level 3
Lower Upper
- 6 6
- 7 7
- 9 12
-13 11
Comparisons of the 95 percent probability limits for the PA and the PARS
results by Region are shown in Figures 9a through d for selected concentra-
tion levels. The figures show considerable variation among Regions.
CQ_. (Figure 9a)
The width of the PARS probability limits for level 2 exceed those for PA
for nine of the ten Regions. For previous years, the PA limits have generally
been wider than the PARS limits.
39
-------
TSP. (Figure 9b)
For five Regions, the width of the probability limits for PARS is less
than for PA. This may be explained by the fact that within each reporting
organization the flow rate checks are not as completely independent from
their internal standards as are the PA audits. Regions I and X have more
variability of PA audit data than other Regions.
Pb_. (Figure 9c)
There is considerable variation in the results from Region to Region.
However, for most Regions, the PARS variability is considerably less than
for PA. This may be explained by the fact that the local independently-
prepared standards for PARS have close traceability to the materials used
for calibration, whereas the standards for PA, since they are prepared at
EMSL/RTP, are more completely independent.
Regions I and III results have much more variability for PA than the
other Regions, indicating a need for investigations to determine the major
causes and appropriate corrective actions.
SO? (Continuous). (Figure 9d)
Figure 9d shows the available comparisons of the PA and PARS data for
the continuous S02 method.
40
-------
PROBABILITY LIMITS, percent
N>
en
in o en o
» » ls» is> -»
en o en o en o?
II II 1
3
CO
CT
O
O
3
o
o
-h
-o
Di
a.
33
en
n
o
oo
I
NJ
1 --
--- 1
1 ---
1 --
H
-H
I
| --
I III II I I I
CO
CD
CO
O
"O
T3
Q>
O
O>
Q.,
"D
20
cn
O
O
co
2.
N)
| I I I I I I I I I I I M
1
1-
5
c/> r
1
._J
,1
1
H
1
1
r>
°
1
-------
-------
PROBABILITY LIMITS, percent
CO
CO
c
CO
Q.
o
o
T3
Ql
I
O
Q.
-o
cn
h
O
O
O
5'
c
o
u
*»
2 «"
o
O
g m
j
oo
H
CO
1-
1 1
1 1
. '
. ., ,i
._ -4 ' -
H^
1 ~
1 ~~
H _
HM»
O
ro
1 1 1 1
-------
In
limits.
eight of the ten regions, the PARS limits are wider than the PA
No explanation can be given to these differences.
National Comparison
Figure 10 shows the available PA and PARS comparisons on a national
basis for all levels for each pollutant method. For the CO and S02 methods,
the PARS limits are slightly wider than for PA which was not the case for
CY-85 results. For Pb and TSP the PA limits are wider than for PARS, the
same as for CY-85 results.
Missing PA and PARS Comparisons.
Comparison of the results from PARS and PA are, of course, possible only
when the data are available from both systems for paired reporting organiza-
tion-laboratory combinations. Paired data were not available for comparison.
Of these, data was not available because of missing data from the PARS for .23
comparisons:
Reporting Laboratory
Region State organization number
I ME 20107 501012
ME 20112 501002
II VI 55001 310001
IV FL 10018 423002
TN 44005 417001
VI NM 32002 430001
TX 45003 433001
VII IA 16001 436001
MO 26003 438003
NE 28003 435002
435003
IX AZ 03100 347001
AZ 03200 447001
HI 12120 348001
NV 29100 346001
346002
NV 29200 446001
NV 29300 446002
GU 54100 349001
Pollutant(s)
TSP
TSP
TSP,*** S02
SO 2**
CO*
SO 2
SO 2
Pb
Pb***
S02*
CO
S02,* Pb
SO 2*
CO***
CO,* TSP*
Pb,** CO
S02,** TSP,***
Pb
*Also missing for
**Also missing for
***Also missing for
1985.
1984 and 1985.
1983, 1984 and 1985.
44
-------
au
40
_ 30
* 20
O)
j 10
t-
5 0
CO u
BO
O
£-10
-20
.in
' T 'C°
-JPA JPARS
I x
_ _
T
~ i ~
1 1 1
CONTINUOUS METHODS
MANUAL METHODS
T
T
* I
T
S02
TT-
I
I
2 3
LEVEL
1 I
* = IGNORING 1AI
Dl
TSP
T VALUE
' FOR NH R.O. 30001 AND
2 AUDIT VALUES
VA R.O. 48001
.,.,-,
T
1 *
1
1
1
1 *
A
.T
i
1
1
, J.
I I
FOR
^ __
T
f
ll
m « w
1
Figure 10. Comparison of PA and PARS, national values, 1986.
45
-------
Lack of laboratory participation in the National Performance Audit Pro-
gram in 1986 is the reason there is no paired data available for 89 cases
compared to 135 for 1985. In these cases, the laboratories (reporting organ-
ization) did not comply with the requirements of the federal regulations. In
some of these cases, the laboratory requested the audit samples but did not
report any results. A listing of missing PA audit data follows:
Region
II
III
IV
Reporting
State organization
NY 33001
NY 33001
NY 33001
NY 33001
NY 33001
NY 33001
NY 33001
NY 33001
NY 33001
NY 33001
PR 40001
DC 09001
MD 21003
WV 50001
WV 50002
AL 01011
AL 01013
FL 10001
FL 10003
FL 10004
FL 10011
FL 10014
FL 10018
NC 34001
NC 34001
NC 34001
TN 44003
TN 44003
TN 44005
IL 14003
IN 15001
IN 15001
IN 15005
IN 15008
IN 15010
IN 15100
MI 23002
MN 24001
Laboratory
number
307001
307002
307003
307004
307005
307006
307007
307008
307009
307010
309001
312100
412004
314001
314002
319001
419003
323005
323004
323008
423003
423005
423002
318001
318004
418005
317001
417003
417001
428003
329001
429009
429005
429004
529002
329002
426001
324001
Pollutant
CO, Pb
CO, Pb
CO
CO
CO
CO
CO
CO
CO
CO
CO
TSP*
SO 2**
SO 2
CO***
Pb
TSP***
TSP
TSP
TSP,* S02
CO,* Pb,* S02
TSP*
CO,** Pb,*
S02,** TSP
SO 2
SO 2
S02
CO**
CO
CO, TSP*
SO 2
CO
CO
SO 2**
Pb**
TSP***
Pb,** S02
Pb,* S02, TSP
SO 2
(continued)
46
-------
Region
V
VI
VII
VIII
IX
Reporting
State organization
OH 36001
OH 36002
OH 36004
OH 36006
OH 36008
OH 36009
OH 36010
OH 36012
LA 19001
NM 32002
OK 37102
TX 45002
IA 16001
IA 16002
IA 16003
MO 26003
NE 28002
NE 28003
NE 28003
CO 06001
MT 27002
MT 27003
MT 27004
AZ 03200
CA 05036
CA 05061
05061
HI 12120
12120
NV 29100
NV 29200
NV 29300
GU 54100
AK 02020
AK 02020
ID 13001
ID 13001
OR 38001
-Laboratory
number
327001
327003
327007
427001
427003
427004
427005
427007
334001
430001
431001
433002
436001
436002
336001
438003
435001
435002
435003
344001
439001
439002
439003
447001
445003
445002
445017
348001
348002
346001
446001
446002
349001
451001
451002
354001
354002
353001
Pollutant
CO,* TSP*
S02
S02
Pb***
Pb***
Pb,*** S02
pb***
TSP
CO*
CO, Pb, TSP*
CO,*** Pb,*
TSP
CO,* TSP
Pb
CO*
TSP
Pb,* S02
TSP
SO 2
SO 2
CO, Pb,***
TSP***
TSP
TSP
TSP
SO 2**
CO, S02, TSP
CO, S02,** TSP
CO, S02,** TSP
CO,*** SO 2*
CO,***, SO 2*
TSP*
CO,* TSP*
Pb**
S02,** TSP***
CO
CO
CO, S02,** TSP
CO, S02,** TSP
CO, S02
*Also missing for 1985.
**Also missing for 1984 and 1985.
***Also missing for 1983, 1984 and
1985,
47
-------
In 13 cases, data were unavailable from both PARS and PA:
Reporting Laboratory
Region State Organization number
IV FL 10018 423002
TN 44005 417001
VII IA 16001 436001
MO 26003 438003
NE 28003 435002
NE 28003 435003
IX AZ 03200 447001
HI 12120 348001
NV 29200 446001
NV 29300 446002
GU 54100 349001
Pollutant
SO 2**
CO*
Pb
Pb***
SO 2*
S02*
S02,* Pb
S02*
CO,* TSP*
Pb**
S02,** TSP***
*Also missing for
**Also missing for
***Also missing for
1985.
1984 and 1985.
1983, 1984 and 1985.
48
-------
SECTION 7
CONCLUSIONS AND RECOMMENDATIONS
The results of PARS data for 1986 indicate some general improvement over
the data for previous years. However, considerable differences exist among
Regions and individual reporting organizations for most measurement methods.
Investigations should be made by the Regions and the States to determine the
causes of these significant differences.
The PA data for TSP and Pb show more variability than for PARS. These
differences are presumably due to the fact that the external PA accuracy au-
dits are more completely independent than the internal PARS accuracy audits.
These differences have been consistent for past years.
Further improvement in the data quality assessments, which are mea-
sures of the monitoring data quality, can be achieved only through contin-
uing efforts of State and local agency personnel involved (first-hand) with
the operation and quality control of their measurement systems. Regional
QA Coordinators can also assist through their review of the operations and
quality control practices across the States in their Regions.
Each Regional QA Coordinator should evaluate the PARS data from all
the reporting organizations within his Region to identify those organiza-
tions having excessively large variations of probability limits. Investi-
gation should be made to determine the causes and correct them to preclude
future excessive deviations. Similarly, Regional QA Coordinators should
review the operations of the reporting organizations having significantly
better precision and accuracy results in order to identify specific proce-
dures which should be uniformly used throughout the Region and the Nation
to further improve the reliability of the monitoring data in the National
Aerometric Data Base.
49
-------
REFERENCES
1. Code of Federal Regulations, Title 40, Part 58, "Ambient Air Quality and
Surveillance, Appendix A, Quality Assurance Requirements for SLAMS."
2. Rhodes, R.C. "Guideline on the Meaning and Use of Precision and Accu-
racy Data Required by 40 CFR Part b8, Appendices A and B." U.S. En-
vironmental Protection Agency Report, EPA 450/4-84-006. Research Tri-
angle Park, NC 27711. June 1983.
3. Evans, E.G., R.C. Rhodes, W.J. Mitchell and J.C. Puzak. "Summary of
Precision and Acuracy Assessments for the State and Local Air Monitor-
ing Networks, 1982." U.S. Environmental Protection Agency Report, EPA-
600/4-85-031. Research Triangle Park, NC 27711. April 1985.
4. Rhodes, R.C. and E.G. Evans. "Precision and Accuracy Assessments for
State and Local Air Monitoring Networks, 1983." U.S. Environmental Pro-
tection Agency Report, EPA-600/4-86-012. Research Triangle Park, NC
27711. February 1986.
5. Rhodes, R.C. and E.G. Evans. "Precision and Accuracy Assessments for
State and Local Air Monitoring Networks, 1984." U.S. Environmental Pro-
tection Agency Report, EPA-600/4-86-031. Research Triangle Park, NC
27711. August 1986.
6. Rhodes, R.C. and E.G. Evans. "Precision and Accuracy Assessments for
State and Local Air Monitoring Networks, 1985." U.S. Environmental
Protection Agency Report, EPA-600/4-87-003. Research Triangle Park, NC
27711. January 1987.
7. Rhodes, R.C., B.I. Bennett and J.C. Puzak. "EPA's National Performance
Audit Program for Ambient Air Pollution Measurements." In Proceedings
of the 75th Annual Meeting of the Air Pollution Control Association,
New Orleans, LA, June 1982. Presentation 82-23.
8. Lampe, R.L., B.F. Parr, G. Pratt, O.L. Dowler and W.J. Mitchell. "Na-
tional Performance Audit Program: Ambient Air Audits of Analytical
Proficiency-1983." U.S. Environmental Protection Agency Report, EPA-
600/4-84-077. Research Triangle Park, NC 27711. October 1984.
9. Parr, B.F., R.L. Lampe, G. Pratt, O.L. Dowler and W.J. Mitchell. "Na-
tional Performance Audit Program: Ambient Air Audits of Analytical Pro-
ficiency, 1984." U.S. Environmental Protection Agency Report, EPA-600/
4-86-013. Research Triangle Park, NC 27711. February 1986.
50
-------
10. Parr, B.F., R.L. Lampe, G. Pratt, O.L. Dowler and W.J. Mitchell. "Na-
tional Performance Audit Program: Ambient Air Audits of Analytical Pro-
ficiency, 1985." U.S. Environmental Protection Agency Report, EPA-600/
4-87-002. Research Triangle Park, NC 27711. January 1987.
11. Parr, B.F., R.L. Lampe, G. Pratt, O.L. Dowler, and W.J. Mitchell. "Na-
tional Performance Audit Program Ambient Air Audits of Analytical Pro-
ficiency, 1986." U.S. Environmental Protection Agency Report, EPA-600/
4-87-xxx. Research Triangle Park, NC 27711. November 1987.
12. Rhodes, R.C., W.J. Mitchell, J.C. Puzak and E.G. Evans. "Comparison of
Precision and Accuracy Estimates from State and Local Agency Air Monitor-
ing Stations with Results of EPA's National Performance Audit Program."
Journal of Testing and Evaluation, JTEVA, Vol. 13, No. 5, September
1985, p. 374-378.
13. Thrall, A.D. and C.S. Burton. "Special Report, Issues Concerning the
Use of Precision and Accuracy Data." U.S. Environmental Protection
Agency Report, EPA-450/4-84-006. Research Triangle Park, North Carolina
27711. February 1984.
51
-------
-------
APPENDIX A
GLOSSARY
State and Local Air Monitoring Stations (SLAMS) monitoring stations se-
lected by the states and included in the State Implementation Plans. The
stations and the plans are approved by the Regional Administrator. The
purposes of the monitoring are to determine compliance to the National Ambi-
ent Air Quality Standards (NAAQS) and to determine background levels of the
criteria pollutants.
National Air Monitoring Sites (NAMS) -- a subset of the SLAMS, selected by
the statesTncollaboration with the Regional Offices and approved by the
Administrator. The purpose of the sites is to monitor in the areas where
pollution concentration and population exposure are expected to be highest in
terms of the NAAQS. Although, in actuality the NAMS are a subset of SLAMS,
the NAMS sites and the non-NAMS SLAMS sites are often referred to as two
separate groups, the NAMS and SLAMS sites, respectively.
Reporting Organization a state, or subordinate organization within the
state, that is responsible for a set of SLAMS stations, monitoring for the
same pollutant and for which PARS data can be logically pooled (statistically
combined). It is important to emphasize that a reporting organization is
pollutant- and site-specific and is responsible for the sampling, calibration,
analysis, data quality assessment, and reporting of the monitoring data for
the specific pollutant. It is possible that a particular SLAMS station may
belong to two different reporting organizations, but the likelihood of this
occurring is small.
Precision (Continuous Analyzers) -- a measure of repeatability obtained from
repeated measurements of a standard concentration in a gas cylinder and the
values indicated by the analyzer. For S02, N02, and 03 analyzers, the gas con-
centration used for the precision check must be between 0.08 and 0.10 ppm and
for CO it must be between 8 and 10 ppm. The data from all biweekly analyzer
checks for a given pollutant are combined, and 95% probability limit values
are reported to EPA each quarter by each reporting organization. For this
report, the quarterly values for 1986 were combined, and overall 95% proba-
bility limits were calculated for each reporting organization, for each
Region, and for the nation, as described in Appendix B.
Precision (Manual Methods) a measure of repeatability for TSP, N02, and
SO? manual methods (bubblers) determined by operating collocated samplers at
selected sites. At each collocated site one sampler is designated as the
"actual" sampler and the other as the "check" sampler, and the difference
between the two samplers provides the precision estimate. For Pb, precision
A-l
-------
estimates are obtained by analyzing duplicate strips from a high volume
filter sample collected at a site where high Pb concentrations exist. These
precision checks are made from samples, usually taken every 6th day, and are
reported quarterly. The data from the manual methods were calculated in a
similar manner as the continuous analyzers.
Accuracy (Continuous Analyzers) the agreement between an analyzer mea-
surement and a known audit standard concentration. Accuracy estimates are
obtained at least once per year for each analyzer by introducing blind audit
standards into the analyzer. The audit samples must span at least three
concentration levels and, whenever possible, must be traceable to NBS or
other authoritative reference. At least 25% of the analyzers in each report-
ing organization must be audited each quarter. The percentage difference for
each audit concentration is determined, and the average for all analyzers
checked within that quarter is calculated for each level. The standard devi-
iation for each level is then used to calculate the 95% probability limits
for the reporting organization, which in turn are submitted quarterly to EPA.
These quarterly values were combined to determine the annual values presented
here. They were calculated in the same manner as described earlier for
precision.
Accuracy (Manual Methods) the agreement between an observed or measured
value and a known or reference value. For N02 and S02 manual methods, the
accuracy of the analytical portion of the method is assessed at three levels by
the analysis of audit materials of known characteristics. For Pb, the accu-
racy of the analytical portion of the method is assessed at two levels. For
TSP, the flow rate (or air volume) portion of the method is assessed at the
nominal flow rate.
Completeness -- the number of the precision and accuracy checks reported as
compared to the number that should have been reported if all checks had been
done ,in accordance with the regulations. This value, expressed as a per-"
centage, is not corrected for instances where equipment failure prevented
conducting the check, or for periods when monitoring data were invalidated.
National Performance Air Audit Program (NPAP) an external performance au-
dit program conducted by EPA on State and local agency organizatons. Organi-
zations operating SLAMS stations are required to participate in this program
directed by the Environmental Monitoring Systems Laboratory (EMSL) of the EPA
at Research Triangle Park, NC. In this program, blind audit materials pre-
pared by EMSL are sent to participating laboratories. The laboratories ana-
lyze the samples and return the results to EMSL. Shortly after the audit is
completed each participant receives a report that compares his performance to
that of all other participants. The audit materials for the manual methods for
S02, N02 and Pb are used to evaluate the accuracy of only the analytical lab-
oratory portion of the method, and are as follows:
A-2
-------
Method Audit Materials
Manual S02 Freeze-dried
Manual N02 NaN02 solution
Pb Filter strips spiked with Pb $04
(Note: Because the manual S02 and manual N02 methods are being replaced by
continuous methods, these performance audits have been discontinued.)
The reference flow device used in the TSP sampler audit evaluates only the
accuracy of the flow calibration. However, the CO and S02 continuous analyzer
audits evaluate the entire measurement system. As explained above, the exter-
nal NPAP audits are conducted in essentially the same manner as the internal
audits (accuracy checks) for the PARS program. The audits for the Pb method
are conducted semi -annually and those for flow (TSP), and continuous CO and
S02 monitors are conducted at least once per year.
95-Percent Probability Limits probability limits are used in the reporting
of precision and accuracy data to measure the expected spread or variability
of the data from a particular population a reporting organization, a state,
a region, or the nation. These expected limits are expressed simply as a
mean plus or minus a constant (1.96) times the standard deviation as follows:
L = 7 ± ks (1)
where: L_ = probability limits (upper limit, LUf lower limit, L|J
x = mean value
k = 1.96, a constant
s = standard deviation
Under the assumptions of (a) an underlying normal population, (b) the mean x,
being the estimate of the true mean, y, of the underlying population, and (c)
the standard deviation, s, being the estimate of the true standard deviation,
o, of the underlying distribution, then x ± 1.96s represents the expected
limits which should include 95 percent of a]_l the individual measurement of
the population. Under the assumption given, x ± 1.96s limits are the expected
95 percent probability limits, regardless of the sample size.
The requirement for the computation of "probability" limits (rather than
confidence limits) is to provide the State and local agencies with limits
which will be of practical meaning and usefulness for internal control appli-
cations without involving overly complicated and sophisticated statistics.
The selection of the 95 percent level was made because even for non-statisti-
cians, the chance or probability of obtaining one value out of twenty exceed-
ing the limits has practical meaning.
A-3
-------
-------
APPENDIX B
FORMULAS FOR COMBINING PROBABILITY LIMITS
Section 5.2, Annual Reports,,of Appendix A of 40 CFR Part 58 required
that simple unweighted arithmetic averages of the probability limits for
precision and accuracy from the four quarterly periods of the calendar
year be reported with the annual SLAMS report. The simple unweighted
arithmetic averages were specified to simplify the calculations for the
states. Such limits would be essentially correct if only random variations
occurred between quarters within a reporting organization and between re-
porting organizations within a State, i.e., if no statistically significant
differences occurred between quarters within reporting organizations or be-
tween reporting organizations within States. However, experience has shown
that significant differences do occur. Because of this fact, it is most
correct to combine the data across quarters and across reporting organiza-
tions within States (and also across States within regions and across re-
gions within the nation) in the manner described below. These formulas
determine the yearly probability limits for the reporting organization
which would have been computed from all the individual percent difference
values, d-j, obtained during the year. To accomplish this, from each quar-
terly pair of probability limits, the average, 0-,-, and standard deviation,
S-j, are back-calculated:
LL + UL
D. = (1)
1 2
UL - LL
S. (2)
i 2(1.96)
where LL = lower probability limit
UL = upper probability limit
Except for the effect of the round-off of the reported probability limits
to integer values, the above equations determine the original D and Sa val-
ues used by the reporting organizations to compute the originally reported
limits.
Yearly average, D, and standard deviation, Sa values are computed from
the quarterly values as follows:
B-l
-------
D =
where nn = the number of individual percent difference,
quarter
-j
l(nrl)sf
(DM) -
(3)
, values for each
(4)
The appropriate yearly probability limits for the reporting organiza-
tion are computed using the formulas:
UL = D + 1.96 Sa
LL = D - 1.96 Sa
(5)
(6)
NOTE: The same formulas are used for combining yearly reporting organiza-
tion limits into State limits, State limits into Region limits, and
Region limits into National limits.
Example: Suppose that the lower and upper 95% probability limits for CO
for precision for the four quarters of a year are:
Quarter
1
2
3
4
Number of
Precision Checks
10
9
13
7
Lower
Probability
Limit
-8
-5
-6
-12
Upper
Probability
Limit
+6
+9
+4
+11
For Quarter 1:
LL + UL -8+6
2
UL - LL
= -1
1 2(1.96) 2(1.96)
= 3.6
by equation (1)
by equation (2)
Similar computations for the other quarters, give values in the follow-
ing table.
B-2
-------
Quarter _n_ TT S D~ - D
1 10 -1 3.6 -0.78
2 9+2 3.6 2.22
3 13 -1 2.6 -0.78
4 7 -0.5 5.9 -0.28
"37
Then
Q _ by equation (3)
10(-1) + 9(2) + 13(-1) + 7(-0.5)
39
-8.5
= -0.22
39
Krii-1) S? + ^flL-D)2
s = / 1 1 ] ] by equation (4)
a
9(3.6)2+8(3.6)2+12(2.6)2+6(5.9)2+10(-0.78)2+9(2.22)2+13(-0.78)2+7(-0.28)2
39 - 1
510.30 + 58.90
38
= \/14.98 =3.87
The upper and lower 95% probability limits are then computed as:
UL = 5 + 1.96 Sa by equation (5)
= -0.22 + 1.96(3.87)
= 7.37 or 7 rounded off to nearest integer
B-3
-------
LL = D - 1.96 Sa by equation (6)
= -0.22 - 1.96(3.87)
= -7.81 or -8 rounded off to nearest integer
In this particular example, the results by the weighted combined form-
ulas are very close to the simple unweighted arithmetic averages. However,
in many cases the weighted combined formulas result in wider limits than
the simple unweighted arithmetic averages and more correctly reflect the to-
tal variability exhibited by the individual percent differences.
Alternate Method of Computation
An alternate method which eliminates the need to compute U"-j - D, the
differences between the quarterly averages and the weighted annual average,
follows.
1. Compute D^i and Si for each quarter according to equations (1) and
(2) as above.
2. Compute for each quarter.
Ed = ni Uj (7)
3. Compute for each quarter.
(Ed)2
Ed2 = (ni - 1) S^ + (8)
"1
4. Compute:
Eni the sum of n for all quarters (9)
EEd = the sum of Ed for all quarters (10)
EEd2 = the sum of Ed2 for all quarters (11)
5. Compute 0 according to equation (3) above, or
EEd
D =
En
B-4
-------
6. Compute Sa:
/EEd2 -
En
S, - / (13)
(En) - 1
7. Then compute the probability limits, UL and LL, according to equa-
tions (5) and (6).
Example
The data for the previous example on page B-2 will be used.
Lower Upper
Number of Probability Probability
Quarter Precision Checks Limit Limit
1 10 -8+6
29 -5+9
3 13 -6 +4
4 7 -12 +11
1. TL and S^ are computed as before. Compute Ed and Ed2 by equations
(7) and (8) respectively.
Quarter
Ed Ed2
1
2
3
4
10
9
13
7
-1
+2
-1
-0.5
3.6
3.6
2.6
5.9
-10
+18
-13
-3.5
-"8.5
126.64
139.68
94.12
210.61
For quarter 1:
(Ed)2
Ed2 = (n - 1) S2 + (8)
n
(-10)2
= (9)(3.6)2 +
10
= 116.64 + 10
= 126.64
B-5
-------
2. By equation (12):
Jt *t j Q C
_ Ad -O.D
= __ = -0.22 the same as before
in 39
3. By equation (13):
(13)
- 1
(-8.5)2
571.05
39
S ,=
39 - 1
571.05 - 1.85
38
= 3.87 the same as before
4. The probability limits are then calculated as before using equa-
tions (5) and (6).
A Second Example
The following example more clearly shows computationally and graphi-
cally that the arithmetic averages of the quarterly upper and lower proba-
bility limits do not correctly reflect the total variability when signifi-
cant differences occur between quarters. Suppose the following individual
percent differences have been obtained for the precision checks for a con-
tinuous instrument during the past year.
Quarter Individual Percent Differences
1 -12, -9, -5, -5, -1, 2
2 1, 4.5, 5, 5, 5.5, 9
3 -6, 0, 5, 5, 10, 16
4 -17, -14, -10, -10, -6, -3
From the previous formulas, the following U, S, and probability limits for
each quarter are calculated.
B-6
-------
Quarter IT _JL_ n LL >.- UL
1
2
3
4
-5
5
5
-10
5.10
2.55
7.64
5.10
6
6
6
6
-15
0
-10
-20
-11.25
5
10
20
0
O i
'5
(-11) (+9)
As Indicated above, the simple arithmetic averages of the lower and
upper probability limits are -11.25 and 8.75, or -11 and 9 when rounded-
off.
The calculations of the annual probability limits by equations (3)
through (6) are shown below.
EnD -30
D = = = -1.25 (3)
En 24
Sa . . -l-J (4)
k I * V d
5(5.10)2+5(2.55)2+5(7.64)2+5(5.10)2+6(-3.75)2+6(6.25)2+6(6.25)2+6(-8.75)2
24 - 1
11596.961
= 8.333
23
UL = D + 1.96 S (5)
= -1.25 + 1.96 (8.333)
= 15.083 or (15)
LL = D - 1.96 S i (6)
= -1.25 - 1.96 (8.333)
= -17.583 or (-18)
B-7
-------
The Individual percent differences, the quarterly probability limits,
the arithmetic annual probability limits and the combined annual probabil-
ity limits are shown graphically on the following figure.
1
cc 2
UJ
1-
ee
o 3
LIMITS:
ARITHMETIC AVERAGE
COMBINED DATA
1 1 1 1 r- r i
> 1 1 1 1 1 1
i ^ || | | i
L|« £ |u
< i i i i i
1 1 1 1 1 1 T
L| U
ii i i i i i
1 ( 1- I 1 I i
' ' i I 1 1 1
1 r~ 1 I 1 1 I
L| K
i i i i i i i
1 T 1 1 1 1 1
L|« uU
II 1 1 1 -1 1
20 -15 -10 -5 05 10
PERCENT DIFFERENCE
15 20
It 1s clear from the above figure that the combined limits more cor-
rectly represent the total spread of the Individual percent differences
during the year. In fact, the calculated values of the average and stan-
dard deviation for all 24 of the Individual percent differences are -1.25
and 8.333, respectively, which are in exact agreement with the prev-ious
calculations as they must be because of the exact equality of the mathe-
matical formulas involved.
B-8
-------
APPENDIX C
LISTING OF REPORTING ORGANIZATIONS
Region
01
01
01
01
01
01
01
01
01
01
01
01
01
01
01
01
02
02
02
02
02
03
03
03
03
03
03
03
03
03
03
03
03
03
03
03
03
04
04
04
04
04
No.
07
20
20
20
20
20
20
20
20
20
20
20
22
30
41
47
31
33
40
55
55
08
09
21
21
21
21
21
39
39
39
48
48
48
48
50
50
01
01
01
01
01
State
Name
CONNECTICUT
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MASSACHUSETTS
NEW HAMPSHIRE
RHODE ISLAND
VERMONT
NEW JERSEY
NEW YORK
PUERTO RICO
VIRGIN ISLANDS
VIRGIN ISLANDS
DELAWARE
DISTRICT OF
COLUMBIA
MARYLAND
MARYLAND
MARYLAND
MARYLAND
MARYLAND
PENNSYLVANIA
PENNSYLVANIA
PENNSYLVANIA
VIRGINIA
VIRGINIA
VIRGINIA
VIRGINIA
WEST VIRGINIA
WEST VIRGINIA
ALABAMA
ALABAMA
ALABAMA
ALABAMA
ALABAMA
No.
001
001
101
102
103
104
106
107
108
109
110
112
001
001
001
001
001
001
001
001
017
001
001
001
002
003
005
006
001
002
003
001
002
003
006
001
002
on
012
013
014
015
Reporting Organization
Name
AIR MONIT. SEC. DEPT. OF ENV. PROTECT.
BUREAU OF A.Q.C. DEPT. OF ENV. PROTECT.
S.D. WARREN CO., WESTBROOK
S.D. WARREN CO., HINCKLEY
INTERNATIONAL PAPER CO., JAY
BOISE CASCADE CO., RUMFORD
DRAGON PRODUCTS, THOMASTON
SCOTT PAPER CO., WINSLOW
CHAMPION INTERNATIONAL CORP., BUCKSPORT
LINCOLN PULP AND PAPER CO., LINCOLN
GREAT NORTHERN PAPER CO., MILLINOCKE
GEORGIA PACIFIC CO., WOODLAND
DIV. OF AQC. DEPT. OF ENV. QUAL. ENG.
AIR RESOURCES AGENCY
DIV. OF A. HAZ. MAT. DEPT. OF ENV. MANAGE
AIR & SOLID WASTE PROGRAMS
DEPT. OF ENV. PROT., DIV. OF ENV. QUAL.
DEPT. OF ENV. CONSERV., DIV. OF AIR
ENVIRONMENTAL QUALITY BOARD
DEPT. OF CONS. AND CULTURAL AFFAIRS
MARTIN MARIETTA
STATE OF DELAWARE, DNR & EC
WASHINGTON, DC DC & RA
STATE OF MARYLAND
ALLEGANY COUNTY
ANNE ARUNDEL COUNTY
BALTIMORE COUNTY
PRINCE GEORGE'S COUNTY
PENNSYLVANIA DER
ALLEGHENY CO. BAPC
PHILADELPHIA AMS
VIRGINIA STATE AIR POLL. CONTROL BOARD
CITY OF ALEXANDRIA
FAIRFAX COUNTY
TENNESSEE VALLEY AUTHORITY - VA
STATE OF WEST VIRGINIA
WVA NORTHERN PANHANDLE REGIONAL OFFICE
ALABAMA DEPT. OF ENVIRONMENTAL MGT.
AL, JEFFERSON CNTY. BUREAU OF ENV. HLTH.
AL DEPT. OF ENV. MANAGEMENT - MOBILE
AL, HUNTSVILLE AIR POLL. CONTROL DEPT.
AL, TRICOUNTY DIV. OF AIR POLL. CONTROL
C-l
-------
Region
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
05
Ob
05
05
05
05
OS
05
05
05
05
05
05
05
05
No.
01
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
11
18
18
18
25
34
34
34
34
42
44
44
44
44
44
44
14
14
14
15
15
15
15
15
15
15
15
23
23
24
36
State
Name
ALABAMA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
GEORGIA
KENTUCKY
KENTUCKY
KENTUCKY
MISSISSIPPI
NORTH CAROLINA
NORTH CAROLINA
NORTH CAROLINA
NORTH CAROLINA
SOUTH CAROLINA
TENNESSEE
TENNESSEE
TENNESSEE
TENNESSEE
TENNESSEE
TENNESSEE
ILLINOIS
ILLINOIS
ILLINOIS
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
MICHIGAN
MICHIGAN
MINNESOTA
OHIO
No.
016
001
002
003
004
005
006
007
Oil
012
013
014
015
016
017
018
020
022
010
001
002
003
100
001
002
003
004
001
001
002
003
004
005
006
001
002
003
001
002
003
005
008
009
010
100
001
002
001
001
Reporting Organization
Name
TENNESSEE VALLEY AUTHORITY - ALABAMA
FDER, NORTHWEST DISTRICT
FDER, NORTHEAST DISTRICT
FDER, ST. JOHNS RIVER DISTRICT
FDER, SOUTHWEST DISTRICT
FDER, SOUTH FLORIDA DISTRICT
FDER, SOUTHEAST FLORIDA DISTRICT
FUER, NORTHEAST DISTRICT BRANCH OFFICE
FL, JACKSONVILLE BIO-ENV. SERVICES DIV.
FL, HILLSBOROUGH CO., ENV. SERVICES DIV.
FL, PINELAS CO. DEPT. OF ENV. MANAGEMENT
FL, MANATEE COUNTY HEALTH DEPARTMENT
FL, SARASOTA CO. AIR POLL. CONTROL DIV.
FL, PALM BEACH COUNTY HEALTH DEPARTMENT
FL, BROWARD CO. ENV. QUAL. CONTROL BOARD
FL, DADE CO. DEPT OF ENV. RESOURCES MGT.
FL, ORANGE CO. ENV. PROTECTION DEPT.
EVERGLADES NATIONAL PARK
GEORGIA AIR QUAL. EVALUATION SECTION EPD
KENTUCKY DIV. OF AIR POLL. CONTROL
KY, JEFFERSON CO. AIR POLL. CONTROL DIST.
TENNESSEE VALLEY AUTHORITY - KENTUCKY
MISSISSIPPI BUREAU OF POLLUTION CONTROL
NC NATURAL RESOURCES & COMMUNITY DEVEL.
NC, FORSYTH COUNTY ENV. AFFAIRS DEPT.
NC, MECKLENBURG CO. DEPT. OF ENV. HEALTH
NC, WESTERN REGIONAL AIR POLL. CONTROL
SC DEPT. OF HEALTH & ENV. CONTROL
TENNESSEE DIV. OF AIR POLL. CONTROL
TN, MEMPHIS-SHELBY CO. HEALTH DEPARTMENT
METRO HEALTH DEPT. NASHVILLE-DAVIDSON CO.
TN, KNOX COUNTY DEPT. OF AIR POLL. CONTROL
TN, CHATTANOGGA-HAMILTON CO. AIR POLL. CONT
TENNESSEE VALLEY AUTHORITY - TENNESSEE
DIV. OF AIR POLL. CONT., ILLINOIS EPA
CHICAGO DEPT. OF CONSUMER SERVICES
COOK COUNTY DEPT. OF ENVIRONMENTAL CONT.
AIR POLL. CONT. DIV. OF INDIANA STATE
DIV. OF AIR POLL. CONT., EVANSVILLE
ST. JOSEPH COUNTY
AIR POLL. CONT. DIV., VIGO COUNTY
INDIANAPOLIS APC DIVISION
ANDERSON LOCAL AGENCY
PORTER COUNTY HEALTH DEPARTMENT
LAKE COUNTY CONSOLDTD. AQ MONIT. WRK. GRP .
AIR QUAL. DIV., MI DEPT. OF NAT. RES.
AIR POLL. CONT. DIV., WAYNE COUNTY
MINNESOTA POLL. CONT. AGENCY, AIR MO
OHIO EPA, CENTRAL DISTRICT OFFICE
C-2
-------
Region
05
05
05
05
05
05
05
05
05
05
05 ,
05
05
05
05
06
06
06
06
06
06
06
06
06
06
06
06
06
06
06
07
07
07
07
07
07
07
07
07
07
07
07
07
07
08
08
08
08
NO.
36-
36
36
36
36
36
36
36
36
36
36
36
36'
36
51
04
04
19
32
32
37
37
37
45
45
45
45
45
45
45
16
16
16
17
26
26
26
26
26
26
26
28
28
28
06
27
27
27
State
Name
OHIO '"..-
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
WISCONSIN
ARKANSAS
ARKANSAS
LOUISIANA
NEW MEXICO
NEW MEXICO
OKLAHOMA
OKLAHOMA
OKLAHOMA
TEXAS
TEXAS
TEXAS
TEXAS
TEXAS
TEXAS
TEXAS
IOWA
IOWA
IOWA
KANSAS
MISSOURI
MISSOURI
MISSOURI
MISSOURI
MISSOURI
MISSOURI
MISSOURI
NEBRASKA
NEBRASKA
NEBRASKA
COLORADO
MONTANA
MONTANA
MONTANA
No.
/002,
003
004
005
006
007
008
009
010
012
013
014
015
016
001
001
002
001
001
002
101
102
103
001
002
003
004
005
006
007
001
002
003
001
001
002
003
004
005
006
007
001
002
003
001
001
002
003
Reporting Organization
, Name > .
OHIO EPA, NORTHEAST DISTRICT OFFICE
OHIO £PA, NORTHWEST DISTRICT OFFICE
OHIO EPA, SOUTHEAST DISTRICT OFFICE
OHIO EPA, SOUTHWEST DISTRICT OFFICE
AKRON AIR POLLUTION CONTROL
AIR POLL. CONT. DIV., CANTON CITY
SOUTHWESTERN OHIO AIR POLL. AGENCY
CLEVELAND DIV. OF AIR POLL. AGENCY
REGIONAL APC AGENCY, DAYTON
AIR POLL. CONT. DIV. OF LAKE COUNTY
AIR POLL. UNIT, PORTSMOUTH CITY
NORTH OHIO VALLEY AIR AUTHORITY
TOLEDO POLL. CONTROL. AGENCY
MAHONING TRUMBULL AIR POLL. CONTROL
WI. DEPT. OF NAT. RES., AIR MONIT. UNIT
DEPT. OF POLL. CONT. & ECOLOGY CONT. MON
DEPT. OF POLL. CONT. & ECOLOGY
DEPT. OF NATURAL RESOURCES, NEW ORLEANS
ENV. IMPROVEMENT DIV., SANTA FE
CITY OF ALBUQUERQUE ENV. HEALTH DIV.
OK STATE DEPT. OF HEALTH
.OKLAHOMA CITY-CNTY. HEALTH DEPT.
TULSA CITY-CNTY. HEALTH DEPT.
TEXAS AIR CONTROL BOARD
DALLAS ENV. HEALTH & CONSERVATION DEPT.
EL PASO CITY-CNTY. HEALTH DEPT.
FT. WORTH PUBLIC HEALTH DEPT.
GALVESTON COUNTY HEALTH DISTRICT
HOUSTON DEPT. OF PUBLIC HEALTH
SAN ANTONIO METRO. HEALTH DISTRICT
POLK COUNTY PHYSICAL PLANNING
LINN COUNTY HEALTH DEPARTMENT
UNIVERSITY HYGIENIC LABORATORY
STATE OF KANSAS
LABORATORY SERVICES PROGRAM
ST. LOUIS COUNTY
ST. LOUIS CITY
KANSAS CITY
SPRINGFIELD
AMAX LEAD CO. OF MO, BOSS, MO
ST. JOE LEAD CO., HERCULANEUM, MO
STATE OF NEBRASKA
LINCOLN
OMAHA
DEPARTMENT OF HEALTH
MT AIR QUAL. BUREAU, DEPT. OF H&ENV.
YELLOWSTONE CNTY . AIR POLL. CONT. AGY.
GREAT FALLS CITY-CNTY. HEALTH DEPT.
C-3
-------
Region
08
08
08
08
08
09
09
09
09
09
09
09
09
09
09
09
09
09
10
10
10
10
No.
27
3b
43
46
52
03
03
03
05
05
05
05
05
12
29
29
29
54
02
13
38
49
State
Name
MONTANA
NORTH DAKOTA
SOUTH DAKOTA
UTAH
WYOMING
ARIZONA
ARIZONA
ARIZONA
CALIFORNIA
CALIFORNIA
CALIFORNIA
CALIFORNIA
CALIFORNIA
HAWAII
NEVADA
NEVADA
NEVADA
GUAM
ALASKA
IDAHO
OREGON
WASHINGTON
No.
004
001
001
001
001
100
200
300
001
004
036
061
061
120
100
200
300
100
020
001
001
001
Reporting Organization
Name
MISSOULA CITY-CNTY HEALTH DEPT.
STATE DEPARTMENT OF HEALTH
DEPT. OF HEALTH, OIV. OF ENV. HEALTH
STATE BUREAU OF AIR QUALITY
DEPT. OF ENV. QUAL., AIR QUAL. DIV.
ARIZONA DEPT. OF HEALTH«SERVICES
MARICOPA COUNTY
PI MA COUNTY
CALIFORNIA AIR RESOURCES BOARD
BAY AREA AIR QUAL. MANAGEMENT DISTRICT
SAN DIEGO AIR POLL. CONTROL DISTRICT
SOUTH COAST AIR QUAL. MANAGEMENT DIST.
SOUTH COAST AIR QUAL. MANAGEMENT DIST.
STATE OF HAWAII, DEPT. OF HEALTH
NEVADA DIV. OF ENV. PROTECTION
WASHOE COUNTY
CLARK COUNTY
GUAM EPA
DEPT. OF ENVIRONMENTAL CONSERVATION
DEPARTMENT OF HEALTH AND WELFARE
DEPT. OF ENVIRONMENTAL QUALITY
DEPT. OF ECOLOGY
C-4
-------
APPENDIX D
PRECISION AND ACCURACY DATA BY REPORTING ORGANIZATIONS
To reduce printing expenses, the detailed tabulations of the numerical
values for each pollutant for each reporting organization are not included
here, but can be obtained by written request to R.C. Rhodes, EPA, MD-77B,
Research Triangle Park, NC 27711. The format of the tables is the same as
for the previous annual reports. Please indicate in your request the par-
ticular pollutant mesurement system(s) you desire copies for.
D-l
-------
-------
APPENDIX E
PROBLEMS INVOLVED IN THE COMPARISON OF PERFORMANCE AUDIT (PA) DATA
AND PRECISION AND ACCURACY (PARS) DATA
Several problems are encountered when attempting to compare Performance
Audit (PA) data and Precision and Accuracy (PARS) data. Obviously, compari-
sons can be made only where the same pollutant measurement methods are
audited in both programs. The following pollutant measurement methods are
audited in both programs.
Continuous Methods CO
S02
Manual Methods TSP
Pb
S02
N02
Further, only the accuracies of the PARS system can be compared because no
precision assessments are currently made from the PA data.
Other factors to consider in making comparisons are:
1. source of data (organization performing the audits),
2. time of audit, and
3. concentration level (or flow rate level for TSP).
Valid comparisons can only be made for those organizations where both
the PA and the PARS audits are performed. The PARS data are reported by
Reporting Organization, whereas the PA data are reported by Laboratory.
A cross-reference listing has been prepared to match up each Reporting
Organization number with its corresponding Laboratory number. The compari-
sons made on a state, regional, or national basis are made using only those
Reporting Organization-Laboratory match-ups where both have reported accu-
racy audit data.
Good agreement should be expected between the PARS and PA data for a
given Reporting Organization-Laboratory combination if the two audits were
performed at nearly the same time. However, the PA audits are scheduled
at various times during the year. And, the regulation requirement for the
PARS accuracy audit is that (1) at least one audit per year shall be con-
ducted on each instrument (or site) for continuous instruments (CO and S02)
and for the TSP method and (2) at least two audits per quarter shall be
E-l
-------
conducted at the laboratory for the manual Pb, S02, and N0£ methods. Fur-
ther, there is no requirement or planned schedules to assure that the two
types of audits are conducted at nearly the same time. The comparisons can
therefore be made only on an annual basis for a given Reporting Organiza-
tion-Laboratory matchup. Comparisons for the continuous methods, CO and
S02, and TSP cannot be made on an individual site (instrument) basis because
the PARS data are not reported on a site basis although the PA data are.
(Beginning January 1, 1987, these PARS data will be reported to EMSL by site
so that it will be possible to make comparisons on a site basis. However,
because of the possible large differences in times of the audits, such com-
parisons may not be meaningful.)
Because of the relatively small amount of data for comparison on a Re-
porting Organization-Laboratory basis and the time differences, stifdy of the
comparisons of PA and PARS data has been limited to comparisons of larger
samples or aggregates of data, i.e., on a Regional or National basis.
Another bothersome problem in comparing PA and PARS data is that the
concentration levels do not correspond. The concentration levels are fixed
by regulation for the PARS accuracy audits whereas the levels for PA vary
from year to year and in some cases from audit to audit. Because of these
variations in concentration for the PA audits, the concentration levels for
PARS are used as a basis for defining concentration ranges for comparison.
The following tables present the concentration levels for PARS as
specified by the regulation and the concentration levels actually used for
PA audits during calendar year 1985.
TABLE E-l. CONCENTRATION LEVELS FOR PARS AND
PA AUDITS FOR 1985 FOR THE CONTIN-
UOUS METHODS
Pollutant
CO
SO 2
Concentration
PARS
3- 8
15-20
35-45
80-90
.03-. 08
.15-. 20
O J "* ^ J
.80-. 90
levels, ppm
PA
6.70
16.50
39.90
.05-. 08
.17-. 20
.22-. 26
.40- .49
.62-.69
E-2
-------
TABLE E-2. CONCENTRATION (OR FLOW) LEVELS FOR
PARS AND PA AUDITS FOR 1985 FOR
MANUAL METHODS
Concentration (or flow) leve
Pollutant PARS
TSP ft3/min
50
(nominal)
40-60
Pb jig/strip
100- 300
600-1000
S02 yg/ml
0.2-0v3
0.5-0.6
0.8-0.9
N°2 yg/ml
0.2-0.3
0.5-0.6
0.8-0.9
m3min
1.416*
1.133-1.699
yg/m3
0.6-1.8*
3.5-5.9
PPm
.01 3-. ,020*
.033-. 040
.053-. 059
Ppm
.018-. 028*
.046-. 055
.074-. 083
m3/min
.7
.9
1.1
1.2
1.3
,-,. 1/85
yg/m3
0.53
1.06
3.03
4.31
4.83
6.65
yg/m3
44.30
61.00
90.60
124.50
271.90
yg/ml
.345
.434
.686
.944
1.114
Is
PA
ft3/min
24.7
31.8
38.8
42.4
45.9
7/85
yg/m3
0.45
1.00
1.15
2.00
2.70
5.40
PPM
.017
.023
.034
.0473
.103
yg/m3 ppm
59.90 .032
75.35 .040
119.10 .063
163.89 .087
193.40 .103
*See conversion factors on following page.
E-3
-------
Conversion Factors
To convert
ft3 to m3
pg/m3 to ppm
S02
N02
CO
03
pg/strip to pg/m3 for Pb
Multiply by
0.02832
0.00038
0.00053
0.00087
0.00051
0.00589
12 exposed
pg strips
3/4"x8" strip 8"xlO" filter 50
= .00589 pg/m3
1
ft3/min
1 filter/day
1440 min/day
1
.02832 m3/ft3
pg/ml to
173.61
1
1000 L
1440 min/day m3
pg 50 ml 1 sample/day
ml sample .2 L/min
= 173.61 pg/m3
pg/ml to ppm
S02 (173.61)(0.00038) = 0.066
N02 (173.61)(0.00053) = 0.092
The following example illustrates the procedure for establishing the
concentration ranges for comparison purposes, for CO the four accuracy
audit levels for the PARS are 3-8, 15-20, 35-45, and 80-90 ppm. During
1985 the three concentration levels for the performance audits were 6.70,
16 5U and 39.90 ppm. The calculated midpoints between the adjacent con-
centration levels for the PARS are considered the boundaries of the ranges
for comparison:
E-4
-------
Comparison
levels
1
2
3
4
Cone.
levels,
ppm
3-8
15-20
35-45
80-90
Calculated
midpoints,
ppm
11.5
27.5
62.5
Ranges for
comparison,
ppm
0-11.5
11.5-27.5
27.5-62.5
62.5-
Performance
audit levels,
ppm
6.70
16.50
39.90
As shown above the calculated midpoint between 8, the upper limit of
PARS level 1, and 15, the lower limit of PARS level 2, is
Similarly the calculated midpoint between comparison levels 2 and 3 is 27.5
ppm, and between levels 3 and 4, 62.5 ppm. Thus, the newly defined CO
ranges for comparison are
0 to 11.5
11.5 to 27.5
27.5 to 62.5
and 62.5 and above.
Therefore, the results.of PA audits at 6.70 ppm are compared with the re-
sults of PARS audits at 3 to 8 ppm, etc., shown in the above table.
Following the same procedure, the comparison ranges for all the pol-
lutant methods have been computed and are summarized in Tables E-3 and E-4.
The problem in comparing results within the defined ranges are illus-
trated by Figures E-l and E-2.
E-5
-------
DA
PA
I
PARS
PA
1
1
CO
I 1
1
502
' w-
9-
} 0.
2
1.5 25
2
> ' m
*
115 0.2
3 4
i I
.5 50 62.5 100
ppm
3 I 4
'
1 V " V | ^ ^
1
! i
75 0.5 0.625 1-0
ppm
Figure E-l. Concentration levels for comparing PARS and
PA data, continuous methods.
E-6
-------
PARS
PA
,
1
. , ,
in
iii it
i i
20
Pb
PARS
PA
30
2.65
40
ft3/min
2
50
60
1
1
LJB LJLJ LJ |_ ! LJ
1
1 ill 1 1 1 1
1 1 1
10
PARS
PA
S02
w 1 w w
1
1
1
1
1 ill 1
1 W 9
1
1
i 1 1 1 1
1 1
0.027
0.047 0.05
ppm
0.1
PARC
DA
i
N02 1
f A
^^""""^^ .
I
) 0.0
2 I 3
0
I
'
i{ mm
I I 111
37 0.05 0.065 0.1
ppm
Figure E-2. Concentration or flow levels for comparing PARS
and PA data, manual methods.
E-7
-------
TABLE E-3. CONCENTRATION RANGES FOR COMPARISON OF PARS
AND PA DATA - CONTINUOUS METHODS
PARS
cone. Calculated Ranges for PA
Comparison levels, midpoints, comparison, levels,
Pollutant level ppm ppm ppm ppm
CO 1 3-8 0-11.5 6.70
11.5
2 15-20 11.5-27.5 16.50
27.5
3 35-45 " 27.5-62.5 39.90
62.5
4 80-90 ' 62.5-
SO? 1 .03-.08 0-.115 .05-.08
.115
2 .15-.20 .115-.275 C.17-.20
1 .22-.26
.275
3 .35-.45 ' .275-.625 .40-.49
.625
4 .80-.90 .625-
E-8
-------
TABLE E-4. RANGES FOR COMPARISON OF PARS AND PA DATA -
MANUAL METHODS
Comparison PARS Calculated Ranges for
Pollutant level level mid-point comparison
Performance
audit levels
TSP
Pb
S02
N02
2
3
1
2
40-60
0.6-1.8
.013-.020
.033-.040
.053-.059
.018-.028
.046-.055
.074-.083
ft3/min
NA
all
pg/m;
.027
.047
.037
.065
0-2.65
2.65
3.5-5.9 2.65-
_PJI!L
0-.027
.027-.047
.047-
0-.037
.037-.065
.065-
f
24.7
31.8
38.8
42.4
45.9
1/85 7/85
.53
1.06
3.03
4.31
4.83
6.65
.017
.023
.034
.0473
.103
.032
.040
.063
.087
.103
.45
1.00
1.15
2.00
!.70
i.40
E-9
-------
As shown in Table E-3, the results of level 2 PARS continuous S02
accuracy audits at concentrations .15-.20 ppm are compared with the results
of performance audits at concentration levels .17-.20 and .22-.26 ppm.
And, from Table E-4, the results of level 1 PARS Pb accuracy audits at
concentrations 0.6-1.8 pg/m3 are compared with the results of performance
audits at concentration levels .53 and 1.06 pg/m3 of the 1/85 audit and
.45, 1.00, 1.15, and 2.00 pg/m3 of the 7/85 audit.
It has been recommended that the concentration levels for the perform-
ance audits be adjusted to more closely correspond to those of the PARS in
order to provide more valid comparisons.
Another factor which makes the PA and PARS comparisons somewhat com-
plicated is the reporting units which differ for some of the pollutant mea-
surement methods and which require the conversion of units. Further, the
persons submitting data are required to convert some informational items
to computer codes: methods, units, laboratory names/addresses, reporting
organization names/addresses, audit levels (for PARS only), etc. These con-
versions could be the source of some errors in proper identification of the
data used in making the comparisons.
E-10
-------
APPENDIX F
COMPARISON OF PARS AND PERFORMANCE AUDIT DATA
To reduce printing expenses, the detailed tabulations of the numerical
values of this appendix are not included here, but can be obtained by writ-
ten request to R.C. Rhodes, EPA, MD-77B, Research Triangle Park, NC 27711.
The format of the tables is the same as for the previous annual reports.
Please indicate in your request the particular pollutant mesurement sys-
tem(s) you desire copies for.
F-l
------- |