United States
            Environmental Protection
            Agency
           Environmental Monitoring
           Systems Laboratory
           Research Triangle Park NC 27711
EPA/600/4-88-007
January 1988
            Research and Development
vvEPA
Precision and Accuracy
Assessments for
State and Local Air
Monitoring Networks,
1986

-------
                                           EPA/600/4-88/007
                                           January 1988
      PRECISION  AND ACCURACY  ASSESSMENTS
 FOR  STATE AND LOCAL AIR MONITORING NETWORKS
                     1986
              Raymond C. Rhodes
          Quality Assurance Division
 Environmental  Monitoring Systems Laboratory
               E. Gardner Evans
      Monitoring and Assessment Division
 Environmental  Monitoring Systems Laboratory
     U.S. ENVIRONMENTAL PROTECTION AGENCY
      OFFICE OF RESEARCH AND DEVELOPMENT
 ENVIRONMENTAL MONITORING SYSTEMS LABORATORY
RESEARCH TRIANGLE PARK, NORTH CAROLINA  27711
                                            _ ,-tection

                                            >PI
                                            36^
                                       60604

-------
                                   NOTICE
     The information in  this  document has  been  subjected to the  Agency's
peer and administrative  review and  it  has  been approved  for  publication.
Mention of trade names or  commercial  products does  not  constitute endorse-
ment or recommendation for use.
                                    11

-------
                                 FOREWORD

     Measurement and monitoring research  efforts  are designed  to  anticipate
potential  environmental  problems, to support  regulatory  actions by  develop-
ing an in-depth  understanding of the nature and  processes that  impact  health
and the ecology, to provide innovative means of monitoring compliance  with
regulations, and to evaluate the effectiveness of health and  environmental
protection efforts through the monitoring of long-term trends.   The  Environ-
mental Monitoring Systems Laboratory,  Research Triangle Park, North Caro-
lina, has  the  responsibility   for  assessment  of  environmental   monitoring
technology and  systems;  implementation  of  agency-wide quality  assurance
programs for  air pollution measurement  systems;  and  supplying technical
support to other groups  in  the  Agency including the  Office  of  Air and Radi-
ation, the  Office  of  Toxic  Substances,   and  the  Office  of Enforcement.

     Ambient air  quality data  collected  by  States  and local  agencies are
used in planning  the nation's   air  pollution  control  strategy,  in  deter-
mining if  National Ambient  Air  Quality Standards  are being  attained,  and in
determining long-term trends of air quality.  Prior to the regulations of
May 10, 1979, the procedures used in site selection, controlling  equipment,
and calculating  and validating  data  varied  considerably  among agencies.
These regulations serve  to  improve and make more uniform the quality  assur-
ance programs of the state  and  local agencies and to require the  assessment
and reporting of  data  quality   estimates  for precision  and accuracy.   Re-
porting of precision and accuracy data was first  required for  calendar  year
1981.  Previous  reports  summarized  the results for 1981,  1982,  1983,  1984
and 1985.    This  report  summarizes  and  evaluates  the   results  for  1986.
                                          John C.  Puzak
                                         Deputy Director
                           Environmental  Monitoring Systems Laboratory
                             Research Triangle Park, North Carolina

-------
                                 ABSTRACT

     Precision and  accuracy  data obtained  from  State  and local  agencies
during 1986 are  summarized  and  evaluated.   Some comparisons are  made  with
the results reported for previous years  to determine the  indication  of any
trends.  Some trends indicate continued  improvement in the  completeness  of
reporting of precision and accuracy data.  The  national  summaries indicate
a further  improvement  in  the precision  and  accuracy  assessments  of  the
pollutant monitoring data collected.   The annual results from each reporting
organization are  given  so that   comparisons may be made  from  year-to-year
and with other reporting organizations.

     A comparison of the precision and accuracy data from the  Precision and
Accuracy Reporting  System  (PARS)  with those from  the  independent National
Performance Audit Program (NPAP)  conducted by the  Environmental  Monitoring
Systems Laboratory is made.

-------
                                 CONTENTS
                                                                       Page

Foreword	    ill

Abstract	     iv

Figures	     vi

Tables	    vii

Acknowledgment	*	     ix

     1.   Introduction	      1
     2.   National Results 	      4
             National Data Reporting 	      4
             National Activity in Performing Precision
                Checks and Accuracy Audits  	      4
             1986 Results from the PARS program	      9
             National Precision Results Comparison 	     11
             National Accuracy Results Comparison  	     12
             National Frequencies  . . . .	     13
     3.   Regional Results 	     16
             Regional Data Reporting	     16
             Regional Comparisons  	     20
     4.   Results by Reporting Organizations 	     31
     5.   Further Evaluation of PARS Data  . .	     33
             Manual Methods  	     34
             Comparison of National Limit Values and 50
              Percentile Values  .	     36
     6.   Comparison of Results from the PARS and the Performance
           Audit Program	     38
     7.   Conclusions and Recommendations  	     49

References	     50

Appendix A - Glossary	    A-l

Appendix B - Formulas for Combining Probability Limits 	    B-l

Appendix C - Listing of Reporting Organizations	    C-l

Appendix D - Precision and Accuracy Data by Reporting Organization .    D-l

Appendix E - Problems Involved in the Comparison of Performance
             Audit (PA) Data and Precision  and Accuracy
             (PARS) Data	    E-l

Appendix F - Comparisons of PARS and Performance Audit Data  ....    F-l

-------
                                 FIGURES                f          >

Number                                                                 Page
 1.  National Precision Probability Limits for 1983 through 1986 . .     12
 2.  National Accuracy Probability Limits for 1983 through 1986  . .     13
 3.  CO Precision and Accuracy by Region for 1983 through 1986 ...     21
 4.  Continuous S02 Precision and Accuracy by Region for 1983
     through 1986	.	     22
 5.  Continuous N0£ Precision and Accuracy by Region for 1983
     through 1986	     23
 6.  Ozone Precision and Accuracy by Region for 1983 through 1986  .     24
 7.  TSP Precision and Accuracy by Region for 1983 through 1986  . .     26
 8.  Lead Precision and Accuracy by Region for 1983 through 1986 . .     27
 9a. Comparison of PA and PARS for CO (Level 2)	    41
 9b. Comparison of PA and PARS for TSP  (Level 2)  .	    41
 9c. Comparison of PA and PARS for Pb (Level 2)   .	    42
 9d. Comparison of PA and PARS for Continuous S02  (Level 3)   ....    43
 10.  Comparison of PA and PARS, National  Values,  1986	    45
 E-l.   Concentration Levels for Comparing  PARS and  PA
       Data,  Continuous Methods 	   E-6
 E-2.   Concentration or Flow  Levels for  Comparing  PARS
       and PA Data, Manual Methods	   E-7
                                     VI

-------
                                    TABLES

Number                                                                   Page

 1.    Requirements for Performing Precision Checks  for
       SLAMS Network	      3

 2.    Concentration Levels for Conducting  Accuracy
       Audits of SLAMS Network 	      3

 3.    Percent of Reporting Organizations Reporting  Precision
       and Accuracy Data	      4

 4.    Year-to-Year Activity of Precision and Accuracy
       Assessments for the Manual  Methods  	      5

 5.    Year-to-Year Activity of Precision and Accuracy
       Assessments for the Continuous Methods  	      6

 6.    National Precision and Accuracy Probability Limit Values
       for Manual Methods	     10

 7.    National Precision and Accuracy Probability Limit Values
       for Automated Analyzers		     11

 8.    Percentiles of Quarterly Probability Limits for All
       Reporting Organizations (1986)  	 .... 	     14

 9.    Total Number of Reporting Organizations Required to Report
       for the Year 1986, by Pollutant	     16

 10.    Percentage of SLAMS Sites with Complete Data in PARS
       for the Year 1986 .	     17

 11.    Number of Reporting Organizations Having Data in the PARS
       Master File for the Year 1986	     31

 12.    Comparison of the 50-Percentile Frequency Distribution
       Values with the National Limit Values for 1986	     33

 13.    Values of Quarterly Probability Limits Considered as
       Excessive Based on  1986 Data	     37

 14.    Summary Comparison of EMSL Performance Audits  (PA) vs.
       PARS  Accuracy Audit Data for  Year 1986	    39
                                     vn

-------
                             TABLES  (continued)

Number                                                                 Pa9e


E-l.  Concentration Levels for PARS  and PA Audits for
      1986 for the Continuous Methods	    E-2

E-2.  Concentration (or Flow) Levels for PARS and PA
      Audits for 1986 for the Manual Methods	    E-3

E-3.  Concentration Ranges for Comparison of PARS and
      PA Data - Continuous Methods	    E-8

E-4.  Ranges for Comparison of PARS and PA Data -
      Manual Methods . .  	 ......    E-9
                                     vi i i

-------
                              ACKNOWLEDGMENTS

     The authors  express  appreciation  to the  following  persons and  organi-
zations who assisted  in  the  preparation  of  this  report:   the  States  and
local agencies,  for  performing  and  reporting  the results  of  the  precision
checks and  accuracy  audits;  the  Regional  Office  persons  responsible  for
reviewing and  coordinating the  reporting of the precision and  accuracy  data
to EMSL/RTP; Robert  L.   Lampe,  EMSL/RTP,  for  reviewing  and processing  the
precision and  accuracy  reports  received  from the  Regional  organizations;
Robert Lyon,  Computer   Sciences  Corporation,  for the  computer  programing,
processing, and  summarization of the precision and accuracy data;  John  Hol-
land, Northrop Services, Incorporated, for assistance in preparing the figures
and tables; Edward Barrows, Northrop Services,  Incorporated, for programming
and reporting  of the comparisons of the results of the EMSL  performance audit
program with the precision and accuracy data;  Les  Sizemore Northrop Services,
Incorporated,  for the programming  and  preparation of the charts  showing the
results from each reporting organization for the years 1981  through 1986; and
to Elizabeth  Hunike  and   Ceci   Ellington,  EMSL,  for  typing   this  report.

-------

-------
                                 SECTION 1

                               INTRODUCTION
     The purpose of  this  document  is to report  the  sixth year  of  data from
the Precision  and  Accuracy  Reporting  System  (PARS).   Federal  regulations
promulgated on May  10,  1979,  require  quality  assurance  precision  and accu-
racy (P and  A)*  data to  be  collected.   Collection started  January  1, 1981,
according to  requirements  set  forth  in  40 CFR  Part  58 Appendix  A.1   These
requirements provide  for  more  uniform Quality Assurance programs  and  speci-
fic precision and accuracy  assessment  and  reporting  requirements  across  all
State and local air monitoring agencies.

     The major portion of this report  consists  of summarizations and evalua-
tions of the  P  and  A data  obtained by the efforts  of the  States  and local
agencies.  In  addition,   comparisons  have   been  made  of  the  accuracy  data
collected for  PARS   with   the  results  of  the   National  Performance  Audit
Program (NPAP) which  has  been an  ongoing  program conducted  by the Environ-
mental  Monitoring Systems  Laboratory (EMSL) since the early 1970's.

     These summarizations   and  evaluations  of  precision  and  accuracy  data
serve the following purposes:

     1.   Provides quantitative estimates  of the precision and accuracy of
          their ambient  air monitoring  data  to  State  and   local  agencies.

     2.   Indicates the  need to improve quality assurance systems in specific
          reporting organizations  if  a comparison of  the data  from  all  the
          agencies shows  excessive variability or bias.

     3.   Indicates the  need for improvement in monitoring methodology if
          precision and  accuracy  estimates  are excessive or erratic.

     4.   Provides users  of data  from the State and Local Air Monitoring
          Stations (SLAMS)  network  a quantitative estimate  of the precision
          and accuracy of  the ambient air quality data.
*When one  speaks  of  precision and accuracy of  measurement  data,2 one really
means the precision  and  accuracy of  the  measurement process  from  which the
measurement data are obtained.   Precision  is  a  measure  of the "repeatability
of the measurement process  under specified conditions."  Accuracy  is  a meas-
ure of "closeness to the truth."

-------
     Ambient air quality data,  collected  by States and local  agencies  since
1957, have  been  stored  in the  National  Aerometric Data Bank  (NADB).   These
data are  used  in (1) planning the  nation's air pollution control  strategy,
(2) determining  if  the  National  Ambient  Air  Quality  Standards  are  being
achieved, and  (3) determining long-term trends  of  air  quality.  Prior to the
EPA air  monitoring  regulations  of May  10,  1979,  the procedures used in  se-
lecting monitoring sites, operating  and controlling the equipment,  and calcu-
lating, validating and report-ing the data varied considerably among agencies.
Frequently the procedures being used were not  well-documented.  These condi-
tions made  it  difficult  to  compare data  from different sites  and agencies.
Furthermore, little information was  available on the reliability of the moni-
toring data.

     To help  alleviate  these  problems,  EPA's  air  monitoring  regulations
imposed uniform  requirements  on network  design,  siting, quality  assurance,
monitoring methods, and data reporting after December 31, 1980.  For example,
only EPA  reference,  equivalent,  or  other EPA-approved air monitoring methods
are to  be  used.   Also,   calibration  standards  are  to  be  traceable  to the
National Bureau of Standards  (NBS) or other  authoritative standards.  Further,
the quality assurance systems of the States  are required to be documented and
approved by the  EPA  Regional  Offices.  Finally, the  reporting organizations
must also follow specific procedures when assessing the P&A of their measure-
ment systems and must report the P&A data to EPA quarterly.   Starting January
1, 1981, these regulations became effective for National Air Monitoring Sites
(NAMS),  and beginning January 1, 1983, for all State and Local Air Monitoring
Stations  (SLAMS).  These regulations have remained  in effect for data obtained
through  1986.  Revised  regulations, March 19,  1986,  require  the reporting of
all the  raw data to EMSL/RTP effective for  all  data obtained after December
31,  1986.

     The  precision assessments  were determined by  performing repeated meas-
urements  of  ambient-level  "calibration"  gases  at  two-week   intervals  for
continuous  methods,  or  by obtaining  duplicate results from  collocated sam-
plers  for manual  methods.   Table 1  summarizes  the requirements for perform-
ing  precision  checks.   The  accuracy  assessments  were  generally  determined
by analyzing  blind  audit  materials  traceable to  NBS.  Table  2  shows the
concentration  levels.   During each  calendar year,  each  site  or  instrument
must be  audited  at  least  once.  Details concerning the  specific procedures
and  computations  used  to  assess  P&A  are  contained  in  the  regulations.

-------
          TABLE 1.  REQUIREMENTS FOR PERFORMING PRECISION CHECKS
                    FOR SLAMS NETWORK
                                 Precision check
                          Frequency"
Parameter
    CO (continuous analyzer)

    S02, N02, and 03
       (continuous analyzer)

    TSP, S02, and N02
         (manual)

    Pb
   8-10 ppm

0.08 - 0.10 ppm
Collocated sampler
(Ambient concentration)

Duplicate strips
(Ambient concentration)
                                                 Once each 2 weeks

                                                 Once each 2 weeks


                                                 Once each 6 days


                                                 Once each 6 days
              TABLE  2.  CONCENTRATION LEVELS FOR CONDUCTING
                        ACCURACY AUDITS OF SLAMS NETWORK
Parameter
S02, N02, 03
(continuous)
CO
TSP (flow only)
S02 (manual)*
N02 (manual)*
Pb**
Level 1
0.03-0.08 ppm
3-8 ppm

0.013-0.020 ppm
0.018-0.028 ppm
0.6-1.8 yg/m3
Level 2
0.15-0.20 ppm
15-20 ppm
1.13-1.70 m3/min
0.033-0.040 ppm
0.046-0.055 ppm
3.5-5.9 yg/m3
Level 3
0.35-0.45
ppm
35-45 ppm

0.053-0.059
ppm
0.074-0.083
ppm
Level 4
0.80-0.90
ppm
80-90 ppm


    Concentration levels corresponding to flow rates  of .2 L/min,
   **Concentration levels corresponding to flow rates  of 50 cfm.
      When a request is made to the NADB for ambient air quality monitoring
data, the requestor receives the P  and  A data along  with  the  routine moni-
toring data.  The requestor, or  user,  of the data can  feel more confident
that the data are of the  quality  indicated  by  the assessments and that the
data have  been  obtained  from  an  agency having  a  planned and  documented
quality assurance system.  The EPA  can  also rely on  the  data in producing
its control  strategies  and determining  whether  standards  have  been  met.

-------
                                 SECTION 2

                             NATIONAL RESULTS
NATIONAL DATA REPORTING

     A measure of the completeness of the precision and accuracy data reporting
is the percentage of reporting organizations which were required to report data
for a particular  pollutant and which  have reported  results  for  at  least  one
calendar quarter  for  that  pollutant.  Table 3  shows  the progress in  data  re-
porting over the years 1981 through 1986.   Reporting for the manual  methods
for Pb, S02, and N02 was required by the regulations beginning January 1, 1983.
The reporting  of  1986 Pb  data  has  remained  the same  as  for  1985.   Only  two
reporting organizations, Guam and  Hawaii, continue to use the manual S02 method.
The bO  percent  reporting indicates that no  reports  were received from  one of
the two.  The  manual  N02  method  is no  longer used  at any SLAMS sites.  The
percentages of  reporting  organizations  reporting  some data  for TSP  and  the
continuous methods  for 1986  have  remained essentially  the  same  as  for 1985.
     The reporting  organizations  which  should  have
but did not are listed in Section 3.
reported  data  for  1986
            TABLE 3.  PERCENT OF REPORTING ORGANIZATIONS
                      REPORTING PRECISION AND ACCURACY DATA
Pollutant
measurement
CO
S02
N02
°3
TSP
Pb
S02 (manual )
N02 (manual)

1981
77
82
56
83
94
__
—
™ —

1982
89
93
72
89
97
--
—
* W

1983
99
96
88
99
99
93
75
86

1984
99
97
94
99
99
92
80
100

1985
96
97
96
95
99
96
75
100

1986
97
96
94
96
98
96
50

 NATIONAL ACTIVITY IN PERFORMING PRECISION CHECKS AND ACCURACY AUDITS

      A review of  Tables  4  and  5 clearly  indicates the considerable  increase
 in the total  number of  precision  and  accuracy  checks  from the beginning  of

-------
TABLE 4.  YEAR-TO-YEAR ACTIVITY OF PRECISION AND ACCURACY ASSESSMENTS FOR THE MANUAL METHODS
Pollutant
TSP





Pb





S02





N02





Year
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
Avg. no. of
samplers
2,334
2,538
2,662
2,650
2,455
2,128
73
164
452
492
486
413
172
63
46
36
20
8
185
83
77
50
36
12
Precision
Avg. no. of
collocated
sites
317
338
342
338
331
316
13
32
76
92
86
61
34
21
15
10
6
2
38
25
25
13
10
5
No. of valid
collocated
data pairs
13,335
16,281
16,816
17,152
16,462
15,744
473
1,704
3,885
3,937
3,508
2,749
965
706
389
297
185
62
1,422
1,168
1,324
691
469
174
No. of
data pairs
per site
42.1
48.2
49.2
50.8
49.7
49.8
36.4
53.2
51.1
42.8
40.8
45.1
28.4
33.6
25.9
28.3
30.8
31.0
37.4
46.7
53.0
53.2
46.9
34.8
Accuracy
No. of
audits
x levels
5,840
6,461
6,989
7,436
6,820
6,292
581
655
1,389
1,657
1,616
1,612
711
551
301
203
174
155
769
583
348
175
161
92
No. of
audits per
sampler
2.5
2.6
2.6
2.8
2.8
3.0
4.0
2.0
1.5
1.7
1.7
2.0
1.4
2.9
1.1
1.9
2.9
6.5
4.2
2.3
1.5
1.2
1.5
2.6

-------
       TABLE  5.   YEAR-TO-YEAR ACTIVITY  OF  PRECISION AND  ACCURACY  ASSESSMENTS  FOR  THE  CONTINUOUS METHODS
CTi
Pollutant
CO





S02




N02




Oi
uo




Year
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
1981
1982
1983
1984
1985
1986
Avg. no. of
analyzers
282
354
447
424
426
391
420
566
633
630
571
560
127
193
235
240
232
206
404
514
598
579
574
529
Precision
No. of
precision
checks
8,248
13,089
15,714
14,692
14,465
13,225
10,851
23,144
36,887
38,312
22,863
30,609
2,498
6,876
9,299
8,653
7,695
6,686
10,536
18,964
21,342
20,031
18,822
17,438
Precision
checks
per analyzer
29.2
37.0
35.2
34.7
34.0
33.8
25.8
36.6
58.3
60.8
40.0
54.1
19.7
35.6
39.6
36.0
33.2
32.5
26.1
36.9
35.7
34.6
32.8
33.0
Accur
No. of
accuracy
audits x levels*
856
1,180
1,501
1,265
1,143
1,052
1,016
1,248
1,625
1,500
1,397
1,272
320
442
635
589
550
510
1,162
1,328
1,705
1,629
1,499
1,328
-acy
No. of
audits
per analyzer
1.01
1.11
1.12
0.99
0.89
0.90
0.81
0.73
0.86
0.79 i
0.82
0.75
0.84
0.76
0.90
0.82
0.79
0.83
0.96
0.86
0.95
0.94
0.87
0.84
   *Levels 1, 2, and 3 only.

-------
the PARS  system through  1984  for  all  pollutant  methods except  the manual
S02 and N02 methods.   The increase in effort  resulted  because of the effec-
tivity of the regulation requirements for P and A data for the NAMS* sites on
January 1, 1981  and  for the  SLAMS on January  1,  1983.   The  reduction in the
manual N02 and ' SO2 methods has  resulted  from the replacement  of the manual
methods with continuous  analyzers.   The  average  number of analyzers/samplers
and the total  number of precision  checks  have decreased  from 1983 and 1984
for all  measurements.    However,  the numbers  of  precision  checks  per  site
have decreased in  some  cases  (manual  and  continuous  NO;?), increased in other
cases  (Pb, manual and continuous S02, and 03), and remained the same for TSP.
The reduction  of the number of  analyzers/samplers since  1983 has reresulted
in corresponding decreases  in the  number  of  precision  checks  and accuracy
audits.  However,  the  numbers of  accuracy  audits per  analyzer/sampler have
increased for all methods except CO, continuous S02 and 03.

     For the manual  methods,  Table 4 shows the  average number of data pairs
per collocated  site  for precision checks and the  average number of accuracy
audits per sampler.  If  the collocated samplers are operated every sixth day,
there  should be  365/6 =  61 data  pairs per year, assuming that  all the results
are above the  detection limit.  This level of precision  checks  is being ap-
proached for the TSP and Pb.   The  regulations  require that each TSP sampler/
site be audited  for accuracy at  least once each year, and that the laboratory
for the  other  manual  methods be  audited  at  least  twice per quarter.   The
computed average number of audits  per TSP  sampler is well above the required
frequency.

     For the  continuous methods, the minimum frequency  for  precision checks
is once every  two  weeks or 26 per year.  Table  5  shows that  CO, N02, and 03
analyzers are  being  checked  somewhat more  frequently, about 33, and that the
S02 analyzers  are  being checked  at  almost twice  a week.   Perhaps experience
has indicated  that  the  S02 analyzers drift  at  a higher  rate  than the other
instruments.  The  regulations  require at  least  one  accuracy  audit per ana-
lyzer/site per  year.   The  average  number  of  audits  per  analyzer  for  the
continuous methods  indicates  that  from  10 to 25 percent  of the analyzers are
not being  audited  as   required  by  the  regulations.   (Note:   The tabulated
values consider  only  the  audits  at the  three  lower  concentration levels.
Analyzers requiring  level  four  audits,  e.g.,   episode   monitors,  are  not
considered.)

     A comparison  can   be  made  between  the  average  number  of  samplers for
which  PARS  data are  reported and  the  number  of  SLAMS/NAMS sites  in  the
nation:
*See Glossary, Appendix A, for definitions,

-------
No. SLAMS/NAMS
sites
Avg. no. samplers
reporting PARS
data


1984
1985
1986
1984
1985
1986
Continuous methods
S02
540
538
534
630
571
566
N02
252
232
231
240
232
206
03
600
617
622
579
574
529
CO
439
440
450
424
426
391
Manual methods
TSP
2477
2424
2363
2650
2455
2128
Pb
382
403
414
492
486
413
S02
14
6
6
36
9
8
N02
15
14
0
50
36
12
     It appears that  for  all of  the  manual  methods  and for  continuous  S02
in 1984 and 1985, P and A data from more  samplers  were received than existed
as SLAMS/NAMS  sites.   Presumably,  these  extra  or  additional  samplers  are
being used for  special purpose monitoring  and/or both  samplers at collocated
sites (manual  methods)  are  being  counted.  However,   in  1986  neglecting  the
manual S02 and manual  N02 methods for which  none or a  few SLAMS sites exist,
the average number of samplers reporting PARS data is  less than the number of
SLAMS/NAMS sites for  all methods  except  continuous S02-  This  would indicate
less than the required P and A reporting  for most  of  the methods (except  for
continuous S02).

-------
1986 RESULTS FROM THE PARS PROGRAM

    Estimates of precision  and  accuracy  are  required to  be  computed  and
reported for each calendar quarter by each Reporting  Organization  (a  State
or local agency)  as  percentage deviation  values.   For precision, the  re-
peatability for each check  is measured  as the deviation from  the  expected
value as a  percentage of  the  expected value.  For accuracy, the  deviation
of the  audit value from  the true  value is measured as a percentage  of the
true value.  For both precision  and accuracy, 95 percent  probability  limits
are computed for the percentage  values from the average and standard  devia-
tions of the individual  percentage values:

                              "U ± 1.96 S

  where D~ = the average of the individual  percent differences;
        S = the  standard  deviation  of the individual  percent differences;*
     1.96 =  the multiplication  factor corresponding  to  95%  probability.

It is these  upper  and  lower 95% probability limits which  are  reported and
discussed in this report.

     Moreover, it should  be noted that the data and the evaluations present-
ed in this  report  include any outlier values which may  have been  reported
by the  States   and  local  agencies.   It  is possible  that  the presence  of
outliers might influence such comparisons by having undue impact on average
values  for individual reporting organizations.

     The probability limits presented throughout this  report  for states, re-
gions,  and the  nation have  been calculated using the  formulas  shown in Ap-
pendix  B and thereby most  appropriately  reflect  the total variability within
the entity involved.   (Note:  Probability  limit values in  this report and the
19845 and 19856 reports in Tables 6, 7, 12, and 14 and Figures 1 through 10
cannot  be validly compared with corresponding tables and figures of previous
reports.3»4 The limits given in this report are generally wider than corre-
sponding limits  of  previous reports for  the  reasons  discussed in  Appendix
B.)

     Table 6 exhibits the  national  probability  limits  for  each of the man-
ual pollutants.
*For the precision of manual methods obtained from paired observations, the
 standard deviation,  S,  is divided by  /2,  to  obtain variability estimates
 that apply to individual  reported values.

-------
           TABLE  6.   NATIONAL  PRECISION AND ACCURACY PROBABILITY
                     LIMIT  VALUES  FOR  MANUAL METHODS FOR  1986




Pollutant
TSP
Lead
Sulfur
dioxide
Nitrogen
dioxide
Precision

Probability
limits (%)
Lower Upper
-12 +13
-20 +20

-19 +22

-48 +45
Accuracy
Probability limits (%)

Level
Lower
„
-14

-43

-8

1
Upper
„
+11

+21

+12

Level 2
Lower Upper
-8 +8
-11 +9

-18 +14

- 6 +8

Level
Lower

__

-17

-4

3
Upper
„
__

+17

+5
     The precision limits reflect the
in the  field  to  collect  and  analyze
spread of the limits  may  be somewhat
tively low concentration levels.
repeatability of  the  methodology used
the  samples  at   ambient  levels.   The
inflated due to measurements  at  rela-
     The accuracy of the manual  methods  indicates  the  limits  at  predetermined
concentration levels for the  chemical  analysis  performed in the  samples  for
lead, sulfur dioxide, and nitrogen dioxide.   For the TSP  method,  the accuracy
measurement is for the flow rate only.   The  probability  limits  for manual  ac-
curacy are  very  good and  reflect  the  quality  of  work  done in the  chemical
laboratories for lead, sulfur dioxide,  and  nitrogen dioxide  analyses,  and in
the field  for flow  rate  measurement  for the  TSP method.   Because  of  the
continual replacement of the manual SOg and  N02  methods  with  continuous meth-
ods, very little  data  are  reported for these methods and  further discussion
of these manual  methods  is limited.  However, the detailed  results,  if any,
are tabulated in Appendix D.

     The precision and accuracy limits  for automated methods  are presented in
Table 7.  The results are nearly the same as reported  for 1985.
                                     10

-------
           TABLE  7.   NATIONAL PRECISION AND ACCURACY PROBABILITY
                     LIMIT VALUES FOR AUTOMATED ANALYZERS FOR 1986

CO
S02
N02
03
Precision
Probability
limits (%)
Lower Upper
-9 +9
-10 +10
-11 +11
-10 +8
Accuracy
Probability limits (%)
Level 1
Lower Upper
-16 +15
-17 +16
-21 +20
-14 +13
Level 2
Lower Upper
-8 +8
-13 +12
-13 +11
-17 +16
Level 3
Lower Upper
-8 +7
-12 +11
-13 +11
-10 +9
Level 4
Lower Upper
-15 +15
-14 +13
-14 +6
-8 +6
NATIONAL PRECISION RESULTS COMPARISON

     Figure 1  shows  the  national  probability limits  for precision  for  the
various methods.   With  data  from the  four  most  recent  years,  some  minor
trends are  evident.   Some  slight improvement,  as measured  by a  continued
reduction in the  spread  of the limits,  is  noted  for  the manual  methods  —
TSP and  S02,  and  the  continuous  methods ~  03,  CO  and N02.  Increases  in
precision for the  manual  methods,  Pb and N02, and for  the continuous method
S02 may be  due  to small  sample (N02) or  statistical  variations.  The slight
but persistent negative bias  for the  continuous  S02  method indicates  that  on
the average there is some negative instrument  drift from the most  recent cal-
ibration or  instrument  adjustment  to  the  time  of   the  biweekly  precision
check.

     Although the manual  methods  for Pb, S02, and N02  were  not  required  to
be reported until  1983,  a number of  agencies began   reporting  in  1981.  The
manual S02 and N02 methods  are much more variable than the continuous methods.
                                     11

-------
   88
   78-
   68-
   58-
   48-
   30-
   28-
   10-
    0
  -10-
L -20-
M ~30~
; -40-
T -50-H
S -60-
% -78-
  -88
                        NATIONAL VALUES FOR PRECISION
                                 1983-1986
                                                       [ 63
                                                        84
                                                        85
                                                       I 86
  Figure 1.   National  precision  probability  limits  for  1983  through  1986.


NATIONAL ACCURACY RESULTS COMPARISON

     Figures 2a and  2b  show the national  values  for accuracy audits  for the
continuous and manual methods,  respectively,  for the four most  recent years,
1983-1986.  Improvement  for the  manual  methods is  not  evident except  perhaps
for Pb and N02.  The  variability for the TSP method  remains the  same  and the
S02 method has shown a definite  increase.  The  results  for the manual  methods
for S02 and N02  vary considerably from  year  to year because the  methods are
used in only 2 or 3 Regions and  are  being  replaced  by the  continuous methods.
Slight improvement was evident for all  the continuous methods over past years,
but has not continued for 1986.   The continuous methods for S02  and N02 show
more inaccuracy than  all  other methods.  However,  it is pointed  out that the
accuracy audits  for  the manual methods  check  only  a portion of  the measure-
ment method.
                                     12

-------
        NATIONAL VALUES FOR ACCURACY
                1983-1986
              MANUAL METHODS
NATIONAL VALUES FOR ACCURACY
        1983-1986
    CONTINUOUS METHODS
          Pollutant and Level
                   a
   Pollutant and Level
            b
   Figure 2.   National  accuracy  probability limits  for  1983  through  1986.


      Although the  continuous  N02  method  is  more  variable than  the  other
methods, it  has  shown the  greatest improvement, particularly  for the  level
1 concentration.

      The general, and expected, pattern of  variability  across levels  is very
evident, with the greatest percentage variability at the lowest concentration
levels.  The slight  negative biases for  the continuous S02 method is  consis-
tent across  all  three levels.   A possible  cause is that,  on the average,  a
negative drift occurs with  these  analyzers  from the time of  last  calibration
or instrument adjustment until  the  time  of  the accuracy  audit.

NATIONAL FREQUENCIES

     Table 8  contains the 1986 percentiles  for  precision probability  limits
and accuracy probability  limits  at  levels  1,  2, 3,  and 4.  The  percentiles
are based  on the  total  number of   reporting-organization  quarters  of  data.
The individual  quarter  of  data  consists of  an  upper  and  lower  probability
limit for precision,  and  upper and lower probability limits  for accuracy  for
each of  the  levels.   The  narrower the  distribution,  the   better  the data
quality.  For example, for precision for CO, the upper  5 percentile  value  for
the upper limit is +15%, and the lower 5 percentile value for the  lower limit
is -15%.  It  can be  seen  from both Figure  2  and  Table 8 that  CO  shows  the
tightest range of the pollutants presented.  The variabilities shown in Table
8 are consistent  with those shown  in Figures  1  and 2.  The  95th  percentiles
provide criteria  beyond which  a  reported probability limit may be  considered
excessive and for which  the computation should  be rechecked or the  measurement
system investigated and corrected,  if so indicated.
                                     13

-------
                              TABLE  8.   PERCENTILES OF  QUARTERLY PROBABILITY  LIMITS  FOR
                                           ALL REPORTING ORGANIZATIONS  (1986)
ENVIRONMENTAL PROTECTION AGENCY

DATE 10/16/87
PROGRAM PA250
                                           MANUAL  METHODS

                                EMSL PRECISION/ACCURACY REPORTING SYSTEM

                              FREQUENCY DISTRIBUTION OF PROBABILITY LIMITS
                                    DATA  SELECTED FOR THE YEAR 1986

                                    NATIONAL  FREQUENCY SUMMARY
                                                                                        PAGE NO.  147
                                                                                        REPORT PA250R01
POLLUTANT
      LEVEL
                                             MANUAL METHODS

        OF   	LOMER PROBABILITY LIMIT	STD  	UPPER PROBABILITY  LIMIT	STD
REP.ORS.-QTR MIN 01X 05X 10X 25X SOX 75X 90X 95X 99X MAX MEAN DEV  MIN 01X 05X 10X 25X SOX 75X 90X 95X 99X MAX MEAN DEV
111101 - TSP
    PRECISION
    ACC-LVL 2

112128 - PB
    PRECISION
    ACC-LVL 1
    ACC-LVL 2
     546
     511
     219
     273
     273
-34 -29 -21 -17 -13 -09 -06 -03 -02 -00 +20 -10
-25 -21 -14 -11 -08 -06 -03 -01 -00 +02 +09 -06
5.9  -03 +01 +02 +03 +06  +09  +14 +19 +24 +33 +46 +11  6.9
4.5  -10 -04 -01 +01 +03  +05  +08 +12 +15 +25 +46 +06  5.4
-51 -45 -37 -28 -19 -10 -07 -03 -01 -00 -00 -14 10.5
-59 -48 -18 -15 -10 -06 -03 -01 +01 +05 +13 -08  7.8
-37 -28 -19 -12 -09 -05 -02 -01 +01 +03 +08 -06  6.0
     -20 -09 -00 +02 +05 +10  +17 +30 +38 +65 +99 +13 13.8
     -30 -12 -03 -02 +01 +04  +07 +13 +17 +38 +45 +05  7.4
     -15 -08 -03 -01 +01 +03  +07 +10 +15 +18 +69 +04  6.5
181102 -
    PRECISION
    ACC-LVL 1
    ACC-LVL 2
    ACC-LVL 3

142401 - S02
    PRECISION
    ACC-LVL 1
    ACC-LVL 2
    ACC-LVL 3

142602 - N02
    PRECISION
    ACC-LVL 1
    ACC-LVL 2
    ACC-LVL 3
      46
      12
      25
       1
       6
      11
      14
      10
      13
      12
      11
      10
_39 .39 -32  -Z6 -17 -09 -04 -00 +05 +11 +11 -11 10.9
-31 -31 -31  -17 -09 -06 -01 -01 -00 -00 -00 -08  8.9
-27 -27 -24  -17 -10 -06 -03 -01 -00 +04 +04 -08  7.3
-13 -13 -13  -13 -13 -13 -13 -13 -13 -13 -13 -13  Q.r
-32 -32 -32 -32 -16 -08 -08 -06 -06 -06 -06 -14  9.7
-75 -75 -75 -51 -48 -30 -11 -08 -07-07 -07 -31 22.6
-37 -37 -37 -32 -14 -10 -04 -01 +02 +02 +02 -12 11.1
-40 -40 -40 -14 -09 -08 -04 -01 -01 -01 -01 -10 11.1
     -09 -09 -01 +02 +04 +06  +17 +22 +24 +33 +33 +10  8.8
     +01 +01 +01 +01 +03 +08  +12 +16 +42 +42 +42 +09 11.4
     -02 -02 -01 -01 +02 +06  +15 +27 +29 +43 +43 +10 11.2
     +17 +17 +17 +17 +17 +17  +17 +17 +17 +17 +17 +17  0.0
     +04 +04 +04 +04 +11 +26 +33 +35  +35 +35 +35 +21 12.3
     +05+05 +05 +05 +05 +10 +20 +23  +40 +40 +40 +13 10.8
     -03 -03 -03 +02 +04 +06 +13 +20  +25 +25 +25 +08  7.6
     -03 -03 -03 +01 +01 +06 +17 +22  +22 +22 +22 +08  9.1
.47 .47 .47 -38 -34  -28 -11 -05 +11 +11 +11 -22 16.6   -41  -41  -41 -16 +05 +07 +55 +65 +89 +89 +89 +20  36.0
-18 -18 -18 -17 -11  -07 -01 -00 -00 -00 -00 -08  6.2   -01  -01  -01 +02 +05 +12 +14 +16 +16 +16 +16 +10  5.8
-10 -10 -10 -08 -06  -03 -01 -01 -00 -00 -00 -04  3.1   -02  -02  -02 -01 +02 +04 +08 +12 +12 +12 +12 +05  4.7
-06 -06 -06 -04 -04  -01 -00 +03 +03 +03 +03 -02  2.6   -01  -01  -01 +01 +01 +04 +04 +06 +06 +06 +06 +03  2.3

-------
(Continued)
  ENVIRONMENTAL PROTECTION AGENCY

  DATE 10/16/87
  PROGRAM PA250
       TABLE 8.   PERCENTILES OF  QUARTERLY  PROBABILITY  LIMITS FOR
                   ALL REPORTING ORGANIZATIONS  (1986)

                   EMSL PRECISION/ACCURACY REPORTING SYSTEM

                 FREQUENCY DISTRIBUTION OF PROBABILITY LIMITS
                       DATA SELECTED FOR THE YEAR  1986

                       NATIONAL FREQUENCY SUMMARY
PAGE NO.   146
REPORT PA250R01
POLLUTANT
NUMBER I
LEVEL REP.ORG.-
C42101 - CO
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C42401 - S02
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C42602 - N02
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C44201 - 03
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3
ACC-LVL 4
C42601 - NO
PRECISION
ACC-LVL 1
ACC-LVL 2
ACC-LVL 3

362
281
274
268
9

413
332
328
322
44

236
168
162
160
9

394
305
298
291
22

4
2
2
2
                                                         AUTOMATED ANALYZERS

                            	LOWER PROBABILITY LIMIT	STD	 UPPER PROBABILITY LIMIT	STD
               REP.ORG.-QTR MIN 01X 05X 10X 25X SOX 75X 90X 95X 99X MAX MEAN DEV  MIN 01X 05X 10X 25X 50X 75X 90X 95X 99X MAX MEAN DEV
                            -27 -24 -15 -12 -09 -06 -04 -02 -01 +01 +05 -07
                            -65 -59 -23 -19 -14 -08 -02 -00 +01 +06 +16 -09
                            -41 -21 -14 -12 -08 -05 -02 -00 +02 +06 +06 -05
                            -61 -20 -15 -11 -07 -04 -02 -00 +02 +06 +07 -05
                                                4.5  -05  -03 -00 +01 +03 +06 +09 +13 +15 +22 +48 +07  5.4
                                                9.7  -14  -11 -04 -01 +03 +07 +14 +20 +26 +52 +99 +09 11.2
                                                5.3  -05  -05 -02 -00 +02 +04 +08 +12 +14+24 +26 +05  5.1
                                                5.9  -05  -05 -01 -00 +02 +04 +07 +11 +13 +23 +34 +05  4.9
                            -25  -25 -25 -25 -13 -10 -01 -00 -00 -00 -00  -09  8.4  -00 -00 -00 -00 +03 +05 +13 +33 +33 +33+33 +09 10.6
-42 -33 -22 -19 -13 -09 -07 -05 -03 +01 +19 -11  6.3
-76 -
-------
                                  SECTION 3

                               REGIONAL RESULTS
REGIONAL DATA REPORTING

     All reporting  organizations  having  SLAMS/NAMS  sites  for  the  criteria
pollutants are  required  to report  P  and A  data.  The  numbers  of  such  re-
porting organizations  are  listed  in Table 9.   Note that only two reporting
organizations use the  manual  SOe  method at  SLAMS sites  and none  uses  the
manual N0£ method.
         TABLE 9.  TOTAL NUMBER OF REPORTING ORGANIZATIONS REQUIRED
                   TO REPORT FOR THE YEAR 1986, BY POLLUTANT

Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation

Automated
CO S02
C42101 C42401
6
3
10
21
20
10
11
2
11
4
98
6
5
12
22
25
10
9
4
9
3
105
methods
N02
C42602
5
2
11
10
12
9
5
3
9
2
68

03
C44201
6
3
12
26
24
10
11
5
11
2
110

TSP
111101
6
4
14
34
30
14
12
9
12
4
139
Manual
methods
Pb S02
112128 142401
5
3
9
11
15
11
9
3
8
4
78
0
0
0
0
0
0
0
0
2
0
2

N0£
141602
0
0
0
0
0
0
0
0
0
0
0
       The breakdown of  data  completeness (defined  as  the percentage  of  re-
 porting organizations  which reported P&A  data  to EPA relative to  the number
 required to report each quarter)  is given in Table 10.
                                      16

-------
    TABLE  10.   PERCENTAGE OF REPORTING ORGANIZATIONS WITH COMPLETE  DATA
                IN  PARS FOR THE YEARS  1983,  1984,  1985  and 1986

                               Manual Methods
Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation
Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation
TSP
111101
83 84
98 100
72 97
99 100
97 99
99 97
95 98
97 95
96 100
82 95
100 92
95 98
85
100
88
98
96
94
97
99
100
73
84
94
86
74
93
98
95
95
96
98
88
56
93
91
83
95
75
88
78
89
83
66
75
59
59
79
CO
C42101
83 84
85 91
92 88
100 1UO
83 84
78 85
91 97
78 78
68 98
77 89
88 94
83 90
85
90
88
99
85
83
90
80
90
63
84
84
86
71
91
97
82
85
89
84
80
72
85
83
83
92
66
100
79
77
82
69
100
60
88
80
Pb
112128
84 85
99 90
100 100
95 92
80 92
85 93
88 95
83 76
83 75
74 53
72 72
85 85
Automated
SO 2
C42401
84 85
98 96
83 63
100 100
87 79
92 86
93 84
77 72
98 91
93 64
97 11
92 82
S02
142401
86
59
96
88
85
81
85
68
77
46
84
73
83


„_
94
--
—
--
50
72
84 85


__


__
86


__
100 100 —
— •
0
—
43
64
—
0
—
19
34
~
--
—
19
19
N02
141602
83 84 85 86


100 	
75 100 — ~
100 — — • -
63 100 100 —
o 	
—
73 100 100 —
Methods
N02
C42602
86
71
80
97
90
92
93
85
82
73
80
86
83
56
100
96
51
65
70
68
92
58
81
69
84
80
100
98
63
79
85
75
92
94
100
88
85
50
88
89
65
66
83
69
83
71
81
73
86
49
100
94
71
75
85
84
80
74
18
78
03
C44201
83 84 85 86
79 79 73 57
96 99 67 91
99 100 99 95
81 79 81 82
76 88 78 82
96 98 96 94
80 73 68 75
96 100 88 69
75 95 59 74
94 100 100 100
84 91 80 82
NOTE — Means no data was required, there being no SLAMS sites for these
        pollutants.
                                     17

-------
     From 1985 to 1986, the percentages of reporting on a national  level
decreased for all manual methods and increased for all  automated methods,
except CO.

     A substantial lack of reporting of 1986 data occurred for the fourth
quarter, during the time when plans and preparations were being made for the
reporting of all raw data, beginning January 1, 1987.  The regulations per-
mitted the reporting organizations to begin using the raw data reporting
system beginning as early as for the third quarter data of 1986.  Start-up
problems with the raw data reporting system were no doubt responsible for
some loss of data for the third and fourth quarters.

     A number of reporting organizations having SLAMS/NAMS sites for certain
pollutants have  reported r\o_ precision or accuracy data for 1986 for these
pollutants:
Regi on
State
                           Reporting organization
Number
Name
Pollutant
 I

 II

 IV



 VII

 VIII


 IX
  IX


  X
 NH       30001      New Hampshire***

 VI       55001      Virgin Islands

 FL       10018      Dade County
 TN       44005      Chattanooga-Hamilton Co.,*
                       Air Pollution Control

 MO       26003      St. Louis City**

 MT       27003      Great Falls City-County
 MT       27004      Missoula City-County

 CA       05036      San Diego***
 HI       12120      Hawaii
 NV       29100      State of Nevada***
 NV       29100      State of Nevada***
 NV       29200      Washoe County

 NV       29300      Clark County**
 GU       54100      Guam***
 GU       54100      Guam***

 WA       49001      Washington
                                                NO 2

                                             TSP, S02

                                                S02
                                                CO


                                                Pb

                                                03
                                                03

                                                Pb
                                             S02, N02
                                                CO
                                                03
                                             CO,  N02,
                                             03,  TSP
                                             Pb,  CO
                                             TSP, Pb
                                                SO 2
                                              (manual)
                                                N02
   *Repeats from 1985.
  **Repeats from 1984 and 1985.
 ***Repeats from 1983, 1984 and 1985.
                                      18

-------
     Precision and accuracy reporting for 1986 was  complete only  for the
following Region and pollutant combinations:

                        Region             Pollutant
                         II
                         X                    03

     Considering the reporting for all  pollutants (omitting the manual  S02
and N02 methods) and all  reporting organizations, the reporting organizations
of Region III were most complete for 1986 (95%).  Region III was also the
most complete in 1983, 1984 and 1985.  Region I data was the least complete
(59%).

                                          Percentage of
                                         reports complete
                      Region            83   84   85   86

                       I
                       II
                       III
                       IV
                       V
                       VI
                       VII
                       VIII
                       IX
                       X

     When considering the various pollutant methods across all Regions, re-
porting was most complete for the TSP and S02 methods and least complete for
the manual S02 method, the same as for 1984 and 1985.
84
84
97
80
83
74
65
88
66
85
91
95
99
86
88
82
80
95
83
93
83
82
96
85
83
81
77
88
57
82
64
92
95
84
85
90
82
79
66
77
                                      19

-------
                    Pollutant

                    TSP
                    03
                    CO
                    S02
                    Pb
                    N02 (manual)
                    S02 (manual)
                    NO 2
                                           Percentage of
                                          reports  complete
83
86
95
84
83
80
79
73
72
69
98
91
90
92
85
100
64
88
94
80
84
82
85
100
34
73
91
82
83
86
73
--
19
78
REGIONAL COMPARISONS

     Figures 3  through  10  compare the  precision  and  accuracy  probability
limits for  1983,  1984,  1985 and  1986.   These  comparisons are  presented  for
each pollutant on a Region by Region basis.

C0_ (Figure 3)

     Only Regions VI, VIII,  and  X showed a noticeable  improvement  from 1985
for precision.   Regions  II,  IV  and  V were  worse in  1986 for  all  accuracy
levels than in  1985.  Regions  I,  IV and  VI,  showed  consistent improvement at
all accuracy levels.

SO? (Figure 4)
     Regions II,  V  and  VI  were  consistent  in  improvements  for  all  three
levels of accuracy; however,  Regions I, VIII  and  X were  worse  in precision
and all levels of accuracy.

N02 (Figure 5)

     More regions  showed improvement  than  not.   Regions  IV,  and  VII  were
better at all  accuracy levels  in  1986 than  in  1985  ~ Regions  II  and  VIII
were worse.

   (Figure 6)

     For 1985 more  regions  showed improvement in  precision  and  accuracy for
ozone than  for  any other measurement.   These significant  improvements  were
possibly attributed to the  use  of the standard reference photometers (SRP's)
developed by  the  National  Bureau   of  Standards  for  EPA  and   located  at:
                                     20

-------
   40-
P
R
0
B
A
B
I
L
I
T
Y
30-
20-
10-
L -10-|

M
I -20-
T
S
  -30-
  -40-
                   CO PRECISION
                     1983-I 986


p
R 30-
0
; 20-
B
1 10-
. IkJ-
I
T 0
V
L -10-
I
M
I -20-
T
S
-30-
%
4d-









•
•

t

























1
\







-i

- m
- m
- E
]w





1
^
CO ACCURACY LEVEL 1
1983-1986



1
a
_ E -J- = • : •

: : :
: : :

: : :
: E : '
: = :

: : : [ [ •
HJ : . : . : .


m n
1 p
= " i ' • ! ' IH •

• - - Hr
: : : B=

: : : H=
: ' : : JF
• j • - • It

1 • it ^ -^
i

•
-
.
~:
u
• "^ D
•


II 1 ! I 1 I 1













|83
Q64
|es
i 86

*J fc^fcl %QtQ
    40


    30-


    20-


    10-


    0


   -10-^


   -20-


   -30-


   -40
                CO ACCURACY LEVEL 2
                     1983-1986
                          ~r
                           
-------
S02 PRECISION
1983-1986
40 —
R 38-
0

B
T
. 10-
L
I
T 0
Y B
L IB-

M
I -20-
T
S
-30-
'/,
4PI-





• - "1 • •
• ^ • fc • B • Ll - '• H
: 1 : 1 : • :Jj : I -
• 1 "'-m :m 'm ' * '
: 1 L f -f ~\£- I '
IF LJ LU g_B





w- \ 1 111
\ 1 -3 fc «,




:
.
_j •

B •

•
•
^


: :

; -
: '.
= -
| t
3


B
. I
: 1
- 1
:J1
LJ




i i i
6 1 % 9



















50-
P 40-
R
n
B 30-
A
B 20-
I
L 10-
I
T o,
T 0_

, -10-
I
M -20-
I
T -30-
s

x -^









!
.
•






| 	 -3KJ —


















\







—
: 9g
: m
'• m
f






^
S02 ACCURACY LEVEL
1 983- 1 986



D



".
.
•
"

-
r :'._,' ' .
! . : . : .
: ! : ! : i :
• I • I '. \ •
: ' i • I '
'• 1 "-f : a :
LJ ^ ' !
'•

3 : a n
i ' - i :
i • = - | Q,
: r :
i : = _
! " i ! = "
1 . ' E • : '
. "~^ -
















n
083
184
85

86
1 1 l 1 1 1 1
P
R
0
B
A
B
I
L
I
T
Y

L
I
M
I
T
S
 50

 40-


 30-

 28-

 10-

  0

-10-

-20-

-30-

-40-

-50
               S02 ACCURACY LEVEL 2
                    1983-1986
               l
               1
                        \
                        6
 50

 40-

 30-

 20-

 10-

  0

-10-

-20-

-30

-40

-50
                                                          S02 ACCURACY LEVEL 3
                                                               1983-1986
                       ~T
                        e
 83

 84

 85

§86
         Figure 4.   Continuous S02  precision
                      1983 through  1986.
                                                   and accuracy by  region for
                                           22

-------
   60-
   40-
   20-
L
I -20
M
I
T
S -40
  -60
                  N02 PRECISION
                    1983-1986
                                                60
                                                40-
                                                20-
P
R
0
B
A
B
I
L
I
T
Y
L
I -20
M
I
T
S -40
                                               -60
N02 ACCURACY LEVEL 1
1983-1986
ri
-
-
L

9
: -



•
E F i • T
-. ' *M_ ' E " = '
: . f . • . • .
I •• • i - 1 •
Fn i i !
i
•
•
E - !
: - : -
).
:
;
|J :
i

:
=
E
j

E.!  B»
                                                                                          85

                                                                                          86
               N02 ACCURACY LEVEL 2
                    1983-1986
   -60
                                                60-
                                                40-
                                                20-
                                               -20-
                                               -40-
                                               -60-
               N02 ACCURACY LEVEL 3
                    1983-1986
                                                    \   ^
                                                               \    i   i    i    i   r
                                                               t,   <3   6   1    
-------
   40-
   30-
   20-
   10-
P
R
0
B
A
B
I
L
I
T
Y
L -10-

M
I -20-
T

  -30-
  -40-
                   03 PRECISION
                     1983-1986
p
R
0
B
A
B
I
L
I
T
Y
                                              L-,e

                                              M
                                              I -20
                                              T
                                              S
                                                -30
                                              •A

                                                -40
                                                 40
                                                 30-
                                                 20-
                                                 10- r
                           6   1
03 ACCURACY LEVEL 1
1 983- 1 986

I :
S
1

a

»
Vi "
i -
m -
P "
I
i:
f -
f :
1
7, *J
r^B n
= : 1 :
= - ! •
l|p :
». * 6

- = - |c
5 \
•_ \ ; [ ;
. B D-^ :
-i %

\ \
I :


a

=
1
|


^
                                               83

                                               84

                                               85

                                               86
 P
 R
 0
 B
 A
 B
 I
 L
 I
 T
 Y
    40-
    30-
    20-
    10-
 L  -10-

 M
 I  -20-
 T
 S
   -30-
   -40
                03 ACCURACY LEVEL 2
                     1983-1986
P
R
0
B
A
B
I
L
I
T
Y

L
I
M
I
T
S
                I
                -i
 40


 30-


 28-


 10-


  0


-10-


-20-


-30-


-40
                                                              03 ACCURACY LEVEL 3
                                                                   1983-1986
                                                                                            I 83
                                                                                            I 8<*
                                                                                             85
                                                                                            lee
                  Figure 6.   Ozone precision  and accuracy  by region
                               for 1983  through 1986.
                                            24

-------
          EPA,  EMSL,  Research  Triangle  Park,  NC
          EPA,  Region II,  Edison,  NJ
          EPA,  Region V,  Chicago,  IL
          EPA,  Region VI,  Houston,  TX
          EPA,  Region VIII,  Denver, CO
          California  Air  Resources  Board,  Sacremento,  CA

and which  are  being  used as  calibration reference  sources throughout  the
nation.  In November  1987, a seventh SRP was  added  at  EPA,  Region  IV,  Athens,
GA.  And, in 1988, an  eighth  SRP  will  be added  at EPA,  Region  I,  Lexington,
MA.

     However, comparison  of  1986 results with those of 1985 does not  indicate
a continuing improvement, except  for  Regions II and  III.   Regions I and  IX
show more variability at  all  accuracy  levels  in  1986 than 1985.

TSP (Figure 7)

     All regions except  III  did better  in  precision  in  1986 than in  1985.
Most Regions, except  II,  III and IV, were  better or the  same in  1986  compared
to 1985.

Pb_ (Figure 8)

     Only four regions —  II, III, V and VI — showed improvement  in  precision.
And Regions II, III,  IV, VI and IX were worse in accuracy  in  1986  than  in  1985.

Ranking Comparisons of Regions

     Ranking comparisons   were  made to determine  the  regions and  pollutant-
measurement methods  which improved most  from 1985 to 1986.   Improvement  was
indicated by a reduction  in the spread  of  the probability limits from 1985 to
1986.  Considering all pollutant-measurement  methods  (except manual   SO?  and
manual NOg) and precision and accuracy  results,  the following table lists  the
regions in order of improvement.

     For comparison,  these measures of  improvement  from 1984 to  1985  are also
shown.  Interestingly, there  were  more indications of  improvement from 1984
to 1985 than from  1985 to 1986.  Also,  Region IV averaged  at  "no  change"  for
both comparisons  and  Region   X  was  least   improved  for  both  comparisons.
                                     25

-------
p
R
0
B
A
B
I
L
I
T
Y
   40-
   30-
   20-
   10-
L -10-
I
M
I -20-
T
S
  -30-
y.

  -40-
                  TSP PRECISION
                    1983- 1 986
                                                40-
p
R
0
B
A
B
I
L
I
T
Y

L
I
M
I
T
S
 30-


 20-


 10-


  0-


-10-


-20-


-30


-40
                                                             TSP ACCURACY  LEVEL 2
                                                                  1983-1986
                                                                                           83

                                                                                           I e4
                                                                                           I 85

                                                                                           I 86
                   Figure 7.   TSP precision and  accuracy  by region for
                                 1983 through 1986.
                                                26

-------
                     p
                     R
                     0
                     B
                     A
                     B
                     I
                     L
                     I
                     T
                     Y

                     L
                     I
                     M
                     I
                     T
                     S
                      tee

                      80-

                      60-

                      40-

                      20-

                       0

                      -20-

                      -40-

                      -60

                      -80

                     -tee
                                         PB PRECISION
                                           1983-1986
                                                ~r
                                                 e
                       83

                       84

                       85

                       86
P
R
0
B
A
B
I
L
I
T
Y
30-
20-
10-
L -|0.

M
I -20-
T
S
  -30'
  -40
                PB ACCURACY LEVEL I
                     1983-1986
                          T~
                          e
   40-



0

5  »•
B
I
L
I
T
Y
                                              10-
L -10-

M
I -28
T
                                                -40'
                                                          PB ACCURACY LEVEL 2
                                                               1983-1986
                                                                                        | 83

                                                                                        I 84
                                                                                        [ 85

                                                                                        80
                Figure  8.  Lead precision and accuracy by  region  for
                            1983 through 1986.
                                            27

-------
Relative
 Score*

   9
   8
   7
   6
   b
   4
   3
   2
   1
   0
   -1
   -2
   -3
   -4
   -5
   -6
   -7
   -8
                                Regions
                     1984 to 1985     1985 to 1986
                      Ill,  IX
                      VII
                      I, VI, VIII
                      V
                      IV
                      II
VI

V

VII
I, II


IV


III

IX

VIII
X
            Most
            improved
            No change
                                                          Least
                                                          improved
*The maximum possible  score  is  +12,  i.e., if  improvement  is  indicated  for
 precision and  accuracy  for  all  6  methods.    Similarly,  the most  negative
 possible score is -12.

The most  improved  measurement method was  determined  by combining  the rank-
ings across regions and across precision and accuracy.
        Relative
         Score

          25
          20
          15
          10
            5

            0
          ™"O
          -10
          -15
                        Pollutants
               1984 to 1985     1985 to 1986

                25fc  03
                 8 » CO
                 7 • S02
                 4 « N02
                 -3 • TSP
   9 • TSP
   7
   4
  -1
  -6
 -12
NO 2
CO
03
Pb
SO 2
      General  comparisons   among  regions  can  be made  on  several  different
 bases.   One basis  is  that of improvement,  as shown  by  the above  analysis.
 However, comparisons  of improvement may  not  be  fair to  those regions  which
 already demonstrate a  history  of good  precision and  accuracy  — they  have
 little  further  room   for  improvement   and  may  be  approaching  the  inherent
 limitations or  capabilities  of the measurement methods.   On  the  contrary,
                                      28

-------
the regions that have
improvement.
shown poorer precision and accuracy have more  room  for
     A better measure  for comparison  may  be the  magnitude of  the  accuracy
assessments.  Not considering any significant biases reflected by the mean of
the upper and lower  probability  limits, the spread of the  limits  would  be a
good measure of  how  well the precision  and accuracy of  measurement  systems
are being controlled.  The following  analysis using the  spread of the limits
provide this additional and perhaps better  way  of  making  general comparisons
across regions.

     Ranking comparisons  were  also made to determine the regions  and pollu-
tant-measurememt methods which were best based on the widths of the probabil-
ity limits  for  1986.  These  comparisons were  made separately  for  the  con-
tinuous methods  and  manual  methods  and also  separately  for  precision and
accuracy.  The rankings were:

                      Ranking of Regions for Achievement

                              Continuous Methods
Precision
Rank 1985
1 (best)
2
3
4
5
6
7
8
9
10 (worst)
VI

III, IV
I
II

V, X
IX
VIII
VII
1986
VI
III
I
VIII
II
V
IX
IV
VII
X
Accuracy
Rank 1985
1 (best)
2
3
4
5
6
7
8
9
10 (worst)
III
II

VIII, IX
I
X
VII

V, VI
IV
1986
I
III
VI
IX
V
II
VII
IV
VIII
X
                       Manual Methods  (TSP  and  Pb Only)
Rank
Precision
1985
1986
1 (best)
2
3
4
5
6
7
8
9








10 (worst)
I, III
X
IX
II

IV, V
VI
VIII
VII
I, II
V
III
X
VI
IX
IV
VIII
VII
Accuracy
Rank 1985
1
2
3
4
5
6
7
8
9
10
(best) V •
II
IV

VI, VIII
X
III
I
IX
(worst) VII
1986
X

I, V
IX
IX
VIII
II
VII
IV
III
                                       29

-------
The above rankings are  similar to those  for  previous  years.   The application
of Spearman's Rank Correlation tests to the above four  sets  of data indicate
significant correlations  (approximate 0.05  significance level)  for  the  pre-
cision rankings,  but  not  significant  correlations  for  the  accuracy  data.

     It could be  said that the comparisons  of improvement relate to measures
of progress, whereas  the  comparisons of  variabilities, i.e., the*  spread  of
the limits, relate to measures of achievement.

General
     Taking into  account  the minor trends  of  improvement, the  general  con-
sistency from year to year of the differences of results among pollutants and
among levels  of  the same pollutants  on  a national basis,  and  among regions
for given  pollutants,  is  truly  surprising.   These  appreciable differences
which persist  from year to  year strongly  indicate  that whatever  forces  or
causal factors are in action in each region and in each pollutant measurement
system are  persistent  over the  years.  These significant differences between
regions should  be investigated to  identify  the major causal  factors,  since
some regions  consistently  produce more precise and accurate  data than other
regions.

     Further, each  region  should evaluate  the differences  among the states
and reporting organizations  in  a similar graphical manner as shown by Figures
3  through  10 and  the ranking comparisons of improvement and accomplishment as
shown above.  Then investigations  should be conducted  to  determine why some
states or  reporting  organizations  produce better precision and  accuracy than
others.  Appropriate  corrective  actions  should   then  be  taken to  improve
the precision  and accuracy  of  the  reporting  organizations  having  the worst
results.
                                      30

-------
                                 SECTION  4

                     RESULTS  BY  REPORTING ORGANIZATIONS
     Table 11 shows  the total  number of  Reporting Organizations  reporting
data to EMSL in  1986.  By comparing the numbers  between Tables  9  and  11,  one
can see the extra effort  exerted  by some of the State and  local  agencies to
provide quality  assurance information in  cases  where they  have  no SLAMS  or
NAMS sites.  There are an additional 4 reporting organizations  for CO, 14  for
continuous S02,  5 for continuous  N02, 10  for  03, 17 for TSP,  8 for Pb, 5  for
manual S02  and  6  for manual  N02.   Apparently, these additional  sites  are
special purpose  monitoring sites or additional local sites  not  in  the SLAMS/
NAMS network.
          TABLE 11.  NUMBER OF REPORTING ORGANIZATIONS HAVING DATA
                     IN THE PARS MASTER FILE FOR THE  YEAR 1986
Automated
Region
I
II
III
IV
V
VI
VII
VIII
IX
X
Nation
CO
C42101
6
3
11
20
20
10
11
5
9
4
99
SO 2
C42401
10
4
13
29
26
11
9
4
7
3
116
pollutants
NO 2
C42602
4
2
11
13
12
10
7
3
7
1
70

03
C44201
7
3
13
33
25
10
11
3
9
2
116

TSP
111101
16
3
15
38
30
14
13
9
10
4
152
Manual pol
Pb
112128
5
3
9
14
19
11
8
3
7
4
83
lutants
SO 2
142401
0
0
0
3
1
0
0
1
1
0
6

NO 2
141602
0
0
0
1
1
1
2
1
0
0
6
     Appendix D shows the annual  combined  upper and lower probability limits
for each  reporting  organization.   Each  reporting  organization can  compare
their values with  those of other  reporting organizations and with the regional
and national  values.   Also  given  for  each  reporting  organization  are  the
following informational items:
                                      31

-------
     Continuous methods
Manual methods
     No.  of  SLAMS  and  NAMS  sites
     No.  of  analyzers
     No.  of  precision  checks
     No.  of  accuracy  audits
No. of SLAMS and NAMS sites
No. of samplers
No. of collocated sites
No. of accuracy audits
     Any user of  monitoring data  from some  specific  site  and time  period
should obtain,  from the local  air monitoring  agency,  the  precision  and accu-
racy data for the specific sites  and time  periods  involved.

     A graphical   summarization   of  the precision  and  accuracy  probability
limits for each  reporting  organization  for  the years 1981 through  1986 will
be issued as  a   supplement  to  this  report.   A review  of these charts  will
showtime trends  and other relationships  for the data from  each  reporting
organization.  In  addition,  some  discussion  will  be  presented on  control
charts which should be plotted  by the reporting organizations for the results
from each monitoring  site.  Also,  some examples  of precision  and analysis
data presented in  graphical form in some  of  the periodic state reports will
be included.
                                       32

-------
                                 SECTION  5

                      FURTHER  EVALUATION  OF  PARS  DATA
     Some Interesting comparisons can be made by considering the  correspond-
ing national  averages of Tables  6  and  7 and the 50-percentile values  of  the
probability limits of Table 8.   Table  12  compares  these limits by  consider-
ing the spread, or range,  of the limits.


     TABLE 12.  COMPARISON OF  THE 50-PERCENTILE  FREQUENCY DISTRIBU-
                TION VALUES WITH THE  NATIONAL LIMIT  VALUES  FOR  1986

Manual methods
TSP Precision
Accuracy*
Pb Precision
Accuracy
Continuous methods
CO Precision
Accuracy
03 Precision
Accuracy
N02 Precision
Accuracy
S02 Precision
Accuracy
Nati
Lower
limit
-12
- 8
-20
-11
- 9
- 8
onal values
Upper
limit Range
13 25
8 16
20 40
9 20
9 18
8 16
-10 8 18
-17 16 30
(-10)** (9) (19)
-11
-13
-10
-13
11 22
11 24
10 20
12 25
50-percentile
Lower Upper
limit limit
- 9 9
- 6 5
-10 10
- 5 3
- 6 6
- 5 4
- 7 7
- 7 6
- 9 9
- 7 5
- 9 7
- 9 7
values
Range
18
11
20
8
12
9
14
13
18
12
16
16
 *A11 accuracy values for all pollutants are for Level 2.

**Values in parentheses were  calculated  omitting the 4th quarter of New York
  State results when limits were -99 and +99.
                                      33

-------
MANUAL METHODS

     For the manual methods,  in  all  cases the spreads  (ranges)  of the prob-
ability limits  are considerably  greater  for  precision  than  for  accuracy.
These differences  are   consistent  for  both the  National  averages  and  the
50-percentile values.  These  same  relationships have existed  for all previ-
ous years.  This means  that  the  short-term  within-sampler  variability (pre-
cision) is  larger  than  the  variability of accuracy which would  normally  in-
clude variations between,  or among,  samplers  as  well  as imprecision within
samplers.  This  may  seem  contradictory  at  first,  but   giving  consideration
to exactly  how  the results  are obtained and what the results  represent will
provide a rational  explanation.

     TSP.   In the  case  of  TSP, the precision results are obtained from col-
located sampler  data.   They  include  variability  from  the  sample collection
process, the  analytical filter  weighing  process,  the  filter  handling  and
conditioning process,  and  also  the  flow  rate  measurement  process;  whereas
the accuracy audit is  a check only  on the  flow  rate measurement.   Further,
the collocated  sampler  results  are  obtained  at  all ambient  concentrations
above 1  wg/m^,  the  detection limit   for  the method.   At  low concentration
levels the  relative variability  is  greater  than at higher concentrations.
The combined  effects  of  these  two  causes  explain the  wider  limits  for
precision.

     Manual SO?  and NO?.  Similar  to  the  TSP  data, the  precision   results
are obtainedfrom   collocated sampler data.   They  include  variability from
the flow  measurement,   absorbing  solutions,  sampling,  sample  handling,  and
storage effects  (stability)   of  the  samples as well  as the laboratory ana-
lytical portion  of the  method;  whereas  the accuracy audit is  a check only
on the  laboratory   analytical portion  of the  method.   Further,  the  collo-
cated sampler  results   are  obtained  at all  ambient  concentrations above the
detection limits   of the methods.    Many  of these  concentrations  are below
the concentrations of  the  accuracy  audits.   At  lower  concentrations,  the
relative  variability is  greater  than at higher  concentrations.

      As  noted  from Table 12,  these  differences are considerable,  indicating
that  only  a  small portion  of  the  variability  results  from  the  laboratory
analytical  part  of the method.   A  very considerable amount  of variability  of
the method  is  attributed to  other portions  of  the  measurement  process.  The
very  wide  limits  of uncertainty attributed  only  to the imprecision  of these
methods  strongly emphasizes  that the manual  methods  should  be  replaced by the
continuous  analyzers.    Alternatively,  if  any  reliance  is to  be placed  on
individual  daily data  from  the manual  methods,  all  of the various portions  of
the measurement processes  must be much more closely  controlled,  if possible.

      Pb.   The precision estimates for Pb are  obtained from the analysis  of
duplicate strips  from  the  same  hi-vol filter.   Consequently,  actual  varia-
bility  of Pb content across  the  length  of the filter,  filter  handling (with
possible loss  of  particulate),  variation in cutting  filter strips,  and  the
                                       34

-------
extraction of real-world participate  are  involved  in addition to  the  chemi-
cal analytical portion of  the  method.  The accuracy audit  data  are obtained
from the chemical analysis of  strips  to  which  known amounts of water-soluble
Pb salts have been  added  and thus do not  involve  the  other portions  of the
measurement process, nor do they involve real-world particulates.

     Further, similar to the  other manual  methods  (TSP, N02, and  S02), the
precision estimates  are  obtained at  all  concentrations above  the detection
limit.  Many  of  these  concentrations are  less  than those  of  the accuracy
audits.  At lower concentrations, the relative  variability  is  expected to be
greater than at higher concentrations.

     Beginning January 1,  1987,  the  precision  for Pb samplers must be esti-
mated by the  use  of collocated samplers similar to  the TSP measurement.  It
is expected  that  more variability will  be exhibited  by the  differences in
the results  between  collocated samplers  then between duplicate strips of the
same filter.   In anticipation  of the  effectivity  of  the  regulation,   some
agencies may  have  begun  using  collocated  samplers  for  Pb  during the latter
part of 1986.

Manual Methods (General).  To  make valid  comparisons   of  the precision and
accuracy data,  such comparisons  should  be made  at the  same   concentration
levels.  Only then  will  it be possible to  determine whether the larger  var-
iabilities of the  precision estimates  are due to differences  in  concentra-
tion level  or  to  the  larger scope  of  the   measurement   system  involved.

     Such comparison  studies  can be  accomplished  when  the raw  concentration
data are  obtained  from the  State and local agencies for  each precision and
accuracy check beginning  January  1,  1987,  as specified  by  the proposed  reg-
ulation revisions  to Appendix  A of  40  CFR,  Part  58  promulgated  March 19,
1986.  Heretofore,  only the  reporting organizations  could  perform  such  stud-
ies, since only they had the raw  data available.

     The estimation  of the  magnitude  of  the   contributions of  the   various
sources of  variability  to the  total measurement  processes  could  also be
systematically studied in  specially designed experiments.

CO, S02, N02, 03  (Continuous Methods).  The  national   values  for  precision
for the  continuous  methods  are  nearly  the same  as the  accuracy  values at
level  2.   For these  continuous  measurement  methods,   the  precision  assess-
ments  reflect  the   within-instrument  variability   obtained  from  bi-weekly
checks  at  relatively low concentrations, namely
                and
  8-10 ppm for CO
.08 -.10 ppm for S02,  N02,  and 03.
 In  comparison, the  accuracy  audits include between-1nstrument  variability  as
 well  as  imprecision,  but are conducted at  somewhat  higher (level 2) concen-
 trations.
                                      35

-------
                       15 - 20 ppm for  CO
                      .15 - .20 ppm for S02,  !%,  and  03.

Thus, the added between-instrument variability for the level  2  accuracy  audit
is almost exactly offset  by the improved percentage within-instrument varia-
bility for the precision.

     Level 1 accuracy audits are conducted  at concentrations  of

                      .03 - .08 ppm for CO
                        3-8 ppm for  S02, N02,  and 03.

     At Level 1, concentrations less than those for the precision checks,  the
probability limits for accuracy are, as expected, wider than  for precision.
(See Table 7.)

COMPARISON OF NATIONAL LIMIT VALUES AND 50-PERCENTILE  VALUES

     With reference again  to  Table 12, in  all cases  the  spreads (ranges)  of
the national  values for  both  precision and accuracy are  greater  than  for  the
corresponding 50-percentile  values.   For  the  continuous  S02  method,  the
ranges for the  national  values  were wider  than for the 50-percentile values.
There are two  reasons  why the spreads  of the national values  are much  wider
than the 50-percentile values.  First,  the  presence of significant differences
between quarters within  reporting  organizations,  between  reporting organiza-
tions within States, between States within  regions, and between regions  cause
some increase  in  the total variability  over and  above  that which would  be
obtained  from only random variability.  Second, the national  values are unduly
influenced by extreme  or outlier  values.   If there were  no  significant dif-
ferences  and  no outlier  values, the 50-percentile values should closely  agree
with the  national values.

     An evaluation of  the shape of the distributions  does in  fact show that
the distributions are not  normal due to an excessive number of extreme values
(i.e., values in the tails of the  distribution).

     All  of  the distributions of  the  upper  and  lower probability limits  are
generally symmetric  about zero.   The  only exception  is for the S02 method.
For prior years the  accuracy  audits for the  manual method  and the precision
and accuracy  audits  for  the  continuous methods  were  biased  negatively.  For
1986, the limits  for the  50  percentile values  for the continuous S02 method
continue  to  indicate  a  slight  negative  bias for  both  the precision  and  the
accuracy  data.  A possible explanation for the negative bias for  precision is
that the  relatively  low  concentrations of  S02 (0.08 - 0.10 ppm)  in cylinders
specially prepared for precision  checks may degrade after preparation.  These
biases for S02 were observed in prior years seem  to be  consistent  in magnitude
and  direction.  These  consistent  biases should be  investigated and corrected,
if possible.
                                      36

-------
     Based on the percentiles of Table 8, quarterly probability limit  values
which exceed those  listed  in  Table  13  should  be considered  excessive  or
outlier values  and should initiate  immediate  investigation  to  determine  and,
hopefully, correct the cause of such  excessive  values.   The values given  in
Table 13  are slightly tighter in  some  cases than  the  corresponding  values
given in the report  for the  1985 data.
        TABLE 13.   VALUES OF QUARTERLY  PROBABILITY  LIMITS  CONSIDERED
                   AS EXCESSIVE  BASED ON 1986  DATA

Manual methods
TSP
Pb
Continuous methods
CO
03
N02
S02
Precision limits
± 23
± 35
± 15
± 17
± 26
± 20
Level

± 18
± 25
± 24
± 38
± 28
Accuracy
1 Level
± 14
± 15
± 15
± 18
± 21
± 22
limits
2 Level 3

—
±14
± 17
± 19
± 21
                                     37

-------
                                 SECTION 6

        COMPARISON OF  RESULTS  FROM  THE  PARS  AND  THE  PA  AUDIT  PROGRAM


     A general  comparison between the  accuracy data of the  PARS  program and
the Performance Audit (PA) data is  included  in this report.   The Performance
Audit data are the  results  of an independent -check  conducted  by the Quality
Assurance Division  (QAD)  of the  EMSL  under the  National Performance  Audit
Program (NPAP).

     In the NPAP, specially  prepared  audit  samples  or devices  are  sent from
QAD to the  participating ambient air  monitoring agencies.   The samples  or
devices are  carefully  and accurately  assessed by  EMSL  utilizing  NBS  Stan-
dard Reference   Materials  (SRM's)   or   standards.   The  monitoring  agencies
analyze or measure the  samples  or  devices as  unknowns  or blinds  and report
their results to  QAD  for evaluation.   Audit  programs  are conducted  for the
following pollutant measurements, using the materials indicated:

                                                      Portion of measure-
Measurement           Audit materials                 ment system audited
    (manual)      Freeze-dried sodium sulfite         Chemical  analysis
N02 (manual)      Aqueous sodium nitrite              Chemical  analysis
Pb                Filter strip with lead nitrate      Chemical  analysis
TSP               Reference flow device               Flow
CO                Cylinders containing CO gas         Sampling  and analysis
S02               Cylinder containing S02 gas         Sampling  and analysis

     The audit materials  or devices  are  prepared  at three to  six different
concentrations or flow  levels.   Separate  reports  on  the evaluation  of  the
PA data  are published  by  EMSL.7'11   Also,  other  reports12*13  have  dealt
with the use of PA and PARS data.

     As indicated above,  the NPAP  does  not  yet  include an  audit  for  the
ozone or continuous  N02 methods.  Therefore, no  comparisons  of  the  NPAP  or
PA data with the PARS data are possible for those pollutants.

     Since precision  assessments  are  not  made  in  the  PA  program,  only
accuracy can be  compared  across  the PARS and the  PA  program?.   For the pur-
pose of  this  report, the  results  from PARS and the  PA system  are compared
at approximately  the  same levels  by  matching laboratories  and  reporting
organizations.   (See Appendix E  for a more detailed discussion  of the prob-
lems involved  in  comparing the PARS  and  PA data.)  Since the  PARS data  are
presented with outliers,  if  any,  the same  approach was  taken  with the audit
data.  Knowledge of  the  past  audit  data reports, however, indicates that the
presence of outliers may  make a significant difference  in the  audit results
for some agencies.
                                     38

-------
     Comparisons of the national  values  of  the  probability  limits  (Table  14)
exhibit fairly  good   agreement  between  the  results  of  the  two  programs.
Variations due to many  sources  of error for both data sets  are  averaged  to-
gether to obtain ttienatio'nal values, thereby masking  any correlations  which
may have  existed  for  the (results  of individual agencies.  There is  consid-
erable variation between the results of  the two programs  when comparisons  are
made on Regional and  reporting  organization  bases.   Lack  of better agreement
results from several   factors.   First, the inclusion  of outlier values in  the
PA and PARS  data  appears  to have  introduced   some  excessive distortion  of
general trends.  Second, the concentration levels for  the two  systems do  not
coincide Exactly at each  of the  audit   levels.  Third,  the PA data  are  the
results of  independent  external  audits^ while  the  PARS  accuracy  data  are
based on the results  of independent internal  audits.   The expected  effects of
the last-mentioned factor would cause the  spread of  the  limits for the PA to
be wider than  that for  the PARS.   Examination  of the  results  (see  Table  14)
confirm these  expectations.  The  PA  data for 1986 are generally better than
the corresponding data for 1985.
          TABLE 14.  SUMMARY COMPARISON OF EMSL PERFORMANCE AUDITS
                     (PA) vs. PARS ACCURACY AUDIT DATA FOR 1986



Pollutant
CO
PA
PARS
S02
PA
PARS
TSP
PA
PARS
Pb
PA
PARS



Audits

501
695

704
961

3350
4357

592
901


National values
95% probability limits (•%).
Level 1
Lower

-13
-14

-10
-15




-16
-14
Upper

11
14

16
14




13
10
Level 2
Lower Upper

- 6 6
- 8' 8

- 9 14
-13 13

- 7 9
- 7 7

-18 13
-13 10
Level 3
Lower Upper

- 6 6
- 7 7

- 9 12
-13 11






     Comparisons of the 95 percent probability limits for the PA and the PARS
results by Region are  shown  in  Figures 9a through d  for  selected  concentra-
tion levels.   The figures show considerable variation among Regions.

CQ_.  (Figure 9a)

     The width of the PARS probability limits  for level  2 exceed those for PA
for nine of the ten  Regions.  For previous years, the PA limits have generally
been wider than the PARS limits.
                                     39

-------
TSP.  (Figure 9b)

     For five Regions, the width  of the probability limits for  PARS  is less
than for PA.  This  may  be explained  by  the fact that within  each  reporting
organization the  flow rate  checks  are  not as  completely independent  from
their internal  standards  as  are the  PA  audits.  Regions  I  and X  have  more
variability of PA audit data than other Regions.

Pb_.  (Figure 9c)

     There is  considerable  variation  in the  results  from Region to  Region.
However, for  most  Regions,  the  PARS  variability  is  considerably  less  than
for PA.  This  may  be explained  by the fact  that the  local  independently-
prepared standards  for PARS have   close traceability  to the  materials  used
for calibration,  whereas  the standards  for PA,  since  they are  prepared  at
EMSL/RTP, are more completely independent.

     Regions I  and  III results  have  much  more variability for  PA than the
other Regions,  indicating a need  for investigations to  determine  the major
causes  and appropriate corrective actions.

SO?  (Continuous).   (Figure 9d)

     Figure  9d  shows  the available comparisons of the PA and PARS data for
the  continuous  S02 method.
                                      40

-------
                                       PROBABILITY LIMITS, percent
          N>
          en
                 in  o  en o
                             •»  —»  ls» is>     •-»
                          en o  en  o en     o?
              II   II   1
3
CO
CT

O
O
3
o

o
-h
-o
Di

a.
•33
en
n
o
oo
I
NJ
                        1 --
                        --- 1
                       1 ---
                         1 --
                        —H

                         -H
                         —I
                        | --
               I   III   II   I    I   I
CO


 CD

 CO


 O
"O


T3
 Q>
                                            O
 O>

 Q.,
 "D

 20
 cn
                                            O
                                            O

                                            co

                                            2.
                                            N)
                                                   |    I   I    I   I   I   I   I   I   I    I   I    M

1












1-
5
c/> r
1









._J

,1

— 1
H
	 1
1

                                                                                                     r>

                                                                                                     °
                                                                                                  1

-------

-------
                                                                         PROBABILITY LIMITS, percent
CO
                                                 CO
                                                 c
                                                 CO
                                                 Q.

                                                 o
                                                 o


                                                 T3
                                                 Ql



                                                 I

                                                 O
 Q.
 -o



 cn
 —h
 O

 O
 O


 5'
 c
 o

 

u

• *»

2 «"
o
O
g m

•j
oo
 H
CO
1-
1 1
1 1



. ' —


	 	 . 	 	 ., ,i
._ -4 ' -

H^

	 1 ~
	 1 ~~
H _

HM»
O
ro
1 1 1 1

-------
     In
limits.
eight  of  the  ten  regions,  the PARS  limits  are  wider than the  PA
 No explanation can be given to these differences.
National Comparison

     Figure 10  shows  the  available  PA and  PARS  comparisons  on a  national
basis for all levels for each pollutant method.   For  the  CO and S02 methods,
the PARS limits  are  slightly wider than  for PA  which  was  not  the  case for
CY-85 results.   For  Pb and  TSP  the  PA limits  are  wider than  for  PARS, the
same as for CY-85 results.

Missing PA and PARS Comparisons.

     Comparison  of the results from PARS and PA are, of course,  possible only
when the data are available  from  both  systems for paired reporting organiza-
tion-laboratory  combinations.  Paired data were not available for comparison.
Of these, data was not available because of missing data from the PARS for .23
comparisons:
                            Reporting       Laboratory
       Region     State     organization       number

        I           ME           20107           501012
                   ME           20112           501002
        II          VI           55001           310001
        IV          FL           10018           423002
                   TN           44005           417001
        VI          NM           32002           430001
                   TX           45003           433001
        VII         IA           16001           436001
                   MO           26003           438003
                   NE           28003           435002
                                              435003
        IX          AZ           03100            347001
                   AZ           03200            447001
                   HI           12120            348001
                   NV           29100            346001
                                               346002
                   NV           29200            446001
                   NV           29300            446002
                   GU           54100            349001
                                                    Pollutant(s)

                                                    TSP
                                                    TSP
                                                    TSP,*** S02
                                                    SO 2**
                                                    CO*
                                                    SO 2
                                                    SO 2
                                                    Pb
                                                    Pb***
                                                    S02*

                                                    CO
                                                    S02,* Pb
                                                    SO 2*
                                                    CO***

                                                    CO,* TSP*
                                                    Pb,** CO
                                                    S02,** TSP,***
                                                    Pb
   *Also missing for
  **Also missing for
  ***Also missing for
              1985.
              1984 and  1985.
              1983,  1984 and  1985.
                                       44

-------
au
40
_ 30
* 20
O)
j 10
t-
5 0
CO u
BO
O
£-10
-20
.in
' T 'C°
-JPA JPARS —
I x
_ _
T

~ i ~
— —
1 1 1
        CONTINUOUS METHODS
                MANUAL METHODS
                   T
                     T
                   * I
T
                               S02
    TT-
                         I
      I
                         2    3
                        LEVEL
1 I
* = IGNORING 1AI

Dl
TSP
T VALUE
' FOR NH R.O. 30001 AND
2 AUDIT VALUES
VA R.O. 48001
—

.,.,-,

•
T
1 *•
1
1
1
1 *•
A
•
.T
i
1
1
, J. 	

I I





FOR
^ __
T
f
ll
m •« w

1
Figure 10. Comparison of PA and PARS, national values, 1986.
                         45

-------
     Lack of laboratory participation  in  the  National  Performance Audit Pro-
gram in  1986  is the  reason  there  is  no  paired  data available  for  89 cases
compared to 135 for 1985.   In these cases, the laboratories (reporting organ-
ization) did not comply with the requirements of the federal regulations.  In
some of these  cases,  the  laboratory requested the audit  samples but did not
report any results.  A listing of missing PA audit data follows:
       Region

        II
        III
        IV
           Reporting
State     organization

 NY          33001
 NY          33001
 NY          33001
 NY          33001
 NY          33001
 NY          33001
 NY          33001
 NY          33001
 NY          33001
 NY          33001
 PR          40001
 DC          09001
 MD          21003
 WV          50001
 WV          50002
 AL          01011
 AL          01013
 FL          10001
 FL          10003
 FL          10004
 FL          10011
 FL          10014
 FL          10018

 NC          34001
 NC          34001
 NC          34001
 TN          44003
 TN          44003
 TN          44005
 IL          14003
 IN          15001
 IN          15001
 IN          15005
 IN          15008
 IN          15010
 IN          15100
 MI          23002
 MN          24001
Laboratory
number
307001
307002
307003
307004
307005
307006
307007
307008
307009
307010
309001
312100
412004
314001
314002
319001
419003
323005
323004
323008
423003
423005
423002
318001
318004
418005
317001
417003
417001
428003
329001
429009
429005
429004
529002
329002
426001
324001
Pollutant
CO, Pb
CO, Pb
CO
CO
CO
CO
CO
CO
CO
CO
CO
TSP*
SO 2**
SO 2
CO***
Pb
TSP***
TSP
TSP
TSP,* S02
CO,* Pb,* S02
TSP*
CO,** Pb,*
S02,** TSP
SO 2
SO 2
S02
CO**
CO
CO, TSP*
SO 2
CO
CO
SO 2**
Pb**
TSP***
Pb,** S02
Pb,* S02, TSP
SO 2
(continued)
                                      46

-------
      Region

       V
       VI
        VII
        VIII
        IX
           Reporting
State     organization

 OH          36001
 OH          36002
 OH          36004
 OH          36006
 OH          36008
 OH          36009
 OH          36010
 OH          36012
 LA          19001
 NM          32002
 OK          37102

 TX          45002
 IA          16001
 IA          16002
 IA          16003
 MO          26003
 NE          28002
 NE          28003
 NE          28003
 CO          06001

 MT          27002
 MT          27003
 MT          27004
 AZ          03200
 CA          05036
 CA          05061
             05061
 HI          12120
             12120
 NV          29100
 NV          29200
 NV          29300
 GU          54100
 AK          02020
 AK          02020
 ID          13001
 ID           13001
 OR          38001
-Laboratory
number
327001
327003
327007
427001
427003
427004
427005
427007
334001
430001
431001

433002
436001
436002
336001
438003
435001
435002
435003
344001

439001
439002
439003
447001
445003
445002
445017
348001
348002
346001
446001
446002
349001
451001
451002
354001
354002
353001

Pollutant
CO,* TSP*
S02
S02
Pb***
Pb***
Pb,*** S02
pb***
TSP
CO*
CO, Pb, TSP*
CO,*** Pb,*
TSP
CO,* TSP
Pb
CO*
TSP
Pb,* S02
TSP
SO 2
SO 2
CO, Pb,***
TSP***
TSP
TSP
TSP
SO 2**
CO, S02, TSP
CO, S02,** TSP
CO, S02,** TSP
CO,*** SO 2*
CO,***, SO 2*
TSP*
CO,* TSP*
Pb**
S02,** TSP***
CO
CO
CO, S02,** TSP
CO, S02,** TSP
CO, S02
  *Also missing for 1985.
 **Also missing for 1984 and 1985.
***Also missing for 1983, 1984 and
                  1985,
                                     47

-------
     In 13 cases,  data were unavailable from both  PARS  and PA:
                             Reporting       Laboratory
       Region     State     Organization       number

        IV         FL          10018           423002
                   TN          44005           417001
        VII        IA          16001           436001
                   MO          26003           438003
                   NE          28003           435002
                   NE          28003           435003
        IX         AZ          03200           447001
                   HI          12120           348001
                   NV          29200           446001
                   NV          29300           446002
                   GU          54100           349001
                                        Pollutant

                                        SO 2**
                                        CO*
                                        Pb
                                        Pb***
                                        SO 2*
                                        S02*
                                        S02,* Pb
                                        S02*
                                        CO,* TSP*
                                        Pb**
                                        S02,** TSP***
  *Also missing for
 **Also missing for
***Also missing for
1985.
1984 and 1985.
1983,  1984 and  1985.
                                     48

-------
                                 SECTION 7

                      CONCLUSIONS AND RECOMMENDATIONS
     The results of PARS data for 1986 indicate some general  improvement over
the data for  previous  years.  However, considerable differences  exist  among
Regions and individual  reporting organizations  for  most measurement methods.
Investigations should be made  by the  Regions  and  the  States  to determine the
causes of these significant differences.

     The PA data for TSP  and Pb show more  variability  than  for PARS.  These
differences are presumably due to the fact that the external  PA  accuracy au-
dits are more completely independent  than the  internal  PARS  accuracy audits.
These differences have been consistent for past years.

     Further improvement in  the data  quality  assessments,   which   are  mea-
sures of the  monitoring data quality,  can  be  achieved only  through contin-
uing efforts  of  State  and  local agency personnel involved  (first-hand) with
the operation and  quality  control   of  their measurement  systems.   Regional
QA Coordinators can  also assist through their  review  of the  operations and
quality control  practices across the States  in  their Regions.

     Each Regional   QA  Coordinator   should  evaluate  the PARS  data   from  all
the reporting  organizations  within his Region to  identify   those  organiza-
tions having  excessively  large variations  of  probability limits.   Investi-
gation should be  made to determine the  causes  and  correct  them  to preclude
future excessive  deviations.   Similarly,   Regional  QA  Coordinators  should
review the  operations  of  the   reporting  organizations having  significantly
better precision and  accuracy   results  in  order to identify  specific proce-
dures which should  be uniformly used throughout  the  Region and the Nation
to further  improve  the reliability of  the  monitoring  data  in the National
Aerometric Data Base.
                                     49

-------
                                REFERENCES


1.    Code of Federal  Regulations, Title  40,  Part  58,  "Ambient Air Quality and
     Surveillance,  Appendix  A,  Quality  Assurance  Requirements  for SLAMS."

2.    Rhodes, R.C.   "Guideline  on the Meaning and  Use  of  Precision and Accu-
     racy Data Required  by  40 CFR  Part  b8, Appendices  A  and  B."  U.S. En-
     vironmental  Protection  Agency  Report,  EPA 450/4-84-006.   Research Tri-
     angle Park,  NC 27711.   June 1983.

3.    Evans, E.G.,  R.C. Rhodes, W.J.  Mitchell and  J.C.  Puzak.   "Summary  of
     Precision and Acuracy Assessments  for  the  State and Local Air Monitor-
     ing Networks, 1982."  U.S. Environmental Protection Agency  Report, EPA-
     600/4-85-031.  Research Triangle Park,  NC 27711.  April 1985.

4.    Rhodes, R.C.  and E.G. Evans.   "Precision and Accuracy  Assessments  for
     State and Local  Air Monitoring Networks, 1983."   U.S.  Environmental Pro-
     tection Agency  Report,   EPA-600/4-86-012.   Research  Triangle  Park,   NC
     27711.  February 1986.

5.   Rhodes, R.C. and E.G. Evans.    "Precision and Accuracy  Assessments  for
     State and Local  Air Monitoring Networks, 1984."   U.S.  Environmental  Pro-
     tection Agency  Report,  EPA-600/4-86-031.   Research  Triangle  Park,  NC
     27711.  August  1986.

6.   Rhodes, R.C. and E.G. Evans.   "Precision and Accuracy  Assessments  for
     State and  Local  Air  Monitoring  Networks,  1985."   U.S.  Environmental
     Protection Agency  Report,  EPA-600/4-87-003.   Research Triangle Park,  NC
     27711.  January  1987.

7.   Rhodes, R.C., B.I.  Bennett and J.C. Puzak.  "EPA's National Performance
     Audit  Program  for Ambient Air Pollution Measurements."   In Proceedings
     of  the  75th Annual  Meeting  of  the  Air Pollution  Control  Association,
     New Orleans, LA,  June  1982.  Presentation 82-23.

8.   Lampe,  R.L., B.F.  Parr,  G. Pratt,  O.L.  Dowler and W.J. Mitchell.  "Na-
     tional  Performance Audit  Program:  Ambient  Air  Audits   of   Analytical
     Proficiency-1983."  U.S.  Environmental  Protection  Agency  Report,  EPA-
     600/4-84-077.   Research  Triangle Park, NC 27711.  October 1984.

9.   Parr, B.F.,  R.L. Lampe,  G. Pratt,  O.L. Dowler and W.J. Mitchell.  "Na-
     tional  Performance Audit Program:  Ambient  Air  Audits of Analytical Pro-
     ficiency,  1984."  U.S. Environmental  Protection Agency Report, EPA-600/
      4-86-013.   Research Triangle  Park, NC  27711.  February 1986.
                                      50

-------
10.  Parr,  B.F., R.L. Lampe, G.  Pratt,  O.L.  Dowler and W.J. Mitchell.   "Na-
     tional  Performance  Audit Program:   Ambient  Air Audits  of Analytical  Pro-
     ficiency,  1985."  U.S. Environmental Protection Agency Report,  EPA-600/
     4-87-002.   Research Triangle Park,  NC 27711.   January  1987.

11.  Parr,  B.F., R.L. Lampe, G.  Pratt,  O.L.  Dowler,  and  W.J. Mitchell.   "Na-
     tional  Performance  Audit Program Ambient Air Audits  of Analytical  Pro-
     ficiency,  1986."  U.S. Environmental Protection Agency Report,  EPA-600/
     4-87-xxx.   Research Triangle Park,  NC 27711.   November 1987.

12.  Rhodes, R.C., W.J. Mitchell, J.C.  Puzak and  E.G. Evans.   "Comparison  of
     Precision  and Accuracy Estimates from State  and Local  Agency  Air Monitor-
     ing Stations with  Results of EPA's National  Performance Audit Program."
     Journal of Testing  and Evaluation,  JTEVA,  Vol.  13,  No. 5,  September
     1985,  p. 374-378.

13.  Thrall, A.D.  and C.S. Burton.   "Special  Report, Issues  Concerning the
     Use of  Precision  and  Accuracy  Data."   U.S.  Environmental  Protection
     Agency Report, EPA-450/4-84-006.   Research  Triangle Park,  North Carolina
     27711.  February 1984.
                                         51

-------

-------
                                 APPENDIX A

                                  GLOSSARY


State and Local Air Monitoring Stations (SLAMS) —  monitoring  stations  se-
lected by the  states  and  included  in the  State Implementation  Plans.   The
stations and  the  plans  are  approved  by  the  Regional  Administrator.   The
purposes of the monitoring are to determine  compliance to  the  National  Ambi-
ent Air Quality Standards  (NAAQS)  and to determine background  levels  of the
criteria pollutants.

National Air Monitoring Sites (NAMS) -- a  subset of  the SLAMS,  selected  by
the statesTncollaboration  with the  Regional  Offices  and approved  by the
Administrator.  The purpose  of  the  sites  is to  monitor in the  areas  where
pollution concentration and population exposure are expected to be highest in
terms of the NAAQS.  Although,  in  actuality the NAMS are a  subset  of  SLAMS,
the NAMS  sites and the  non-NAMS  SLAMS  sites  are often  referred to  as two
separate groups, the NAMS and SLAMS sites, respectively.

Reporting Organization —  a  state,  or subordinate  organization within the
state, that is  responsible for  a  set  of  SLAMS  stations, monitoring  for the
same pollutant and for which PARS  data can be logically pooled (statistically
combined).  It  is  important  to emphasize  that  a  reporting  organization  is
pollutant- and site-specific  and is  responsible for the sampling, calibration,
analysis, data quality assessment,  and reporting of the monitoring  data for
the specific pollutant.  It  is  possible  that a  particular  SLAMS  station may
belong to two  different  reporting  organizations, but the  likelihood  of this
occurring is small.

Precision (Continuous  Analyzers) -- a measure  of  repeatability  obtained from
repeated measurements  of a standard  concentration in a  gas  cylinder  and the
values indicated by the analyzer.  For S02, N02, and 03 analyzers,  the gas con-
centration used for the precision  check must be between 0.08 and 0.10 ppm and
for CO it must  be  between  8  and 10  ppm.  The data from all  biweekly analyzer
checks for a given  pollutant  are  combined, and  95%  probability  limit  values
are reported to EPA each quarter  by  each  reporting  organization.   For this
report, the quarterly values  for  1986 were combined, and overall  95%  proba-
bility limits  were calculated  for  each  reporting  organization,  for  each
Region, and for the nation, as described  in Appendix  B.

Precision (Manual  Methods)  — a measure  of  repeatability  for TSP, N02,  and
SO? manual methods  (bubblers) determined by  operating  collocated  samplers at
selected sites.  At each collocated  site  one  sampler is  designated  as  the
"actual" sampler and  the other as  the "check"  sampler, and the difference
between the two samplers provides the  precision estimate.   For Pb,  precision
                                    A-l

-------
estimates are  obtained  by  analyzing  duplicate  strips  from  a  high  volume
filter sample collected at a site  where  high  Pb  concentrations exist.   These
precision checks are made from  samples,  usually  taken  every  6th  day,  and are
reported quarterly.  The data  from the manual methods were calculated  in  a
similar manner as the continuous analyzers.

Accuracy (Continuous Analyzers) —  the agreement  between  an  analyzer  mea-
surement and  a  known audit  standard concentration.    Accuracy  estimates are
obtained at least  once per year  for each  analyzer  by introducing blind audit
standards into  the analyzer.  The  audit  samples  must  span at  least  three
concentration levels  and,  whenever  possible, must  be traceable  to  NBS  or
other authoritative reference.   At least 25% of the analyzers in each report-
ing organization must be audited each quarter.  The percentage difference for
each audit  concentration  is determined,  and the  average for  all  analyzers
checked within that quarter is calculated for each level.  The standard devi-
iation for  each level  is then  used to calculate  the  95% probability limits
for the  reporting  organization, which  in turn are submitted quarterly to EPA.
These quarterly  values were combined to determine the  annual values presented
here.  They  were  calculated  in  the  same  manner  as  described  earlier for
precision.

Accuracy  (Manual Methods) — the agreement  between  an  observed  or measured
value and  a known  or  reference  value.  For N02 and  S02 manual  methods, the
accuracy  of the analytical  portion of the method is  assessed at  three levels by
the analysis  of audit materials  of known characteristics.  For Pb, the  accu-
racy  of  the  analytical  portion  of the  method is assessed at two levels.  For
TSP,  the flow rate (or air  volume)  portion  of the method is assessed at the
nominal  flow rate.

Completeness  -- the number of the  precision  and accuracy checks reported as
compared to the number  that  should have been reported if all  checks  had been
done  ,in  accordance with  the regulations.  This  value,   expressed  as  a  per-"
centage, is  not  corrected   for  instances where equipment failure prevented
conducting the check, or for  periods  when monitoring data were  invalidated.

National Performance Air  Audit  Program (NPAP) — an  external  performance  au-
dit  program conducted  by  EPA on  State  and local  agency organizatons.   Organi-
 zations  operating SLAMS stations are  required to  participate  in  this  program
directed by the Environmental  Monitoring Systems Laboratory  (EMSL)  of  the EPA
 at Research Triangle Park,  NC.   In this program,  blind  audit materials pre-
 pared by EMSL  are sent to participating  laboratories.  The  laboratories ana-
 lyze the samples  and return the  results  to  EMSL.  Shortly after the  audit is
 completed each participant  receives a  report that  compares his performance to
 that of all  other  participants.  The audit materials  for the manual methods for
 S02, N02 and Pb are used  to evaluate the accuracy  of only the  analytical lab-
 oratory portion of the method, and are as follows:
                                     A-2

-------
            Method              Audit Materials
            Manual  S02          Freeze-dried
            Manual  N02          NaN02  solution
            Pb                 Filter strips  spiked  with  Pb  $04

(Note:   Because the manual  S02 and manual N02  methods are being  replaced  by
        continuous  methods,  these performance audits have been discontinued.)

The reference flow device  used in the TSP  sampler  audit evaluates  only  the
accuracy of the flow calibration.  However, the CO and S02 continuous  analyzer
audits evaluate the entire measurement system.  As explained  above, the exter-
nal NPAP audits are conducted  in essentially the same manner as the  internal
audits (accuracy checks) for the PARS program.   The  audits  for  the Pb method
are conducted semi -annually and  those  for flow (TSP), and continuous  CO  and
S02 monitors are conducted at  least  once per year.

95-Percent Probability Limits  — probability limits  are used in  the reporting
of precision  and accuracy data to measure the expected spread or variability
of the data  from a particular  population— a  reporting organization,  a state,
a region,  or the nation.  These  expected limits are  expressed simply as  a
mean plus or minus a constant  (1.96) times the  standard deviation as  follows:

                      L = 7 ± ks                             (1)

where:  L_ =  probability limits (upper limit, LUf lower limit, L|J
        x =  mean value
        k =  1.96, a constant
        s =  standard deviation

Under the assumptions of (a) an underlying normal population, (b) the mean x,
being the estimate of the true mean, y, of the underlying population, and (c)
the standard deviation, s, being the estimate of the true standard deviation,
o, of  the  underlying  distribution,  then x  ±  1.96s  represents  the  expected
limits  which should  include  95 percent of a]_l  the  individual  measurement of
the population.  Under the assumption given,  x  ± 1.96s limits are  the expected
95 percent  probability  limits, regardless of the  sample  size.

The requirement  for the  computation  of "probability"  limits   (rather than
confidence  limits)  is  to  provide  the State and local  agencies  with  limits
which will  be of practical meaning and usefulness for  internal  control  appli-
cations  without  involving overly  complicated   and  sophisticated  statistics.
The selection of the 95 percent  level was made  because even  for non-statisti-
cians,  the  chance  or probability of  obtaining one value  out  of  twenty exceed-
ing the  limits  has practical meaning.
                                      A-3

-------

-------
                                 APPENDIX B

                 FORMULAS FOR COMBINING PROBABILITY LIMITS
     Section 5.2, Annual  Reports,,of Appendix A of 40  CFR  Part  58  required
that simple unweighted  arithmetic averages  of  the probability  limits  for
precision and  accuracy  from  the  four  quarterly  periods  of the  calendar
year be  reported with  the  annual  SLAMS  report.   The  simple  unweighted
arithmetic averages were  specified to  simplify the  calculations  for  the
states.  Such limits would be essentially correct  if  only random variations
occurred between quarters  within  a  reporting organization and  between  re-
porting organizations within a State,  i.e., if no  statistically  significant
differences occurred between quarters  within reporting organizations or  be-
tween reporting organizations within States.  However, experience has shown
that significant differences do  occur.   Because of this fact,   it  is most
correct to combine the data  across  quarters  and across  reporting organiza-
tions within States  (and  also  across  States within regions  and  across  re-
gions within the  nation)  in  the  manner  described  below.   These  formulas
determine the  yearly  probability  limits  for the  reporting  organization
which would have been  computed from all  the  individual  percent  difference
values, d-j, obtained during the year.  To  accomplish  this,  from each quar-
terly pair of  probability  limits, the average, 0-,-, and standard deviation,
S-j, are back-calculated:

                         LL + UL
                    D. = 	—                          (1)
                     1      2

                        UL - LL
                   S.	                           (2)
                    i    2(1.96)

where  LL = lower probability limit
       UL = upper probability limit

Except for the  effect  of  the round-off of the  reported  probability limits
to integer values, the above equations determine the  original D  and Sa val-
ues used  by the  reporting  organizations  to compute the originally reported
limits.

     Yearly average, D, and standard deviation, Sa values are computed from
the quarterly values as follows:
                                    B-l

-------
                   D  =

where nn  = the number of individual  percent difference,
           quarter
       -j
                     l(nrl)sf
                         (DM) -
      (3)
      , values for each
                                                         (4)
     The appropriate yearly  probability limits for the reporting organiza-
tion are computed using  the  formulas:
                   UL =  D  +  1.96  Sa

                   LL =  D  -  1.96  Sa
      (5)

      (6)
NOTE:  The same formulas  are  used  for  combining yearly reporting organiza-
       tion limits into State limits,  State  limits into Region limits, and
       Region limits into National  limits.

Example:  Suppose that the lower and upper 95% probability limits for CO
          for precision for the four quarters of  a year are:


Quarter
1
2
3
4

Number of
Precision Checks
10
9
13
7
Lower
Probability
Limit
-8
-5
-6
-12
Upper
Probability
Limit
+6
+9
+4
+11
For Quarter 1:
                LL + UL   -8+6
                  2

                UL - LL
                               = -1
           1   2(1.96)   2(1.96)
                                  = 3.6
by equation (1)
by equation (2)
     Similar computations for the other quarters, give values  in the follow-
ing table.
                                    B-2

-------
            Quarter      _n_      TT       S        D~ -  D

                1          10     -1        3.6       -0.78
                2           9+2        3.6        2.22
                3          13     -1        2.6       -0.78
                4           7     -0.5      5.9       -0.28
                         "37
     Then


          Q _ 	                                     by equation (3)


              10(-1)  + 9(2)  + 13(-1)  + 7(-0.5)
                           39

              -8.5
              	 = -0.22
               39
       Krii-1) S? + ^flL-D)2
s  = /    1      1     ]   ]	                         by equation (4)
 a
      9(3.6)2+8(3.6)2+12(2.6)2+6(5.9)2+10(-0.78)2+9(2.22)2+13(-0.78)2+7(-0.28)2
                                   39 - 1
      510.30 + 58.90

            38


  = \/14.98   =3.87


The upper and lower 95% probability limits are then computed  as:

          UL = 5 + 1.96 Sa                               by  equation  (5)

             = -0.22 + 1.96(3.87)

             = 7.37 or 7  rounded off to nearest  integer
                                    B-3

-------
          LL  =  D -  1.96  Sa                               by equation  (6)

             =  -0.22 - 1.96(3.87)

             =  -7.81 or  -8  rounded off to nearest integer

     In this  particular example, the results by the weighted combined form-
ulas are very close to the  simple unweighted arithmetic averages.  However,
in many cases the weighted  combined formulas result in wider limits than
the simple unweighted arithmetic averages and more correctly reflect the to-
tal variability exhibited by the individual percent differences.

Alternate Method of Computation

     An alternate method which eliminates the need to compute U"-j - D, the
differences between the quarterly averages and the weighted annual average,
follows.

     1.  Compute D^i and Si  for each quarter according to equations (1) and
         (2) as above.

     2.  Compute for  each quarter.

               Ed =  ni Uj                                             (7)

     3.  Compute for  each quarter.

                                    (Ed)2
               Ed2  = (ni  - 1)  S^  +  	                            (8)
                                     "1

     4.   Compute:

               Eni     the sum  of n for all  quarters                  (9)

               EEd   = the sum  of Ed  for all  quarters                 (10)

               EEd2 = the sum  of Ed2 for all  quarters                (11)


      5.  Compute 0 according  to equation (3)  above, or

                   EEd
               D = —
                   En
                                     B-4

-------
     6.   Compute  Sa:
                    /EEd2 -
                              En
              S,  - /	                                 (13)
                       (En)  - 1

     7.  Then compute the probability limits, UL and LL, according to equa-
         tions (5) and (6).

Example

     The data for the previous example on page B-2 will be used.

                                            Lower          Upper
                        Number of        Probability    Probability
          Quarter    Precision Checks       Limit          Limit

             1             10                 -8+6
             29                 -5+9
             3             13                 -6             +4
             4              7                -12            +11

     1.  TL  and S^ are computed as before.  Compute Ed  and Ed2 by equations
         (7) and  (8)  respectively.
              Quarter
Ed       Ed2
1
2
3
4

10
9
13
7

-1
+2
-1
-0.5

3.6
3.6
2.6
5.9

-10
+18
-13
-3.5
-"8.5
126.64
139.68
94.12
210.61

         For quarter 1:
                                  (Ed)2
               Ed2  =  (n  -  1)  S2  +  	                              (8)
                                    n

                                 (-10)2
                   =  (9)(3.6)2 + 	
                                   10

                   =  116.64 + 10

                   =  126.64
                                     B-5

-------
    2.  By equation (12):

                  Jt *t j    Q C
             _    Ad   -O.D
                 — = __— = -0.22     the same as before
                 in     39
    3.  By equation (13):
                                                                    (13)
                            - 1
                               (-8.5)2
                      571.05 	
                                39
              S  ,=
                           39  -  1
                     571.05  -  1.85

                          38

                =    3.87                 the  same  as  before

     4.  The probability limits  are then  calculated as  before  using  equa-
         tions (5)  and (6).

A Second Example

     The following example more  clearly shows computationally  and graphi-
cally that the arithmetic averages  of the quarterly upper and  lower  proba-
bility limits do not correctly reflect the total  variability when signifi-
cant differences occur between quarters.   Suppose the following individual
percent differences have been obtained for the precision checks for  a con-
tinuous instrument during the past  year.

          Quarter              Individual Percent Differences

             1                   -12, -9, -5, -5, -1, 2
             2                     1, 4.5, 5, 5, 5.5, 9
             3                    -6, 0, 5, 5, 10, 16
             4                   -17, -14, -10, -10,  -6, -3

From the previous formulas, the following U, S, and probability  limits for
each quarter are calculated.
                                    B-6

-------
          Quarter      IT     _JL_     n       LL    >.- UL
1
2
3
4

-5
5
5
-10

5.10
2.55
7.64
5.10

6
6
6
6

-15
0
-10
-20
-11.25
5
10
20
0
O • i




'5
                                              (-11)      (+9)

     As Indicated above,  the  simple  arithmetic averages  of the lower and
upper probability limits  are  -11.25  and  8.75,  or  -11 and 9 when rounded-
off.

     The calculations of  the  annual  probability limits by equations  (3)
through (6) are shown below.
     EnD   -30
 D = — = — = -1.25                                              (3)
     En     24
 Sa .      •.     •	-l-J	                                 (4)
      k I        *    V    d
       5(5.10)2+5(2.55)2+5(7.64)2+5(5.10)2+6(-3.75)2+6(6.25)2+6(6.25)2+6(-8.75)2
                                      24 - 1
       11596.961
        	 = 8.333
          23
               UL = D + 1.96 S                                      (5)

                  = -1.25 + 1.96 (8.333)

                  = 15.083 or (15)



               LL = D - 1.96 S        i                             (6)

                  = -1.25 - 1.96 (8.333)

                  = -17.583 or (-18)
                                    B-7

-------
     The Individual percent differences,  the quarterly probability limits,
the arithmetic annual probability  limits  and the combined annual probabil-
ity limits are shown graphically on the following figure.
1
cc 2
UJ
1-
ee
o 3


LIMITS:
ARITHMETIC AVERAGE
COMBINED DATA
	 1 	 1 	 1 	 1 	 r- — r 	 i
> 1 1 1 1 1 1
	 — i 	 ^ || | | i
L|« •£• • |u
• < i i i i i
	 1 	 1 1 1 1 1 T
L| • • • • • U
ii i i i i i 	
	 1 	 ( 	 1- I 1 I i
' ' i I 1 1 1
	 1 — 	 r~ 	 1 I 1 1 I
L| K
i i i i i i i
	 1 	 T 	 1 1 1 1 1
L|« • • •• •• • •••• ••• • • uU
II 1 1 1 	 -1— — 1 	
                     •20    -15     -10    -5      05     10

                                      PERCENT DIFFERENCE
15     20
      It 1s  clear  from the above figure  that the combined limits more  cor-
 rectly represent  the  total  spread  of the  Individual  percent  differences
 during the  year.   In fact, the calculated  values of the average and  stan-
 dard deviation  for all 24 of  the  Individual percent differences are  -1.25
 and 8.333,  respectively,  which  are in  exact agreement  with the  prev-ious
 calculations  as they must be  because  of the  exact equality of  the mathe-
 matical formulas  involved.
                                      B-8

-------
            APPENDIX C
LISTING OF REPORTING ORGANIZATIONS
Region
01
01
01
01
01
01
01
01
01
01
01
01
01
01
01
01
02
02
02
02
02
03
03

03
03
03
03
03
03
03
03
03
03
03
03
03
03
04
04
04
04
04

No.
07
20
20
20
20
20
20
20
20
20
20
20
22
30
41
47
31
33
40
55
55
08
09

21
21
21
21
21
39
39
39
48
48
48
48
50
50
01
01
01
01
01
State
Name
CONNECTICUT
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MAINE
MASSACHUSETTS
NEW HAMPSHIRE
RHODE ISLAND
VERMONT
NEW JERSEY
NEW YORK
PUERTO RICO
VIRGIN ISLANDS
VIRGIN ISLANDS
DELAWARE
DISTRICT OF
COLUMBIA
MARYLAND
MARYLAND
MARYLAND
MARYLAND
MARYLAND
PENNSYLVANIA
PENNSYLVANIA
PENNSYLVANIA
VIRGINIA
VIRGINIA
VIRGINIA
VIRGINIA
WEST VIRGINIA
WEST VIRGINIA
ALABAMA
ALABAMA
ALABAMA
ALABAMA
ALABAMA

No.
001
001
101
102
103
104
106
107
108
109
110
112
001
001
001
001
001
001
001
001
017
001
001

001
002
003
005
006
001
002
003
001
002
003
006
001
002
on
012
013
014
015
Reporting Organization
Name
AIR MONIT. SEC. DEPT. OF ENV. PROTECT.
BUREAU OF A.Q.C. DEPT. OF ENV. PROTECT.
S.D. WARREN CO., WESTBROOK
S.D. WARREN CO., HINCKLEY
INTERNATIONAL PAPER CO., JAY
BOISE CASCADE CO., RUMFORD
DRAGON PRODUCTS, THOMASTON
SCOTT PAPER CO., WINSLOW
CHAMPION INTERNATIONAL CORP., BUCKSPORT
LINCOLN PULP AND PAPER CO., LINCOLN
GREAT NORTHERN PAPER CO., MILLINOCKE
GEORGIA PACIFIC CO., WOODLAND
DIV. OF AQC. DEPT. OF ENV. QUAL. ENG.
AIR RESOURCES AGENCY
DIV. OF A. HAZ. MAT. DEPT. OF ENV. MANAGE
AIR & SOLID WASTE PROGRAMS
DEPT. OF ENV. PROT., DIV. OF ENV. QUAL.
DEPT. OF ENV. CONSERV., DIV. OF AIR
ENVIRONMENTAL QUALITY BOARD
DEPT. OF CONS. AND CULTURAL AFFAIRS
MARTIN MARIETTA
STATE OF DELAWARE, DNR & EC
WASHINGTON, DC DC & RA

STATE OF MARYLAND
ALLEGANY COUNTY
ANNE ARUNDEL COUNTY
BALTIMORE COUNTY
PRINCE GEORGE'S COUNTY
PENNSYLVANIA DER
ALLEGHENY CO. BAPC
PHILADELPHIA AMS
VIRGINIA STATE AIR POLL. CONTROL BOARD
CITY OF ALEXANDRIA
FAIRFAX COUNTY
TENNESSEE VALLEY AUTHORITY - VA
STATE OF WEST VIRGINIA
WVA NORTHERN PANHANDLE REGIONAL OFFICE
ALABAMA DEPT. OF ENVIRONMENTAL MGT.
AL, JEFFERSON CNTY. BUREAU OF ENV. HLTH.
AL DEPT. OF ENV. MANAGEMENT - MOBILE
AL, HUNTSVILLE AIR POLL. CONTROL DEPT.
AL, TRICOUNTY DIV. OF AIR POLL. CONTROL
                C-l

-------
Region
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
04
05
Ob
05
05
05
05
OS
05
05
05
05
05
05
05
05

No.
01
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
11
18
18
18
25
34
34
34
34
42
44
44
44
44
44
44
14
14
14
15
15
15
15
15
15
15
15
23
23
24
36
State
Name
ALABAMA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
FLORIDA
GEORGIA
KENTUCKY
KENTUCKY
KENTUCKY
MISSISSIPPI
NORTH CAROLINA
NORTH CAROLINA
NORTH CAROLINA
NORTH CAROLINA
SOUTH CAROLINA
TENNESSEE
TENNESSEE
TENNESSEE
TENNESSEE
TENNESSEE
TENNESSEE
ILLINOIS
ILLINOIS
ILLINOIS
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
INDIANA
MICHIGAN
MICHIGAN
MINNESOTA
OHIO

No.
016
001
002
003
004
005
006
007
Oil
012
013
014
015
016
017
018
020
022
010
001
002
003
100
001
002
003
004
001
001
002
003
004
005
006
001
002
003
001
002
003
005
008
009
010
100
001
002
001
001
Reporting Organization
Name
TENNESSEE VALLEY AUTHORITY - ALABAMA
FDER, NORTHWEST DISTRICT
FDER, NORTHEAST DISTRICT
FDER, ST. JOHNS RIVER DISTRICT
FDER, SOUTHWEST DISTRICT
FDER, SOUTH FLORIDA DISTRICT
FDER, SOUTHEAST FLORIDA DISTRICT
FUER, NORTHEAST DISTRICT BRANCH OFFICE
FL, JACKSONVILLE BIO-ENV. SERVICES DIV.
FL, HILLSBOROUGH CO., ENV. SERVICES DIV.
FL, PINELAS CO. DEPT. OF ENV. MANAGEMENT
FL, MANATEE COUNTY HEALTH DEPARTMENT
FL, SARASOTA CO. AIR POLL. CONTROL DIV.
FL, PALM BEACH COUNTY HEALTH DEPARTMENT
FL, BROWARD CO. ENV. QUAL. CONTROL BOARD
FL, DADE CO. DEPT OF ENV. RESOURCES MGT.
FL, ORANGE CO. ENV. PROTECTION DEPT.
EVERGLADES NATIONAL PARK
GEORGIA AIR QUAL. EVALUATION SECTION EPD
KENTUCKY DIV. OF AIR POLL. CONTROL
KY, JEFFERSON CO. AIR POLL. CONTROL DIST.
TENNESSEE VALLEY AUTHORITY - KENTUCKY
MISSISSIPPI BUREAU OF POLLUTION CONTROL
NC NATURAL RESOURCES & COMMUNITY DEVEL.
NC, FORSYTH COUNTY ENV. AFFAIRS DEPT.
NC, MECKLENBURG CO. DEPT. OF ENV. HEALTH
NC, WESTERN REGIONAL AIR POLL. CONTROL
SC DEPT. OF HEALTH & ENV. CONTROL
TENNESSEE DIV. OF AIR POLL. CONTROL
TN, MEMPHIS-SHELBY CO. HEALTH DEPARTMENT
METRO HEALTH DEPT. NASHVILLE-DAVIDSON CO.
TN, KNOX COUNTY DEPT. OF AIR POLL. CONTROL
TN, CHATTANOGGA-HAMILTON CO. AIR POLL. CONT
TENNESSEE VALLEY AUTHORITY - TENNESSEE
DIV. OF AIR POLL. CONT., ILLINOIS EPA
CHICAGO DEPT. OF CONSUMER SERVICES
COOK COUNTY DEPT. OF ENVIRONMENTAL CONT.
AIR POLL. CONT. DIV. OF INDIANA STATE
DIV. OF AIR POLL. CONT., EVANSVILLE
ST. JOSEPH COUNTY
AIR POLL. CONT. DIV., VIGO COUNTY
INDIANAPOLIS APC DIVISION
ANDERSON LOCAL AGENCY
PORTER COUNTY HEALTH DEPARTMENT
LAKE COUNTY CONSOLDTD. AQ MONIT. WRK. GRP .
AIR QUAL. DIV., MI DEPT. OF NAT. RES.
AIR POLL. CONT. DIV., WAYNE COUNTY
MINNESOTA POLL. CONT. AGENCY, AIR MO
OHIO EPA, CENTRAL DISTRICT OFFICE
C-2

-------
Region
05
05
05
05
05
05
05
05
05
05
05 ,
05
05
05
05
06
06
06
06
06
06
06
06
06
06
06
06
06
06
06
07
07
07
07
07
07
07
07
07
07
07
07
07
07
08
08
08
08

NO.
36-
36
36
36
36
36
36
36
36
36
36
36
36'
36
51
04
04
19
32
32
37
37
37
45
45
45
45
45
45
45
16
16
16
17
26
26
26
26
26
26
26
28
28
28
06
27
27
27
State
Name
OHIO •'"..-•
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
OHIO
WISCONSIN
ARKANSAS
ARKANSAS
LOUISIANA
NEW MEXICO
NEW MEXICO
OKLAHOMA
OKLAHOMA
OKLAHOMA
TEXAS
TEXAS
TEXAS
TEXAS
TEXAS
TEXAS
TEXAS
IOWA
IOWA
IOWA
KANSAS
MISSOURI
MISSOURI
MISSOURI
MISSOURI
MISSOURI
MISSOURI
MISSOURI
NEBRASKA
NEBRASKA
NEBRASKA
COLORADO
MONTANA
MONTANA
MONTANA

No.
/002,
003
004
005
006
007
008
009
010
012
013
014
015
016
001
001
002
001
001
002
101
102
103
001
002
003
004
005
006
007
001
002
003
001
001
002
003
004
005
006
007
001
002
003
001
001
002
003
Reporting Organization
, Name > .
OHIO EPA, NORTHEAST DISTRICT OFFICE
OHIO £PA, NORTHWEST DISTRICT OFFICE
OHIO EPA, SOUTHEAST DISTRICT OFFICE
OHIO EPA, SOUTHWEST DISTRICT OFFICE
AKRON AIR POLLUTION CONTROL
AIR POLL. CONT. DIV., CANTON CITY
SOUTHWESTERN OHIO AIR POLL. AGENCY
CLEVELAND DIV. OF AIR POLL. AGENCY
REGIONAL APC AGENCY, DAYTON
AIR POLL. CONT. DIV. OF LAKE COUNTY
AIR POLL. UNIT, PORTSMOUTH CITY
NORTH OHIO VALLEY AIR AUTHORITY
TOLEDO POLL. CONTROL. AGENCY
MAHONING TRUMBULL AIR POLL. CONTROL
WI. DEPT. OF NAT. RES., AIR MONIT. UNIT
DEPT. OF POLL. CONT. & ECOLOGY CONT. MON
DEPT. OF POLL. CONT. & ECOLOGY
DEPT. OF NATURAL RESOURCES, NEW ORLEANS
ENV. IMPROVEMENT DIV., SANTA FE
CITY OF ALBUQUERQUE ENV. HEALTH DIV.
OK STATE DEPT. OF HEALTH
.OKLAHOMA CITY-CNTY. HEALTH DEPT.
TULSA CITY-CNTY. HEALTH DEPT.
TEXAS AIR CONTROL BOARD
DALLAS ENV. HEALTH & CONSERVATION DEPT.
EL PASO CITY-CNTY. HEALTH DEPT.
FT. WORTH PUBLIC HEALTH DEPT.
GALVESTON COUNTY HEALTH DISTRICT
HOUSTON DEPT. OF PUBLIC HEALTH
SAN ANTONIO METRO. HEALTH DISTRICT
POLK COUNTY PHYSICAL PLANNING
LINN COUNTY HEALTH DEPARTMENT
UNIVERSITY HYGIENIC LABORATORY
STATE OF KANSAS
LABORATORY SERVICES PROGRAM
ST. LOUIS COUNTY
ST. LOUIS CITY
KANSAS CITY
SPRINGFIELD
AMAX LEAD CO. OF MO, BOSS, MO
ST. JOE LEAD CO., HERCULANEUM, MO
STATE OF NEBRASKA
LINCOLN
OMAHA
DEPARTMENT OF HEALTH
MT AIR QUAL. BUREAU, DEPT. OF H&ENV.
YELLOWSTONE CNTY . AIR POLL. CONT. AGY.
GREAT FALLS CITY-CNTY. HEALTH DEPT.
C-3

-------
Region
08
08
08
08
08
09
09
09
09
09
09
09
09
09
09
09
09
09
10
10
10
10

No.
27
3b
43
46
52
03
03
03
05
05
05
05
05
12
29
29
29
54
02
13
38
49
State
Name
MONTANA
NORTH DAKOTA
SOUTH DAKOTA
UTAH
WYOMING
ARIZONA
ARIZONA
ARIZONA
CALIFORNIA
CALIFORNIA
CALIFORNIA
CALIFORNIA
CALIFORNIA
HAWAII
NEVADA
NEVADA
NEVADA
GUAM
ALASKA
IDAHO
OREGON
WASHINGTON

No.
004
001
001
001
001
100
200
300
001
004
036
061
061
120
100
200
300
100
020
001
001
001
Reporting Organization
Name
MISSOULA CITY-CNTY HEALTH DEPT.
STATE DEPARTMENT OF HEALTH
DEPT. OF HEALTH, OIV. OF ENV. HEALTH
STATE BUREAU OF AIR QUALITY
DEPT. OF ENV. QUAL., AIR QUAL. DIV.
ARIZONA DEPT. OF HEALTH«SERVICES
MARICOPA COUNTY
PI MA COUNTY
CALIFORNIA AIR RESOURCES BOARD
BAY AREA AIR QUAL. MANAGEMENT DISTRICT
SAN DIEGO AIR POLL. CONTROL DISTRICT
SOUTH COAST AIR QUAL. MANAGEMENT DIST.
SOUTH COAST AIR QUAL. MANAGEMENT DIST.
STATE OF HAWAII, DEPT. OF HEALTH
NEVADA DIV. OF ENV. PROTECTION
WASHOE COUNTY
CLARK COUNTY
GUAM EPA
DEPT. OF ENVIRONMENTAL CONSERVATION
DEPARTMENT OF HEALTH AND WELFARE
DEPT. OF ENVIRONMENTAL QUALITY
DEPT. OF ECOLOGY
C-4

-------
                                 APPENDIX  D


           PRECISION AND ACCURACY DATA  BY  REPORTING  ORGANIZATIONS
     To reduce printing expenses,  the detailed  tabulations  of  the  numerical
values for each pollutant for each reporting organization  are  not included
here, but can be obtained by  written request to R.C. Rhodes,  EPA,  MD-77B,
Research Triangle Park, NC 27711.  The format  of the tables is the  same  as
for the previous annual reports.  Please indicate in your  request the  par-
ticular pollutant mesurement  system(s) you  desire copies  for.
                                    D-l

-------

-------
                                 APPENDIX E

     PROBLEMS INVOLVED•• IN  THE  COMPARISON  OF  PERFORMANCE  AUDIT  (PA)  DATA
                   AND PRECISION AND ACCURACY (PARS) DATA


     Several problems are encountered when attempting to compare Performance
Audit (PA) data and Precision  and Accuracy  (PARS) data.  Obviously, compari-
sons can  be made  only   where  the   same  pollutant  measurement  methods  are
audited in  both programs.  The  following pollutant  measurement  methods  are
audited in both programs.

             Continuous Methods       CO
                                      S02

             Manual Methods           TSP
                                      Pb
                                      S02
                                      N02

Further, only the accuracies of the PARS system  can be compared because no
precision assessments are currently made from the PA data.

     Other factors to consider in making comparisons are:

     1.  source of data  (organization performing the audits),
     2.  time of audit, and
     3.  concentration level (or flow rate level  for TSP).

     Valid comparisons can  only  be made  for  those  organizations where both
the PA  and  the PARS audits are performed.   The  PARS data are  reported  by
Reporting Organization,  whereas the PA  data  are  reported  by  Laboratory.

     A cross-reference listing has been prepared to match up each Reporting
Organization number with its corresponding Laboratory number.  The compari-
sons made on a state, regional, or national basis are made using only those
Reporting Organization-Laboratory match-ups  where  both have  reported accu-
racy audit data.

     Good agreement should  be  expected  between the PARS and  PA data for a
given Reporting Organization-Laboratory  combination  if the two  audits were
performed at  nearly  the  same time.  However,  the  PA audits  are scheduled
at various times during the year.   And,  the  regulation requirement for the
PARS accuracy audit  is  that (1)  at least one audit per year  shall  be con-
ducted on each instrument (or site) for continuous instruments  (CO and S02)
and for the  TSP  method   and (2)  at least two  audits  per quarter  shall  be
                                    E-l

-------
conducted at the laboratory for the manual Pb, S02, and  N0£  methods.   Fur-
ther, there is  no  requirement  or  planned schedules to assure  that  the two
types of audits are conducted at nearly the same  time.   The comparisons can
therefore be made  only  on  an annual  basis for a given  Reporting  Organiza-
tion-Laboratory matchup.   Comparisons  for the continuous  methods, CO  and
S02, and TSP cannot be made on an individual site (instrument) basis because
the PARS data  are  not  reported on a  site basis  although the  PA  data  are.
(Beginning January 1, 1987, these PARS data will  be reported to EMSL by site
so that it will be possible  to make comparisons  on a  site  basis.   However,
because of the possible large differences in times  of  the audits,  such com-
parisons may not be meaningful.)

     Because of the relatively small  amount of data for comparison on  a Re-
porting Organization-Laboratory basis  and the time differences, stifdy of the
comparisons of  PA  and  PARS data has been  limited to  comparisons  of larger
samples or  aggregates   of  data, i.e.,  on  a  Regional  or  National  basis.

     Another bothersome problem in  comparing  PA  and  PARS  data is that the
concentration  levels do not correspond.  The concentration  levels  are fixed
by regulation  for  the  PARS accuracy audits whereas the  levels  for PA vary
from year to year  and in some  cases from audit to  audit.  Because of these
variations in  concentration for the PA audits, the concentration levels for
PARS are used  as a  basis for defining  concentration  ranges for comparison.

     The following  tables  present  the  concentration  levels  for  PARS  as
specified by the  regulation  and the concentration  levels actually used for
PA audits during calendar year 1985.

                TABLE E-l.  CONCENTRATION LEVELS FOR PARS AND
                           PA AUDITS FOR  1985 FOR THE CONTIN-
                           UOUS METHODS
Pollutant
CO
SO 2
Concentration
PARS
3- 8
15-20
35-45
80-90
.03-. 08
.15-. 20
• O J "* • ^ J
.80-. 90
levels, ppm
PA
6.70
16.50
39.90
.05-. 08
.17-. 20
.22-. 26
.40- .49
                                                         .62-.69
                                     E-2

-------
         TABLE E-2.  CONCENTRATION (OR FLOW) LEVELS FOR
                     PARS AND PA AUDITS FOR 1985 FOR
                     MANUAL METHODS
Concentration (or flow) leve
Pollutant PARS
TSP ft3/min
50
(nominal)
40-60



Pb jig/strip
100- 300
600-1000




S02 yg/ml
0.2-0v3
0.5-0.6
0.8-0.9


N°2 yg/ml
0.2-0.3
0.5-0.6
0.8-0.9


m3min
1.416*

1.133-1.699



yg/m3
0.6-1.8*
3.5-5.9




PPm
.01 3-. ,020*
.033-. 040
.053-. 059


Ppm
.018-. 028*
.046-. 055
.074-. 083


m3/min
.7
.9
1.1
1.2
1.3
,-,. 1/85
yg/m3
0.53
1.06
3.03
4.31
4.83
6.65
yg/m3
44.30
61.00
90.60
124.50
271.90
yg/ml
.345
.434
.686
.944
1.114
Is
PA
ft3/min
24.7
31.8
38.8
42.4
45.9
7/85
yg/m3
0.45
1.00
1.15
2.00
2.70
5.40
PPM
.017
.023
.034
.0473
.103
yg/m3 ppm
59.90 .032
75.35 .040
119.10 .063
163.89 .087
193.40 .103
*See conversion factors on following page.
                              E-3

-------
                           Conversion Factors
      To convert

      ft3 to m3

      pg/m3 to ppm
         S02
         N02
         CO
         03

       pg/strip to  pg/m3  for Pb
          Multiply by

            0.02832
            0.00038
            0.00053
            0.00087
            0.00051

            0.00589
12 exposed
pg strips
3/4"x8" strip 8"xlO" filter 50
= .00589 pg/m3
1
ft3/min
1 filter/day
1440 min/day
1
.02832 m3/ft3
        pg/ml to
                                                         173.61
                                                  1
                1000 L

1440 min/day      m3
     pg     50 ml     1 sample/day

     ml     sample      .2 L/min


        =  173.61  pg/m3


pg/ml to ppm

     S02  (173.61)(0.00038)  = 0.066

     N02  (173.61)(0.00053)  = 0.092
     The following example illustrates the procedure for establishing the
concentration ranges for comparison purposes,  for CO the four accuracy
audit levels for the PARS are 3-8, 15-20, 35-45, and 80-90 ppm.  During
1985 the three concentration levels for the performance audits were 6.70,
16 5U  and 39.90 ppm.  The calculated midpoints between the adjacent con-
centration levels for the PARS are considered the boundaries of the ranges
for comparison:
                                    E-4

-------
Comparison
levels
1

2

3

4
Cone.
levels,
ppm
3-8

15-20

35-45

80-90
Calculated
midpoints,
ppm

11.5

27.5

62.5

Ranges for
comparison,
ppm
0-11.5

11.5-27.5

27.5-62.5

62.5-
Performance
audit levels,
ppm
6.70

16.50

39.90


     As shown above the calculated midpoint between 8, the upper limit of
PARS level 1, and 15, the lower limit of PARS level 2, is
Similarly the calculated midpoint between comparison levels 2 and 3 is  27.5
ppm, and between levels 3 and 4, 62.5 ppm.  Thus,  the newly defined CO
ranges for comparison are

                  0   to 11.5
                 11.5 to 27.5
                 27.5 to 62.5
       and       62.5 and above.

Therefore, the results.of PA audits at 6.70 ppm are compared with the re-
sults of PARS audits at 3 to 8 ppm, etc., shown in the above table.

     Following the same procedure, the comparison  ranges  for all  the pol-
lutant methods have been computed and are summarized in Tables E-3 and  E-4.

     The problem in comparing results within the defined  ranges are illus-
trated by Figures E-l and E-2.
                                    E-5

-------


DA
PA
I



PARS
PA
1

1
CO

—
I 1

1
502
' w-
•9-
} 0.

2


	
1.5 25

2

> ' m 	
•* — • •• •
115 0.2

3 4


i I
.5 50 62.5 100
ppm
3 I 4
• • ' • •
1 	 V " V | ^ •^
1
! i
75 0.5 0.625 1-0
ppm
Figure E-l.  Concentration levels  for comparing PARS and
             PA data, continuous methods.
                          E-6

-------
PARS
  PA

•,
1
• . , ,
• • • in
iii it


i i
   20
     Pb
PARS
  PA
30
                    2.65
40
                                  ft3/min

                                      2
50
60
1
1

LJB LJLJ LJ |_ !• • • LJ •
1
1 ill 1 1 1 1




1 1 1
                                                     10
PARS
  PA
     S02
• w 	 1 	 w w —
1
1
• • • 1 •
• • 1 •
1 ill 1
	 1 	 W 9 	
1
1
i 1 1 1 1
	
1 1
                    0.027
                0.047 0.05
                    ppm
                                        0.1
PARC
DA

i
N02 1
f A
^^"""•"^•^ .
I
) 0.0
2 I 3
• • 0 •
I
• • ' • •
• i{ mm
I I 111
37 0.05 0.065 0.1
ppm
    Figure E-2.  Concentration or  flow levels  for comparing PARS
                  and  PA data, manual  methods.
                                   E-7

-------
          TABLE E-3.   CONCENTRATION  RANGES FOR COMPARISON OF PARS
                       AND PA DATA - CONTINUOUS METHODS
                            PARS
                            cone.    Calculated     Ranges for       PA
             Comparison    levels,    midpoints,    comparison,    levels,
Pollutant     level	ppm         ppm	ppm  	ppm

  CO            1            3-8                        0-11.5       6.70
                                        11.5
                2           15-20                    11.5-27.5      16.50
                                        27.5
                3           35-45         "          27.5-62.5      39.90
                                        62.5
                4           80-90         '          62.5-


  SO?           1           .03-.08                      0-.115      .05-.08
                                         .115
                2           .15-.20                   .115-.275    C.17-.20
                                                                  1  .22-.26
                                         .275
                3           .35-.45       '            .275-.625      .40-.49
                                         .625
                4           .80-.90                   .625-
                                     E-8

-------
          TABLE E-4.  RANGES FOR COMPARISON OF PARS AND PA DATA -
                      MANUAL METHODS
             Comparison    PARS     Calculated    Ranges for
Pollutant	level	level    mid-point	comparison
                                               Performance
                                               audit levels
  TSP
  Pb
  S02
  N02
2

3


1

2
          40-60
         0.6-1.8
.013-.020


.033-.040

.053-.059


.018-.028

.046-.055


.074-.083
                                              ft3/min
             NA
    all
                                               pg/m;
                                         .027

                                         .047
                                         .037


                                         .065
                           0-2.65
                                        2.65
                          3.5-5.9                 2.65-
                                                 _PJI!L
   0-.027


.027-.047

.047-


   0-.037

.037-.065


.065-
 f
24.7
31.8
38.8
42.4
45.9
                                                                  1/85    7/85
                .53
               1.06
                                                 3.03
                                                 4.31
                                                 4.83
                                                 6.65
.017
.023

.034

.0473
.103

.032

.040
.063

.087
.103
        .45
       1.00
       1.15
       2.00

        !.70
        i.40
                                    E-9

-------
     As shown in  Table  E-3, the  results of  level  2  PARS  continuous  S02
accuracy audits  at concentrations  .15-.20 ppm are compared with the results
of performance  audits  at  concentration  levels  .17-.20  and   .22-.26  ppm.

     And, from Table E-4, the results of level  1 PARS Pb accuracy audits at
concentrations 0.6-1.8 pg/m3 are  compared with the results of performance
audits at concentration  levels  .53  and 1.06  pg/m3 of  the  1/85  audit  and
.45, 1.00, 1.15, and 2.00 pg/m3 of the 7/85 audit.

     It has been recommended that  the concentration levels for the perform-
ance audits be  adjusted  to  more closely correspond to those of the PARS in
order to provide more valid comparisons.

     Another factor  which  makes  the PA  and  PARS  comparisons  somewhat com-
plicated is the reporting units which differ for some of the pollutant mea-
surement methods  and  which require the  conversion  of units.   Further, the
persons submitting  data  are required  to convert some  informational  items
to computer  codes:   methods, units, laboratory  names/addresses, reporting
organization names/addresses, audit levels (for PARS only),  etc.   These con-
versions could  be the source of some errors  in proper identification of the
data used in making the  comparisons.
                                     E-10

-------
                                 APPENDIX  F


               COMPARISON OF  PARS AND  PERFORMANCE  AUDIT DATA
     To reduce printing expenses,  the  detailed  tabulations  of the numerical
values of this appendix are not  included  here,  but  can  be obtained by writ-
ten request to R.C. Rhodes, EPA, MD-77B, Research Triangle Park, NC 27711.
The format of  the  tables  is the same  as  for the previous annual reports.
Please indicate in your request the  particular  pollutant  mesurement  sys-
tem(s) you desire copies for.
                                    F-l

-------