vvEPA
United States
Environmental Protection
Agency
Atmospheric Research and
Exposure Assessment Laboratory
Research Triangle Park, NC 27711
EPA/600/3-90/004
February 1990
Research and Development
National Performance
Audit Program
Acid Rain Audits
1988
-------
EPA/600/3-90/004
February 1990
NATIONAL PERFORMANCE AUDIT PROGRAM
ACID RAIN AUDITS
-1988-
Robert L. Lampe and William J. Mitchell
Quality Assurance Division
Atmospheric Research and Exposure Assessment Laboratory
Research Triangle Park, North Carolina 27711
ATMOSPHERIC RESEARCH AND EXPOSURE ASSESSMENT LABORATORY
OFFICE OF RESEARCH AND DEVELOPMENT
U.S. ENVIRONMENTAL PROTECTION AGENCY
RESEARCH TRIANGLE PARK, NORTH CAROLINA 27711
-------
NOTICE
This document has been reviewed in accordance with U.S. Environmental
Protection Agency policy and approved for publication. Mention of trade names
or commercial products does not constitute endorsements or recommendation for
use.
-------
ABSTRACT
This report presents results of the Environmental Protection Agency's
Semiannual U.S. Acid Rain Audit Program for 1988. When these audits, which are
a part of the National Performance Audit Program (NPAP), were initiated in 1981,
participation was limited to laboratories routinely analyzing acid rain samples.
However, in 1988 invitations to participate were issued to all NPAP participants.
As a result, only 52% of the results received for the 1988 audits came from
laboratories that had been regular participants in the previous years' audits.
Despite the many new participants, the percentage of participants who
analyzed for a specific analyte was similar to that from previous audits*. Also,
as in previous audits, many participants had difficulty measuring accurately the
concentration of the lowest cation and anion concentrations. Not unexpectedly,
the percentage of the reported results identified as outliers was larger than
in the 1985-1987 audits. The results for the heavy metals (Mn, Fe, Cd, Cu, Ni,
Pb and Zn) were similar to earlier audits in that most of the participants
reported an average result (for all seven metals) close to the expected value.
iii
-------
CONTENTS
Page
Abstract , iii
Tables v
Acknowledgments vi
Introduction 1
Procedure 2
Results and Discussion 3
References 5
v
-------
TABLES
Number Page
1 6
2 7
3 8
4 9
5 10
6 11
7 12
8 13
9 14
10 15
VI
-------
ACKNOWLEDGMENTS
Recognition is due to the technical staff of NSI Technology Services Inc. ,
Research Triangle Park, NC, who produced the high quality samples used in these
audits. Also, we thank the staff of Global Geochemistry Inc., Canoga Park, CA,
the referee laboratory.
vii
-------
SECTION 1
INTRODUCTION
The National Performance Audit Program (NPAP) is conducted by the
Atmospheric Research and Exposure Assessment Laboratory (AREAL) of the U.S
Environmental Protection Agency (EPA) at Research Triangle Park, NC. NPAP
provides EPA management a means to assess the analytical proficiency of state
and local laboratories. The acid rain audits, which are a part of the NPAP
program, are open to governmental, academic and commercial organizations.
Participation is voluntary.
When our acid rain audit program began in 1975 it was restricted to members
of the World Meteorological Organization (WHO). In 1981, U.S. participants of
the NPAP who were known to be routinely analyzing acid rain samples were invited
to participate in an acid rain audit conducted only for U.S. laboratories.
Thirty-two laboratories accepted the invitation and 18 actually reported their
results.
In 1988, participation in the U.S. acid rain audit program was opened to
all NPAP participants. One hundred and fourteen agencies accepted the invitation
for the May 1988 (0588) audit and 118 accepted the invitation for the October
1988 (1088) audit. However, only 57 of these laboratories (50%) returned results
for the 0588 audit and only 51 (43%) returned results for the 1088 audit. We
believe these low levels of participation in the 1988 audits resulted because
many agencies that requested to participate were unfamiliar with the extensive
analyses required for precipitation samples. When they received the samples they
decided not to analyze them. Only 52% of the results reported came from
laboratories who had been regular participants in the previous audits.
-------
SECTION 2
PROCEDURE
The five samples used in the audit are contained in polyethylene containers
to improve sample stability. The participating laboratories analyze three of
the samples for pH, conductivity, acidity and the cations and anions normally
measured in precipitation samples. The other two samples, which are acid
stabilized to prevent loss of metals from the solution, are analyzed for heavy
metals.
The samples are prepared using formulations developed by the National
Institute of Standards and Technology (NIST) and certified by an independent
analytical (referee) laboratory using NIST reference materials. The participants
analyze the samples using the analytical procedures they normally employ when
analyzing their precipitation samples. Each laboratory is shipped a set of
samples consisting of one sample from each of the five series. When diluted
1:50, each sample simulates a precipitation sample.
The participants dilute each sample, analyze it in triplicate and average
the results for each analyte. They also analyze at least one blank of the
distilled water they used to dilute the samples to correct their results for any
contribution from the distilled water. All results are then forwarded to EPA
for summarization and evaluation. Each participant is then sent a report that
compares his results to the expected values for each analyte. At the end of the
year, this summary report is prepared so that all participants can see how their
performance compared to that of the other laboratories.
-------
SECTION 3
RESULTS AND DISCUSSION
The data and statistics in this report are descriptive of the quality of
the analytical data currently being reported by agencies measuring acid rain.
At present, there are no federal regulations specifying the accuracy required
in the analysis of acid rain samples. Therefore, we recommend that individual
laboratories evaluate their progress and improve their analytical capabilities
as they feel necessary based on their own results and the averages reported in
this report for the 0588 and 1088 acid rain audits.
Less than 35% of the 1988 audit participants analyzed all samples for all
the analytes. The approximate percentage of participants who reported results
for each analyte or class of analyte (minor ions, heavy metals) are pH and
conductivity, 94%; acidity, 26%; sulfate and nitrate, 69%; minor anions (F, Cl),
31%; minor cations (NH4, Ca, Mg, Na, K), 46%; and heavy metals (Mn, Fe, Cd,
Cu, Ni, Pb, Zn), 33%. These percentages are generally lower than those for the
previous audits.
The number of audit participants is small and less than 50% of the
participants analyze for the minor anions (Cl, F), cations (NH^, Ca, Mg, Na, K)
and heavy metals (Mn, Fe, Cd, Cu, Ni, Pb, Zn). Further, the concentrations of
many of the analytes are near or below the quantification limits of the
analytical methods, so many participants report their results to only one
significant figure (despite our request to report them to two significant
figures). For the above reasons, it is difficult to obtain a sufficient number
of results on which to perform an in-depth statistical analysis. For example,
suppose a laboratory obtains a value of 0.14 mg per L for Na when the true value
is 0.15 mg per L but only reports a value of 0.1 mg per L. The apparent error
is -33% but in actuality it is only -7%.
Also, in previous audits, we had observed that many participants had
relatively large positive and negative percent differences for analytes in the
same general class (anions, cations, metals) even though the same analytical
method was used to determine all the analytes in that class. It was also
observed that participants who had participated in many of the previous audits
tended to show large variations in the magnitude and size of the percent
difference for the same analyte from one audit to the next. This variation was
quite randomly distributed.
To remove as much of these random variations as possible from the data,
it was decided to calculate a percent difference by lavel for each laboratory
for each class of analyte rather than calculate a percent difference for each
analyte within each class of analyte (the approach used in earlier reports).
This was accomplished by determining the sum of the concentrations for all the
analytes in a specific class for the expected and for the reported values. The
difference between the two sums was then determined and the percent difference
--(reported-expected) 100/expected--was calculated.
-------
In the case of the minor anions and the minor cations almost all
participants who reported results for one analyte in these classes reported
results for all the analytes in these classes. Thus, the percent difference for
each participant for each analyte class was used to calculate the average percent
difference for that analyte class, i.e., cation, anion.
However, because a significant number of the participants who reported
results for the metals did not analyze for all seven of the metals, it was
decided to report each participant's average percent difference and not to
combine them to obtain a grand average.
Tables 1 and 3 present the results reported (n) for all analytes except
the heavy metals for the 0588 and 1088 audits, respectively. Tables 2 and 4
present the results for thes^e two audits after outliers were removed using an
outlier rejection method that is based on the standard deviation historically
obtained for these audits. This procedure considers all percent differences that
exceed 3 times the historical standard deviation (10% for these audits) to be
outliers (1). For information purposes, the concentrations of the individual
anions and cations in the 1988 samples are presented in Table 5.
Tables 6 through 9 present the results for the 0588 and 1088 audit samples
that contained the heavy metals. Tables 6 and 8 present the results reported
by all participants for the 0588 and 1088 audits, respectively. Tables 7 and
9 present the results after outliers were removed. For information purposes,
the concentrations of the individual metals in each 0588 and 1088 audit sample
is presented in Table 10.
A comparison of Tables 1 and 2 and a comparison of Tables 3 and 4
demonstrate that quite a few participants encountered difficulty measuring
acidity and the cations and anions. The number of outliers reported is larger
than those reported for 1985, 1986 and 1987. It is quite possible that this
increase resulted because many laboratories were analyzing this type of synthetic
precipitation sample for the first time. Also, the lower the concentration of
the analyte, the higher percentage of the results that were rejected as outliers.
This is consistent with the previous audits. Unfortunately, a majority of
precipitation samples contain ions at these levels.
Inspection of the average results for 1985, 1986 and 1987 (2,3,4) and those
in this report for the analytes most frequently measured show no trend in the
performance obtained from audit to audit.
Inspection of the individual laboratory results for the heavy metals
(Tables 6,7,8 and 9) shows that most laboratories obtained results that agreed
very well with the expected value in both 1988 audits. When a laboratory
reported results that differed markedly from the expected value in one audit,
it generally showed good agreement between the expected and reported values in
the other 1988 audit.
-------
REFERENCES
1. Duncan, A.J. Quality Control and Industrial Statistics. R.D. Erwin and
Co., Inc., 1967. P. 89.
2. Parr, B.F., R.L. Lampe, G. Pratt, E.T. Hunike ndW.J. Mitchell. National
Performance Audit Program. Ambient Air Audits of Analytical Proficiency,
1987. U.S. Environmental Protection Agency Report EPA 600/3-89-006.
Research Triangle Park, North Carolina 27711. January 1989.
3. Parr, B.F. , R.L. Lampe, G. Pratt, O.L. Dowler andW.J. Mitchell. National
Performance Audit Program. Ambient Air Audits of Analytical Proficiency,
1986. U.S. Environmental Protection Agency Report EPA 600/4-87-038.
Research Triangle Park, North Carolina 27711. December 1987.
4. Parr, B.F., R.L. Lampe, G. Pratt, O.L. Dowler andW.J. Mitchell. National
Performance Audit Program. Ambient Air Audits of Analytical Proficiency,
1985. U.S. Environmental Protection Agency Report EPA 600/4-87-002.
Research Triangle Park, North Carolina 27711. January 1987.
-------
Table 1. Results Reported For All Analytes Except
Heavy Metals—0588 Audit
Audit Analyte
0588 pH
Conductivity
,i S/cm
Acidity
ueq/L
804
mg S/L
N03
mg N/L
Anions (Gl, F)
mg/L
Cations
(NH4, Na, Ca,
Mg, K)
mg/L
Sample
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
N
52
52
52
48
48
48
17
17
17
38
40
40
38
40
40
19
21
20
22
25
22
Average
Reported
Value
4.37
3.53
3.55
20.98
148.00
133.00
56.82
262.00
268.00
0.80
4.48
3.37
0.38
2.24
1.89
0.43
3.03
1.45
0.50
4.00
1.15
Expected
Value
4.40
3.50
3.52
16.50
156.00
130.00
41.10
312.00
292.00
0.66
3.91
2.89
0.11
2.14
1.81
0.33
3.21
1.42
0.42
3.86
1.02
% Diff.
-0.68
0.86
0.85
27.15
-5.13
2.30
38.24
-16.02
-8.22
21.21
14.58
16.61
245.45
4.67
4.42
30.30
-5.61
2.11
19.05
3.63
12.75
-------
Table 2. Table 1 Results After Outliers Removed
Audit Analyte
0588 pH
Conductivity
}i S/cm
Acidity
Meq/L
S04
mg S/L
N03
tag N/L
Anions (Cl, F)
mg/L
Cations
(NH4, Na, Ca,
Mg, K)
mg/L
Sample
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
N
51
52
52
42
43
46
5
12
11
31
36
35
34
38
38
11
19
19
18
24
18
Average
Reported
Value
4.40
3.53
3.55
18.44
147.00
127.00
40.14
342.00
319.00
0.66
4.02
2.96
0.11
2.13
1.81
0.33
3.19
1.49
0.45
3.93
1.10
Expected
Value
4.40
3.50
3.52
16.50
156.00
130.00
41.10
312.00
292.00
0.66
3.91
2.89
0.11
2.14
1.81
0.33
3.21
1.42
0.42
3.96
1.02
% Diff.
0.00
0.86
0.85
11.76
-5.76
-2.31
-2.34
9.62
9.25
0.00
2.81
2.42
0.00
-0.47
0.00
0.00
-0.62
4.93
7.14
1.81
7.84
-------
Table 3. Results Reported For All Analytes Except
Heavy Metals—1088 Audit
Audit Analyte
1088 pH
Conductivity
iiS/cm
Acidity
ueq/L
S04
ng S/L
N03
mg N/L
Anions (Cl, F)
mg/L
Cations
(NH4, Na, Ca,
Mg, K)
mg/L
Sample
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
N
49
49
48
43
43
43
11
11
11
34
33
35
35
34
34
16
13
15
27
26
26
Average
Reported
Value
3.94
4.25
3.71
62.37
23.94
86.64
116.00
64.78
192.00
2.55
1.07
4.80
1.04
0.14
0.14
1.29
0.38
0.66
2.46
0.47
1.09
Expected
Value
3.91
4.27
3.68
66.14
24.08
96.43
124.00
53.67
209.00
2.11
0.90
4.01
0.87
0.13
0.11
1.27
0.32
0.43
2.38
0.41
0.99
% Diff.
0.77
-0.47
0.82
-5.70
-0.58
-10.15
-6.45
20.70
-8.13
20.85
18.89
19.70
19.54
7.69
27.27
1.58
18.75
53.48
3.36
14.63
10.10
-------
Table 4. Table 3 Results After Outliers Removed
Audit Analyte
1088 pH
Conductivity
,j S/cm
Acidity
lieq/L
S04
mg S/L
N03
mg N/L
Anions (Cl, F)
mg/L
Cations
(NH4> Na, Ca,
Mg, K)
mg/L
Sample
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
N
49
48
48
42
41
39
8
6
8
29
30
30
32
29
31
13
8
11
27
22
21
Average
Reported
Value
3.94
4.30
3.71
62.93
24.85
93.53
132.00
53.43
235.00
2.12
0.89
4.05
0.87
0.12
0.11
1.32
0.32
0.45
2.46
0.43
1.04
Expected
Value
3.91
4.27
3.68
66.14
24.08
96.43
124.00
53.67
209.00
2.11
0.90
4.01
0.87
0.13
0.11
1.27
0.32
0.43
2.38
0.41
0.99
% Diff.
0.77
0.70
0.82
-4.85
3.20
-3.00
6.45
-0.45
12.44
0.47
-1.11
1.00
0.00
-7.69
0.00
3.44
0.00
4.65
3.36
4.88
5.05
-------
Table 5. Concentrations of the Anions and Cations in Each
1988 Audit Sample8
Cl
F
NH4
Na
Ca
Mg
K
Sample 1
0.280
0.051
0.078
0.186
0.047
0.021
0.084
0588
Sample 2
2.738
0.474
0.835
1.771
0.388
0.063
0.798
Sample 3
1.254
0.167
0.471
0.393
0.038
0.042
0.078
Sample 1
1.078
0.191
0.335
1.340
0.115
0.068
0.524
1088
Sample 2
0.283
0.039
0.080
0.179
0.047
0.028
0.073
Sample 3
0.349
0.084
0.610
0.235
0.046
0.017
0.077
Expressed as mg analyte per L except for NH^ which is mg N/L.
10
-------
Table 6. Heavy Metal Results for Sample 4 of 0588 Audit
Laboratory
317003
320001
334001
335002
336001
339001
342002
345001
347003
406001
412001
436007
501001
506012
508016
517008
525009
529005
542010
548005
Na
7
7
7
7
7
7
7
7
7
4
7
7
7
5
7
7
6
7
7
5
Reported
Valueb
0.97
0.94
0.96
0.92
0.97
0.96
1.02
1.04
0.92
0.78
0.40
1.00
1.01
0.85
0.99
0.93
0.82
0.95
0.99
0.80
Expected
Value
0.96
0.96
0.96
0.96
0.96
0.96
0.96
0.96
0.96
0.82
0.96
0.96
0.96
0.83
0.96
0.96
0.85
0.96
0.96
0.83
% Diff.
1.04
-2.08
0.00
-4.17
1.04
0.00
6.25
8.33
-4.17
4.88
-58.33
4.17
4.99
2.41
3.13
-3.13
-3.53
-1.04
3.13
-3.61
reported.
aN - the number of metals for which results were reported.
average concentration (mg per L) for all metals for which results were
11
-------
Table 7. Heavy Metal Results for Sample 5 of 0588 Audit
Laboratory
317003
320001
334001
335002
336001
339001
342002
345001
347003
406001
412001
436007
501001
506012
508016
517008
525009
529005
542010
548005
Na
7
7
7
7
7
7
7
7
7
2
7
7
7
5
7
7
6
7
7
5
Reported
Valueb
0.33
0.34
0.33
0.32
0.33
0.34
0.36
0.40
0.32
0.17
0.32
0.39
0.35
0.26
0.35
0.32
0.26
0.33
0.36
0.26
Expected
Value
0.33
0.33
0.33
0.33
0.33
0.33
0.33
0.33
0.33
0.17
0.33
0.33
0.33
0.26
0.33
0.33
0.27
0.33
0.33
0.26
% Diff.
0.00
3.03
0.00
-3.03
0.00
3.03
9.09
21.21
-3.03
0.00
-3.03
18.18
6.06
0.00
6.06
-3.03
-3.70
0.00
9.09
0.00
an - the number of metals for which results were reported.
''The average concentration (mg per L) for all metals for which results were
reported.
12
-------
Table 8. Heavy Metal Results for Sample 4 for 1088 Audit
Laboratory
334001
335002
336001
342002
347003
353001
406001
412001
436007
501001
508016
521004
527002
529005
542010
Na
7
7
6
6
7
1
6
7
7
7
7
6
7
7
6
Reported
Value6
0.85
0.80
0.81
0.83
0.82
0.13
0.79
0.86
0.85
0.87
0.84
0.79
0.91
0.83
0.77
Expected
Value6
0.79
0.79
0.76
0.76
0.79
0.09
0.71
0.79
0.79
0.79
0.79
0.76
0.79
0.79
0.76
% Diff.
7.59
1.27
6.58
9.21
3.80
44.44
11.27
8.86
7.59
10.13
6.33
3.95
15.19
5.06
1.32
aN = the number of metals for which results were reported.
''The average concentration (mg per L) for all metals for which results were
reported.
13
-------
Table 9. Heavy Metal Results for Sample 5 of 1088 Audit
Laboratory
334001
335002
336001
342002
347003
353001
406001
412001
436007
501001
508016
521004
527002
529005
542010
Na
7
7
6
5
7
1
6
5
7
7
7
5
7
6
5
Reported
Valueb
0.43
0.40
0.39
0.40
0.41
0.41
0.40
0.45
0.43
0.43
0.40
0.38
0.41
0.42
0.40
Expected
Value6
0.41
0.41
0.38
0.38
0.48
0.07
0.40
0.38
0.41
0.41
0.41
0.38
0.41
0.40
0.38
% Diff.
4.88
-2.44
2.63
5.26
-14.58
14.28
0.00
18.42
4.88
4.88
-2.44
0.00
0.00
5.00
5.26
8N - the number of metals for which results were reported.
The average concentration (mg per L) for all metals for which results were
reported.
14
-------
Table 10. Concentrations of the Heavy Metals in Each 1988
Audit Sample in mg Analyte/L
0588 1088
Sample 4 Sample 5 Sample 4 Sample 5
Mn
Fe
Cd
Cu
Ni
Pb
Zn
0.034
0.082
0.034
0.070
0.022
0.110
0.610
0.019
0.049
0.019
0.029
0.014
0.054
0.142
0.030
0.081
0.030
0.056
0.022
0.089
0.482
0.019
0.007
0.018
0.041
0.018
0.067
0.238
t. U S. Government Printing Office 1990-768-159/00396
15
------- |