600A96020
The National Performance Audit Program (NPAP)
Elizabeth T, Hunike
U.S. Environmental Protection Agency
National Exposure Research Laboratory (MD-77B)
Research Triangle Park, North Carolina 27711
David R. Musick
U.S. Environmental Protection Agency
Office of Air Quality Planning and Standards (MD-14)
Research Triangle Park, North Carolina 27711
Joseph B. Elkins, Jr.
U.S. Environmental Protection Agency
Office of Air Quality Planning and Standards (MD-14)
Research Triangle Park, North Carolina 27711
-------
96-FA150.0I
INTRODUCTION
The Nation's ambient air monitoring program contains monitors for the six criteria pollutants for which
national ambient air quality standards (NAAQS) have been designated. These pollutants are carbon
monoxide (CO), lead (Pb), nitrogen dioxide (NO2), ozone (O,), particulate matter smaller than 10 microns
(PM-10), and sulfur dioxide (SO2), The standards for the criteria pollutants are shown in Table 1.
There are approximately 4,682 air pollution monitors in the ambient air network. These monitors
comprise the State and Local Air Monitoring Stations (SLAMS), the National Air Monitoring Stations
(NAMS), and the Prevention of Significant Deterioration (PSD) sites. The distribution of monitors by
pollutant is: S02,613; CO, 497; NO2, 312; 03> 822; Pb, 426; and PM-10,2012. The data from these
monitors are reported to EPA under the Aerometric Information Retrieval System (AIRS). Although the
data from these monitors are used primarily for determining compliance with the NAAQS, they also
support numerous other regulatory and research activities.
The quality assurance/quality control program for these monitors has three major components: the EPA
Regional Office Systems Audits; the Precision and Accuracy Reporting System (formerly PARS); and the
National Performance Audit Program (NPAP). The NPAP is a cooperative effort between EPA's
National Exposure Research Laboratory (NERL), the 10 EPA Regional Offices, and the 170 state and
local agencies that operate the SLAMS/NAMS air pollution monitors. Also included in the NPAP are
approximately 135 organizations (governmental and private) that operate air monitors at PSD sites.
Participation in the NPAP is required for agencies operating SLAMS/NAMS and PSD monitors as per
Section 2.4 of 40 CFR Part 58, Appendix A and Section 2.4 of 40 CFR Part 58, Appendix B. The NPAP
is operated by the Quality Assurance Branch of NERL. Participation in the NPAP program is also
mandatory for the 22 agencies which monitor for photochemical oxidants under EPA's Photochemical
Assessment Monitoring (PAMS) program. These agencies monitor for carbonyl compounds, volatile
organic compounds, Nox and ozone.
The NPAP's goal is to provide audit materials and devices that will enable EPA to assess the proficiency
of agencies that are operating monitors in the SLAMS/NAMS and PSD networks. To accomplish this, the
NPAP has established acceptable limits or performance criteria, based on the data quality needs of the
SLAMS/NAMS and PSD requirements, for each of the audit materials and devices used in the NPAP.
Any device or material not meeting these criteria is not used in the program.
All audit devices and materials used in the NPAP are certified as to their true value, and that certification
is traceable to a National Institute of Standards and Technology (NIST) standard material or device
wherever possible. The audit materials used in the NPAP are as representative and comparable as
possible to the calibration materials and actual air samples used and/or collected in the SLAMS/NAMS
and PSD networks. The audit material/gas cylinder ranges used in the NPAP are specified in the Federal
Register (Table 2).
The objectives for the NPAP audits are two-fold: (1) to complete at least 95% of the scheduled audits by
the end of the year, and (2) to determine if the participants' performance exceeds the limits shown below.
-------
96-FA150.01
Audit EPAdeterminedlimitg
High volume/PM-10 (SSI) % difference > ± 15% for 1 or more flows
Dichot (PM-10) % difference > ± 15% for 1 or more flows
Pb (analytical) % difference > ± 15% for 1 or more levels
SO2, NO2, O3 and CO Mean absolute % difference > 15%
PAMS The EPA determined limits were still being reviewed
and developed at the time of this paper's publication
The NPAP audits are accomplished using a variety of mailable audit systems. The participants use these
audit systems to generate pollutant concentrations and flowing air streams which are introduced into their
sampling system. The pollutant concentrations and air stream flow rate are unknown to the audit
participants. The outputs from the sampler that result from the use of the audit system are recorded on a
data form, returned to EPA, and compared to the concentration or flow rate that should have been
generated by the audit system under the environmental conditions at the site. The differences between the
EPA expected (certified) values and the NPAP participants' reported values are calculated and returned to
the participant.
DESCRIPTION OF NPAP AUDIT MATERIALS/DEVICES
High-VoIume/PM-10 (SSI) Flow Audits
The reference flow (ReF) device used for the high volume flow audit consists of a modified orifice, a
wind deflector, a manometer, and five resistance plates. The ReF for the PM-10 (SSI) flow audit is
similar except a filter is used as the only resistance.
Sulfur Dioxide/Carbon Monoxide (GDS) Audits
The GDS consists of a dilution device, a zero air. generator and a cylinder of gas containing
appproximately 30 ppm sulfur dioxide and 3000 ppm carbon monoxide.
Ozone (TECO 165) Audit
The audit device is self-contained with its own zero air and ozone generation system.
Lead Audit
The samples are 1.9 cm wide and 20 cm long glass fiber filter strips that have been spiked with an
aqueous solution of lead nitrate and oven-dried. Two filter strips comprise a sample,
Dichotomous (PM-10) Flow Audit
The audit device consists of a laminar flow element (LFE), an inclined manometer, an altimeter, and a
small dial thermometer. It measures fine flow (15,001pm) and total flow (16.7 1pm).
-------
96-FA150.01
Ozone/Nitrogen Dioxide/Sulfur Dioxide/Carbon Monoxide (TECO 175) Audit
The audit device is a combination of the TECO 165 and the GDS audit systems. It uses the same zero air
generation system as the GDS, the ozone generation system of the TECO 165, and a gas cylinder
containing approximately 3000 ppni carbon monoxide, 30 ppm sulfur dioxide and 30 ppm nitric oxide.
The ozone generation system is used with the pollutant gas to convert nitric oxide to nitrogen dioxide via
a gas phase titration. The TECO 175s were introduced into the NPAP in the fall of 1994; however, flow
stability problems delayed full utilization of the equipment. The problems were traced to the pre-set
regulator and replacement was completed in time for the 1995 audits.
PAMS Volatile Organic Compound (VOC) Audit
This audit uses a gas transfer system (GTS), stock (concentrated) compressed gas mixtures containing
PAMS compounds and 1.5L compressed gas (audit) cylinders. The stock mixtures are mixed and diluted
using the GTS and the resulting mixture is placed in the 1.5L audit cylinders. These audit cylinders are
pressurized to 500 psi to yield recoverable gas volumes of 40 to 60 L. Three audits are scheduled for each
year. Each of me 22 PAMS agencies receives one cylinder for each audit. The cylinders contain between
15 and 35 PAMS analytes at concentrations from 10 to 60 ppbv as carbon. The PAMS VOC audit was
added to the NPAP in 1995.
PAMS Carbonyl Compound Audit
This audit uses three glass tubes containing DNPH which have been spiked with solutions containing
acetone, formaldehyde and acetaldehyde. Each tube contains from 0.2 to 10 micrograms of each carbonyl
compound. The audit is conducted on the same schedule as for the PAMS VOC audit. Each PAMS
agency recovers the carbonyl compounds from the three DNPH tubes and reports the results to EPA. The
PAMS carbonyl audit was added to the NPAP in 1995.
SITE SELECTION
Historically, the State and local agencies have been allowed to select the NPAP sites to be audited. The
1989 General Accounting Office (GAO) audit raised concerns about the NPAP site selection process. The
GAO believed there could be a possible bias in the NPAP data base because (1) NPAP participants
selected the samplers to be audited, and (2) NPAP participants may have performed unscheduled
calibrations on samplers prior to the audits. In 1991 EPA sent an audit team to 79 SLAMS monitoring
stations located in all ten EPA regions. The audit team used the equipment and procedures of the NPAP
to assure that their audits simulated as closely as possible a normal NPAP audit. To ensure that the
SLAMS agencies did not take any special precautions, the audit team notified the agencies only 1 to 2
days prior to arrival. Also, the site log books were checked by the audit team to verify that the samplers
had not been calibrated prior to the audit. In all 172 samplers were audited including samplers for CO
(53), NO-NO2 (25), SO2 (38) and 03 (34). Ninety- seven of the samplers had been audited in the NPAP
during 1989 - 1990, but the rest had not. Based on a statistical analysis of the data from the 1991 study
and the 1989 and 1990 NPAP, EPA concluded that (1) overall the NPAP participants were auditing their
samplers properly and not conducting special unscheduled calibration tests, and (2) the assumption that
the NPAP data base was biased because the participants selected the sampler to be audited was not
supported. EPA additionally responded to the GAO comments by developing site selection criteria (Table
4) that were incorporated into the 1993 NPAP. Priority 1 site selection criteria should be audited
annually; priority 2 at least once every 2 years; priority 3 at least once every 3 years; priority 4 at least
-------
96-FA150.01
once every 4 years. All other sites should be audited at least once every 5 years. Based on this criteria,
EPA now selects specific sites that are to be audited in the NPAP. The criteria are reviewed annually, and
site selection is updated accordingly,
RESULTS
The number of NPAP audits performed from 1991 through 1995 is shown in Table 3.
In 1995,100% of the scheduled audits were completed with the exception of NO2 (63%), At the present
time, EPA does not have enough equipment to meet the demand for N02 audits. Due to the current
budgetary constraints, this may be a long term problem.
The percentage of 1995 NPAP participants whose performance fell within the EPA guidelines of 15% of
the certified values was: CO, 97%; S02,96%; NO, 97%; N02> 87%; O3,97%; hi-vol/PM-10 (SSI), 95%;
dichot (PM-10), 70%; and Pb (analytical), 96%. These percentages have remained similar since 1991 for
O3, SO2, CO, hi-vol/PM-10 (SSI) and Pb. NO (91% to 97%) and NO2(49% to 87%) have increased
substantially which may be attributable to the improved NPAP audit equipment (TECO 175). PM-10
(dichotomous) (75% to 70%) has decreased slightly.
CONCLUSIONS
EPA's Quality Assurance Guidance mandates that all data collected for regulatory or research purposes be
of known and documented quality. The NPAP program is critical to the National monitoring network
because it establishes the quality of the environmental data and provides a basis for improvements by
identifying monitors and monitoring programs that are not producing data of adequate quality. EPA
benefits from this program because its policy and decision makers receive continuously, an independent
assessment of the quality of the monitoring data and because compliance decisions are based on
defensible information. EPA also benefits because it frequently uses data from the National monitoring
network to support and/or supplement important research projects such as CASTNET, the Great Lakes
Monitoring Program and the particulate matter exposure/epidemiology studies which are addressing
current health concerns. The program also directly benefits EPA by providing an enormous spacial and
temporal criteria pollutant data base that is useful to virtually all EPA research projects that involve the
collection of atmospheric pollutant data. Specific examples of how EPA uses these data include; as
information for selecting and characterizing sites for research projects that require both urban and non-
urban field monitoring; for complementing research monitoring networks with criteria pollutant
information that is critical to the assessment and interpretation of field monitoring measurements; and
evaluating/validating modeled estimates of the transport and dispersion of pollutants released from
emission sources.
The data resulting from the NPAP audits are also used extensively by EPA to evaluate the performance of
existing and emerging monitors for measuring the criteria pollutants. The use of the National monitoring
network data to support EPA's research is only viable if the quality of the data is of known and
documented quality and the NPAP is the only mechanism available that provides this information.
-------
96-FA 150.01
EPA uses the NPAP to independently quality assure the SLAMS, NAMS, PSD, and PAMS monitoring
data it is receiving and permanently storing on its Aerometric Information Retrieval System (AIRS). The
AIRS is the conduit for which the research community, commercial and industrial communities, and the
public at large, gain access to the EPA's monitoring data. EPA also uses the NPAP as its only available
national, independent mechanism for continually assessing the quality of the ambient air quality data
necessary to develop and defend its research initiatives.
The cornerstone of any data collection system is the quality assurance component. The data utilization
resulting from the Nation's air monitoring network continues to increase in importance. The strategies
developed from the information can cost millions of dollars. Henceforth, we must remain vigilant in our
efforts to maintain the integrity of this important data set. In these efforts, the NPAP continues to be
refined. The NPAP has expanded to include all the criteria pollutants, volatile organic compounds and
carbonyl compounds. The associated instruments and equipment have been improved to incorporate the
latest technologies. The NPAP continues to respond to comments from the GAO audits as well as state
and local agency contacts. With the increased interest in the data from the Nation's air monitoring
community, it was inevitable that the site selection process would also be modified. These modifications
were incorporated into the 1993 NPAP.
In summary, the NPAP has expanded to include all criteria pollutants, VOCs and carbonyls, improved the
associated equipment and instruments, and changed its site selection process.
-------
96-FA 150.01
T?ble 1. National ambient air quality standards.
Pollutant
CO
Pb
N02
03
PM-10
Primary standard
(health related)
Type of average
8-hi"
l-hrb
Maximum quarterly
average
Annual arithmetic
mean
Maximum daily 1-hr
averagec
Annual arithmetic
Std. level
conc.a
9 ppm
(10 mg/m3)
35 ppm
(40 mg/m3)
1.5«ta>
0.053 ppm
(100^g/m3>
0.1 2 ppm
(225 ^g/m3>
50 ug/m3)
Secondary standard
(welfare related)
Std. level
Type of average cone.
No secondary
standard
No secondary
standard
Same as primary
standard
Same as primary
standard
Same as primary
standard
Same as primary
SO,
mean0
24-hrd
Annual arithmetic
mean
80
(0.03 ppm)
standard
Same as primary
standard
3-hrb
1300,ug/m3
(0.50 ppm)
24-hrb 365
"Parenthetical value is an approximately equivalent concentration.
bNot to be exceeded more than once per year.
The standard is attained when the expected number of days per calendar year with maximum hourly
average concentrations above 0.12 ppm is equal to or less than 1, as determined according to Appendix H
of the Ozone NAAQS.
dParticulate standards use PM-10 (particles less than 10 micrograms in diameter) as the indicator
pollutant. The annual standard is attained when the expected annual arithmetic mean concentration is less
than or equal to 50 jug/m3; the 24-hour standard is attained when the expected number of days per
calendar year above 150 ^ig/m3 is equal to or less than 1, as determined according to Appendix K of the
PM NAAQS.
-------
96-FA 150.01
Table 2. NPAP audit material/gas cylinder concentration ranges.
Audit
level
S025 O3, and NO2 1
2
3
CO 1
2
3
Audit
level
Pb 1
2
Concentration
range, ppm'
0.03-0.08
0.15-0.20
0.35-0.45
3-8
15-20
35-45
Concentration
range, ^g/strip1
100-300
600-1000
'Federal Register, 40 CFR Part 58, Appendix A, revised July 1,1987.
-------
96-FA150.01
Table 3. NPAP audits - 1991 to 1995,
Pollutant
CO
S02
NO part of NO2
NO2
03
SSI/hi-vol
Dichot(PM-10)
Pb
VOCs
Carbonyls
Number of NPAP
1991 1992 1993
Labs
Samplers
Labs
Samplers
Labs
Samplers
Labs
Samplers
Labs
Samplers
Labs
Samplers
Labs
Samplers
Audits
Audits
Audits
114
183
123
204
83
119
14
19
43
77
315
612
9
17
322
-
_
134
261
142
279
101
170
14
27
135
340
308
1087
12
24
335
-
.
142
300
154
321
105
182
55
122
157
426
322
1315
17
52
352
-
.
Audits
1994
146
303
167
364
104
183
3
3
163
480
299
1551
11
29
336
-
_
1995
135
311
150
309
98
170
62
185
182
519
292
1481
10
28
320
72
32
-------
96-FA150.01
Table 4. Site selection criteria for the NPAP.
Pollutant
Priority
Criteria
03
PM-10
CO
Pb
SO,
NO,
PAMS
1
2
3
3
4
1
2
3
Sites that had expected exceedances of the O3 NAAQS
i 1.1 days from 1991 through 1993.
Sites recording values > the 24-hr. NAAQS.
Sites recording values > the 80% but < 100% of the
24-hr NAAQS.
Sites recording values > 50% but s to 80% of the
24-hr NAAQS.
Sites recording exceedances of the CO NAAQS from
1992-1993.
Some selected sites within CO nonattainment areas.
Sites recording CO values between 7.5 ppm and 9.4 ppm.
Sites located near sources which are subject to potential
regulatory compliance, out of compliance, and/or subject
to a consent decree.
Sites located near sources that are either in compliance
with no violations, are closed for business, or are well
above the Pb NAAQS with no significantly questionable
data.
Sites recording values > the 24-hr SO2 NAAQS.
Sites recording values between 80% and 100% of the
24-hr SO2 NAAQS.
Sites recording values > 50% but < 80% of the 24-hr S02
NAAQS.
Sites recording values > 50% of the annual NO2 NAAQS.
Type 2 PAMS sites.
Remaining PAMS sites.
10
------- |