600/A-98-052
98-MP27.01
The Future of the National Performance Audit Program (NPAP)
Joseph B. Elkins, Jr.
U.S. Environmental Protection Agency
Office of Air Quality Planning and Standards (MD-14)
Research Triangle Park, North Carolina 27711
Elizabeth T. Hunike
U.S. Environmental Protection Agency
National Exposure Research Laboratory (MD-46)
Research Triangle Park, North Carolina 27711

-------
TECHNICAL REPORT DATA
1. REPORT NO.
EPA 600/A-98-052
2.

4. TITLE AND SUBTITLE


5.REPORT DATE

The Future of the National



Performance Audit Program (NPAP)

6.PERFORMING ORGANIZATION CODE
7. AUTHOR)S)


8.PERFORMING ORGANIZATION REPORT NO.
Joseph B. Elkins & Elizabeth T. Hunike



9. PERFORMING ORGANIZATION NAME AND ADDRESS

10.PROGRAM ELEMENT NO.
Same as 12.

11. CONTRACT/GRANT NO.
12. SPONSORING AGENCY NAME AND
ADDRESS

13.TYPE OF REPORT
AND PERIOD COVERED
NATIONAL EXPOSURE RESEARCH LABORATORY

Symposium Proceedings
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711

14. SPONSORING AGENCY CODE
15. SUPPLEMENTARY NOTES



r
16. ABSTRACT




The National Performance Audit Program (NPAP) is one of the major components in the
quality assurance of the nation's air monitoring program. The nation's ambient air
monitoring program contains approximately 5000 monitors collecting data for the six
pollutants for which national ambient air quality standards (NAAQS) have been
designated. The quality assurance/quality control program has several major
components including the EPA Regional Systems Audits, the Precision and Accuracy
Program, and the NPAP. The NPAP has historically been operated by EPA's Office of
Research and Development (ORD>. On January 1, 1998 EPA's ORD transferred the NPAP
to the Office of Air Quality Planning and Standards (OAQPS). The future of this
important performance evaluation program will be discussed.
17.
KEY WORDS AND DOCUMENT ANALYSIS

a. DESCRIPTORS
b.IDENTIFIERS/ OPEN ENDED
TERMS
c.COSATI
National Performance Audit Program;
Quality Assurance; Audits; Criteria Air


Pollutants
18. DISTRIBUTION STATEMENT

19. SECURITY
Report)
CLASS (This
21.NO. OF PAGES
RELEASE TO PUBLIC

UNCLASSIFIED
11


20. SECURITY
Page)
CLASS (This
22. PRICE


UNCLASSIFIED

-------
98-MP27.01
INTRODUCTION
The Nation's ambient air monitoring program contains monitors for the six criteria pollutants for which
national ambient air quality standards (NAAQS) have been designated. These pollutants are carbon
monoxide (CO), lead (Pb), nitrogen dioxide (N02), ozone (03), particulate matter smaller than 10 microns
(PM-10) and particulate matter smaller than 2.5 microns (PM-2.5), and sulfur dioxide (S02). The standards
for the criteria pollutants are shown in Table 1.
There are approximately 4,98lair pollution monitors in the ambient air network. These monitors comprise
the State and Local Air Monitoring Stations (SLAMS), the National Air Monitoring Stations (NAMS),
and the Prevention of-Significant Deterioration (PSD) sites. The distribution of monitors by pollutant is:
S02, 645; CO, 540; N02, 373; 03 943; Pb, 418; and PM-10, 1584. The data from these monitors are
reported to EPA's Aerometric Information Retrieval System (AIRS). Although the data from these
monitors are used primarily for determining compliance with-the NAAQS, they also support numerous
other regulatory and research activities.
The federal regulations for the SLAMS/NAMS/PSD sites are codified in 40 CFR 58 including information
pertaining to the quality assurance/quality control (QA/QC) program for these monitors. The QA/QC
program has three major indexes of quality: the EPA Regional Office Systems Audits; the Precision and
Accuracy System (formerly PARS); and the National Performance Audit Program (NPAP).
The NPAP was managed for the first 17 years of its existence by EPA's Office of Research and
Development. On January 1, 1998, the management of the NPAP was transferred to EPA's Office of Air
Quality Planning and Standards (OAQPS). The audit devices and materials associated with the NPAP
have been supplied to the participating agencies and the resulting data managed by a contractor. The
program has encompassed the 6 pollutants for which there are National Ambient Air Quality Standards
(NAAQS) and those ozone precursors identified in 40 CFR 58. OAQPS will continue to manage the
program through a contractor. The NPAP will continue to include the criteria pollutants and the ozone
precursors identified in 40 CFR 58. The NPAP audit procedures for the volatile organic carbon (VOC)
compounds are being reexamined and may change. EPA has recently promulgated a NAAQS for PM 2.5
and the regulations (40 CFR 58) now require a PM 2.5 Federal Reference Method (FRM) audit. This PM
2.5 FRM audit may be incorporated into the next NPAP contract. Internally the quality control on the EPA
NPAP contractor was provided by the EPA ORD's "Standards Laboratory." This important function is
now being carried out by EPA Region 7.
The NPAP is a cooperative effort among EPA's Office of Air Quality Planning and Standards (OAQPS),
the 10 EPA Regional Offices, and the 170 state and local agencies that operate the SLAMS/NAMS air
pollution monitors. Also included in the NPAP are approximately 135 organizations (governmental and
private) that operate air monitors at PSD sites. Participation in the NPAP is required for agencies
operating SLAMS/NAMS and PSD monitors as per Section 2.4 of 40 CFR Part 58, Appendix A and
Section 2.4 of 40 CFR Part 58, Appendix B. The NPAP is operated by the Monitoring and Quality
Assurance Group of OAQPS. Participation in the NPAP program is also mandatory for the 22 agencies
which monitor for photochemical oxidants under EPA's Photochemical Assessment Monitoring (PAMS)
program. These agencies monitor for carbonyl compounds, volatile organic compounds, Nox and ozone.
The NPAP's goal is to provide audit materials and devices that will enable EPA to assess the proficiency of
2

-------
98-MP27.01
agencies that are operating monitors in the SLAMS/NAMS and PSD networks. To accomplish this, the
NPAP has established acceptable limits or performance criteria, based on the data quality needs of the
SLAMS/NAMS and PSD requirements, for each of the audit materials and devices used in the NPAP. Any
device or material not meeting these criteria is not used in the program.
All audit devices and materials used in the NPAP are certified as to their true value, and that certification is
traceable to a National Institute of Standards and Technology (NIST) standard material or device wherever
possible. The audit materials used in the NPAP are as representative and comparable as possible to the
calibration materials and actual air samples used and/or collected in the SLAMS/NAMS and PSD
networks. The audit material/gas cylinder ranges used in the NPAP are specified in the Federal Register
(Table 2).
The objectives for the NPAP audits are two-fold: (1) to complete at least 95% of the scheduled audits by
the end of the year, and (2) to determine if the participants' performance exceeds the limits shown below.
The NPAP audits are accomplished using a variety of mailable audit systems. The participants use these
audit systems to generate pollutant concentrations and flowing air streams which are introduced into their
sampling system. The pollutant concentrations and air stream flow rate are unknown to the audit
participants. The outputs from the sampler that result from the use of the audit system are recorded on a
data form, returned to EPA, and compared to the concentration or flow rate that should have been
generated by the audit system under the environmental conditions at the site. The differences between the
EPA expected (certified) values and the NPAP participants' reported values are calculated and returned to
the participant.
DESCRIPTION OF NPAP AUDIT MATERIALS/DEVICES
High-VoIume/PM-10 (SSI) Flow Audits
The reference flow (ReF) device used for the high volume flow audit consists of a modified orifice, a wind
deflector, a manometer, and five resistance plates. The ReF for the PM-10 (SSI) flow audit is similar
except a filter is used as the only resistance.
Sulfur Dioxide/Carbon Monoxide (GDS) Audits
The GDS consists of a dilution device, a zero air generator and a cylinder of gas containing approximately
30 ppm sulfur dioxide and 3000 ppm carbon monoxide.
Ozone (TECO 165) Audit
Audit
EPA determined limits
High volume/PM-10 (SSI)
Dichot (PM-10)
Pb (analytical)
S02, N02, 03 and CO
% difference > ± 15% for 1 or more flows
% difference > ± 15% for 1 or more flows
% difference > ± 15% for 1 or more levels
Mean absolute % difference > 15%
PAMS
The EPA determined limits were still being reviewed
and developed at the time of this paper's publication
3

-------
98-MP27.01
The audit device is self-contained with its own zero air and ozone generation system.
Lead Audit
The samples are 1.9 cm wide and 20 cm long glass fiber filter strips that have been spiked with an aqueous
solution of lead nitrate and oven-dried. Two filter strips comprise a sample.
Dichotomous (PM-10) Flow Audit
The audit device consists of a laminar flow element (LFE), an incliried manometer, an altimeter, and a small
dial thermometer! It measures fine flow (15.001pm) and total flow (16.7 1pm).
Ozone/Nitrogen Dioxide/Sulfur Dioxide/Carbon Monoxide (TECO175) Audit
The audit device is a combination of the TECO 165 and the GDS audit systems. It uses the same zero air
generation system as the GDS, the ozone generation system of the TECO 165, and a gas cylinder
containing approximately 3000 ppm carbon monoxide, 30 ppm sulfur dioxide and 30 ppm nitric oxide.
The ozone generation system is used with the pollutant gas to convert nitric oxide to nitrogen dioxide via a
gas phase titration. The TECO 175 s were introduced into the NPAP in the fall of 1994; however, flow
stability problems delayed full utilization of the equipment. The problems were traced to the pre-set
regulator and replacement was completed in time for the 1995 audits.
PAMS Volatile Organic Compound (VOC) Audit
This audit uses a gas transfer system (GTS), stock (concentrated) compressed gas mixtures containing
PAMS compounds and 1,5L compressed gas (audit) cylinders. The stock mixtures are mixed and diluted
using the GTS and the resulting mixture is placed in the 1.5L audit cylinders. These audit cylinders are
pressurized to 500 psi to yield recoverable gas volumes of 40 to 60 L. Three audits are scheduled for each
year. Each of the 22 PAMS agencies receives one cylinder for each audit. The cylinders contain between
15 and 35 PAMS analytes at concentrations from 10 to 60 ppbv as carbon. The PAMS VOC audit was
added to the NPAP in 1995.
PAMS Carbonyl Compound Audit
This audit uses three glass tubes containing DNPH which have been spiked with solutions containing
acetone, formaldehyde and acetaldehyde. Each tube contains from 0.2 to 10 micrograms of each carbonyl
compound. The audit is conducted on the same schedule as for the PAMS VOC audit. Each PAMS
agency recovers the carbonyl compounds from the three DNPH tubes and reports the results to EPA. The
PAMS carbonyl audit was added to the NPAP in 1995.
SITE SELECTION
Historically, the State and local agencies have been allowed to select the NPAP sites to be audited. The
1989 General Accounting Office (GAO) audit raised concerns about the NPAP site selection process. The
GAO believed there could be a possible bias in the NPAP data base because (1) NPAP participants
selected the samplers to be audited, and (2) NPAP participants may have performed unscheduled
calibrations on samplers prior to the audits. In 1991 EPA sent an audit team to 79 SLAMS monitoring
stations located in all ten EPA regions. The audit team used the equipment and procedures of the NPAP to
assure that their audits simulated as closely as possible a normal NPAP audit. To ensure that the SLAMS
agencies did not take any special precautions, the audit team notified the agencies only 1 to 2 days prior to
arrival. Also, the site log books were checked by the audit team to verify that the samplers had not been
calibrated prior to the audit. In all 172 samplers were audited including samplers for CO (53), NO-N02
4

-------
98-MP27.01
(25), S02 (38) and 03 (34). Ninety- seven of the samplers had been audited in the NPAP during 1989 -
1990, but the rest had not. Based on a statistical analysis of the data from the 1991 study and the 1989 and
1990 NPAP, EPA concluded that (1) overall the NPAP participants were auditing their samplers properly
and not conducting special unscheduled calibration tests, and (2) the assumption that the NPAP data base
was biased because the participants selected the sampler to be audited was not supported. EPA
additionally responded to the GAO comments by developing site selection criteria (Table 3) that were
incorporated into the 1993 NPAP. Priority 1 site selection criteria should be audited annually; priority 2 at
least once every 2 years; priority 3 at least once every 3 years; priority 4 at least once every 4 years. All
other sites should1 be audited at least once every 5 years. Based on this criteria, EPA now selects specific
sites that are to be audited in the NPAP. The criteria are reviewed annually, and site selection is updated
accordingly.
RESULTS
The number of NPAP audits performed from 1993 through 1997 is shown in Table 4.
In 1997, 100% of the scheduled audits were completed with the exception of N02 (84%). At the present
time, EPA does not have enough equipment to meet the demand for N02 audits. Due to the current
budgetary constraints, this may be a long term problem.
The percentage of 1997 NPAP participants whose performance fell within the EPA guidelines of 15% of
the certified values was: CO, 99%; S02, 98%; NO, 98%; N02, 84%; 03, 96%; hi-vol/PM-10 (SSI), 96%;
dichot (PM-10), 80%; and Pb (analytical), 94%. These percentages have remained similar since 1993 for
03> S02> CO, hi-vol/PM-10 (SSI), Pb, NO, and N02. PM-10 (dichotomous) (70% to 80%) has increased
substantially which may be attributable to increased familiarity with the audit equipment.
CONCLUSIONS
EPA's Quality Assurance Guidance mandates that all data collected for regulatory or research purposes be
of known and documented quality. The NPAP is a critical component in the SLAMS/NAMS/PSD
programs for accomplishing this goal. The defensibility of the decisions resulting from the
SLAMS/NAMS/PSD ambient air data is enormously important. The data from these networks are used
for determining compliance with the NAAQS and result in input to the State Implementation Plans (SIPS)
These SIPs can have very large economic impacts.
EPA uses the NPAP to independently quality assure the SLAMS, NAMS, PSD, and PAMS monitoring
data it is receiving and permanently storing on its Aerometric Information Retrieval System (AIRS). The
AIRS is the conduit for which the research community, commercial and industrial communities, and the
public at large, gain access to the EPA's monitoring data. EPA also uses the NPAP as its only available
national, independent mechanism for continually assessing the quality of the ambient air quality data
necessary to develop and defend its research initiatives.
The cornerstone of any data collection system is the quality assurance component. The data utilization
resulting from the Nation's air monitoring network continues to increase in importance. The strategies
developed from the information can cost millions of dollars. Henceforth, we must remain vigilant in our
efforts to maintain the integrity of this important data set. In these efforts, the NPAP continues to be
refined. The NPAP has expanded to include all the criteria pollutants, volatile organic compounds and
carbonyl compounds. The associated instruments and equipment have been improved to incorporate the
5

-------
98-MP27.01
latest technologies. The NPAP continues to respond to comments from the GAO audits as well as state
and local agency contacts. With the increaseu interest in the data from the nation's air monitoring
community, it was inevitable that the site selection process would also be modified. These modifications
were incorporated into the 1993 NPAP. The most important sites are now audited each year through the
NPAP and every site is expected to be audited minamally once in a five year period.
In summary, the importance of the NPAP can not be understated. It has proven to be an important safe-
guard on one of the most important environmental data bases in the country. The NPAP is now being
managed by a different organizational unit of the EPA. This organizational unit, OAQPS, recognizes the
importance of this program and unequivocally states that this program will continue in the future. It will
continue to improve as it has under the management of EPA's ORD.
6

-------
98-MP27.01
Table I. National ambient air quality standards.
Pollutant
Primary standard
(health related)
Secondary standard
(welfare related)
Type of average
Std. level
cone.*
Type of average
Std. level
conc.
CO
£-hrb
9 ppm
(10 mg/m3)
No secondary
standard

l-hr*
35 ppm
(40 mg/m3)
No secondary
standard
Pb
Maximum quarterly
average
1.5 //g/m3
Same as primary
standard
no2
Annual arithmetic
mean
0.053 ppm
(100 ii g/m3)
Same as primary
standard
o3
Maximum daily 1-hr
average0
0.12 ppm
(225 //g/m3)
Same as primary
standard
PM-10
Annual arithmetic
meand
50 /ig/m3)
Same as primary
standard

24-hrd
150 /zg/m3
Same as primary
standard
SO,
Annual arithmetic
mean
24-hrb
80 Mg/m3
(0.03 ppm)
365 fig/m3
3-hr*5
1300/zg/m3
(0,50 ppm)
"Parenthetical value is an approximately equivalent concentration.
"Not to be exceeded more than once per year.
cThe standard is attained when the expected number of days per calendar year with maximum hourly
average concentrations above 0.12 ppm is equal to or less than 1, as determined according to Appendix H
of the Ozone NAAQS.
Particulate standards use PM-10 (particles less than 10 micrograms in diameter) as the indicator pollutant.
The annual standard is attained when the expected annual arithmetic mean concentration is less than or
equal to 50 //g/m3; the 24-hour standard is attained when the expected number of days per calendar year
above 150 //g/m3 is equal to or less than 1, as determined according to Appendix K of the PM NAAQS.
7

-------
98-MP27.01
Table 2. NTAP audit material/gas cylinder concentration ranges.

Audit
level
Concentration
range, ppm1
S02, 03) and N02
l
0.03-0.08

2
0.15-0.20
r
3
0,35-0.45
CO
1
3-8

2
15-20

3
35-45

Audit
l^vel
Concentration
range, tig/strip1
Pb
1
100-300

2
600-1000
federal Register, 40 CFR Part 58, Appendix A, revised July 1, 1987.
8

-------
98-MP27.01
Table 3. Site selection criteria for the 1997 NPAP audits.
Pollutant	Priority	Criteria
03	1	Sites with expected average exceedances of the 03
NAAQS> 1.1 days from 1992 through 1994.
PM-10	2	Sites recording values > the 24-hr. NAAQS in 1994. '
3	Sites recording values > the 80% but < 100% of the
24-hr NAAQS in 1994.
4	Sites recording values > 50% but s to 80% of the
24-hr NAAQS in 1994.
CO	3	Sites recording exceedances of the CO NAAQS from
1993 through 1994 plus selected CO sites within CO
nonattainment areas.
4	Sites recording CO values between 7.5 ppm and 9.4 ppm
from 1993 through 1994.
Pb	1	Sites located near sources which are subject to potential
regulatory compliance, out of compliance, and/or subject
to a consent decree or above the NAAQS for 1994.
2	Sites located near sources that are either in compliance
with no violations, are closed for business, or are well
below the Pb NAAQS with no significantly questionable
data.
S02	2	Sites recording values > the 24-hr S02 NAAQS in 1994.
3	Sites recording values between 80% and 100% of the
24-hr S02 NAAQS in 1994.
4	Sites recording values > 50% but < 80% of the 24-hr S02
NAAQS in 1994.
N02	3	Sites recording values > 50% of the annual N02 NAAQS
9

-------
Table 4. NPAP audits - 1993 to 1997,
98-MP27.01
Pollutant
1993 1994 1995 1996 1997
CO
SO,
NO part of N02
NO,
O,
SSI/hi-vol
Dichot (PM-10)
Pb
Labs
Samplers
Labs
Samplers
Labs
Samplers
142
300
154
321
55
122
Labs	157
Samplers 426
146
303
167
364
Labs	105 104
Samplers 182 183
163
480
135
311
150
309
98
170
62
185
182
519
135
310
162
357
107
203
69
146
176
562
Labs	17 11 10 18
Samplers 52 29 28 41
144
331
153
350
112
227
97
201
180
Labs	322 299 292 301 314
Samplers 1315 1551 1481 1489 1743
Audits
352 336 320 291
20
43
287
VOCs
Audits
72 36 101
Carbonyls
Audits
32 24
48
10

-------