\ V
United States
Environmental Protection
Agency
Atmospheric Research and Exposure•>",.
Assessment Laboratory ^-'
Research Triangle Park NC 27711 ',
Research and Development
EPA/600/S3-89/012 June 1989
&EPA Project Summary
Interim Report: Quality
Assurance Support for the
National Atmospheric Deposition
Program and National Trends
Network Monitoring Activities:
1987-1990
David S. Bigelow
The full report summarizes the
quality assurance activities of the
NADP/NTN Quality Assurance
Manager from mid-1987 through mid-
1988. The report presents some
accomplishments and makes recom-
mendations for the network.
The report discusses data quality
issues relating to site representa-
tiveness, field versus laboratory
measurements of pH, site operator
turnover rates and various monitoring
network reviews. The report also
contains the previously released
"NADP/NTN Instruction Manual for
Site Operation" and the "Quality
Assurance Report of NADP/NTN
Deposition Monitoring for Laboratory
Operations" as well as an NADP/NTN
quality assurance bibliography.
Recommendations emphasize the
quality assurance manager's role in
coordinating quality assurance
activities, the need for more inte-
grated quality assurance evaluations,
the need to standardize quality
assurance data reporting formats and
data transfer protocols in the network
and the need for establishing a
method for estimating overall
precision and accuracy in spatial and
temporal data summary statistics.
This Project Summary was devel-
oped by ERA'S Atmospheric Research
and Exposure Assessment Laboratory,
Research Triangle Park, NC, to
announce key findings of the research
protect that Is fully documented In a
separate report of the same title (see
Project Report ordering Information at
back).
Introduction
The National Atmospheric Deposition
Program (NADP) monitoring network
began operations in July of 1978. The
Network uses cooperating local, state,
and federal agencies as well as private
industry to operate a network of sites
according to a single set of protocols. In
1983, NADP assumed responsibility for
managing the day-to-day operations of
the National Acid Precipitation Assess-
ment Program's National Trends Network
(NTN). This action resulted in the
combined, cooperative monitoring pro-
gram now known as the NADP/NTN
monitoring network. The NADP/NTN
monitoring network serves both the
National Atmospheric Deposition Pro-
gram and the National Acid Precipitation
Assessment Program.
Because the NADP/NTN monitoring
program both represents and relies on
-------
many different public and private
agencies to accomplish its goals, it is
unique in its structure and mode of
operation. For quality assurance, this has
meant that responsibility for the
development of quality assurance
programs has been and continues to be a
voluntary effort assumed by each of the
agencies who have responsibility for
managing specific network operations
(i.e., Field Sites, Central Analytical
Laboratory, Coordination Office, U.S.
Geological Survey, U.S. Environmental
Protection Agency). These quality
assurance procedures are then reviewed
and accepted by the various NADP
committees who have responsibility for
overseeing each network function. This
practice has been strongly encouraged
since it would require considerable effort
and expertise to centralize this function
within a single organization.
A decentralized approach to quality
assurance, however, creates its own
problems. Many of the network quality
assurance procedures are innovative
adaptations of recognized quality
assurance practices, and as such are
reported in the scientific literature, at
professional and technical meetings or in
various agency reports rather than in a
standard network document or operating
procedure publication. This lack of
centralized quality assurance reporting
makes documenting the achievement of
network data quality goals and objectives
difficult and also makes it difficult to
continue some key quality assurance
programs once scientific recognition for
the techniques has been achieved. The
likelihood that those quality assurance
activities that cannot be maintained by a
single agency, either because of financial
or cooperative restraints, will be
overlooked is also increased. Finally,
decentralized reporting can lead to
situations where results from one effort
may conflict with another such that no
clear picture of 'true' data quality can
emerge.
Recognizing the need for a more
coordinated and thorough quality
assurance program, NADP formed a
study committee in 1982 to address this
issue. The work of this committee
resulted in the development of a Quality
Assurance Plan to cover all aspects of
network operation and the recommenda-
tion that a full time Quality Assurance
Manager be appointed to administer this
quality assurance program. The
desirability of a coordinated quality
assurance program was again recognized
in the NAPAP Peer Review held in
Boulder, Colorado in 1985.
Accomplishments
Coordination of NADP/NTN
Quality Assurance Plan
Implementation
The primary coordination efforts have
involved attending the frequent NADP
Technical and Subcommittee meetings;
writing, revising and reviewing NADP/
NTN documentation; reviewing agency
reports and proposals that utilize
NADP/NTN data or documentation; and
disseminating quality assurance informa-
tion to the various NADP committees and
scientists. Each of these activities serves
as a vehicle for obtaining and document-
ing the quality assurance activities of the
network and for promoting the evaluation
and reporting of quality assurance data.
An additional part of the coordination
effort has involved overseeing the Site
Visitation program and providing admin-
istrative assistance to the program's
auditors. The Quality Assurance Manager
is responsible for distributing the results
of the site audits and for initiating
corrective action to correct any dis-
covered deficiencies. As with all of the
external quality assurance programs,
results are reported separately by each
sponsoring agency.
Review and Revision of the
NADP/NTN Quality Assurance
Plan
During October of 1986, the NADP
Quality Assurance Steering Committee
began the task of reviewing and revising
the NADP Quality Assurance Plan. As
part of this process, external reviewers
were asked to make suggestions to
improve the plan. By 1987 reviewer
comments had been received and
assignments made for revising all
portions of the Plan. This process
continues with the Quality Assurance
Manager taking a lead role in the
development of the Data Management
portion of the Plan. A first revised working
draft was completed in August of 1988.
As part of this revision and in response
to reviewer comments, a Remedial Action
Plan (RAP) was developed for correcting
instances of non-compliance detected
during network operations. The RAP was
completed and approved in November
1987. It outlines for the first time the
responsibilities of the subcommittees and
program managers for resolving netwoi
deficiencies.
Review and Evaluation of
NADP/NTN Network Operations
Four areas of Network operations wer
targeted for review during the past ye<
by the Quality Assurance Manager. Th
reviews covered regional represents
tiveness of NADP/NTN monitoring dab
differences between NADP/NTN labors
tory and field data, turnover rates c
NADP/NTN site operators and the revie'
and revision of the site operator'
instruction manual.
Future Needs of the NADP/NTN
Monitoring Network
Precision and Bias Estimates
A proposal for establishing NADP/NTf
network precision and accuracy wa
developed and submitted for funding it
September 1987. The proposal outlined .
duplicate sampling program to determim
the precision and accuracy of NADP
NTN weekly data, seasonal summarie
and regional estimates of depositioi
based upon monitoring network data
Estimates of precisions were to com*
from a proposed within network colocatet
sampling program. Bias and accurac;
were to be estimated using data fron
existing between network colocate<
sampling programs and the propose<
colocated sampling program. Th<
proposal was accepted but not funded
Unfortunately this leaves the networl
without the means to establish the
precision and accuracy estimates callec
for in the NADP Quality Assurance Plan
Data from which estimates of precisior
can be made are available at only twc
locations and for limited time periods
One location in North Carolina producec
data only for one year. The seconc
location in Wisconsin has only 18 month;
of data. Neither location can be
considered as representative of the entire
network and neither can be used foi
estimates of bias.
In addition to duplicate sampling, the
proposal described a second program fpi
determining precision and bias of rair
gauge measurements. In this seconc
program, precision would have beer
determined by the rain gauge's ability tc
weigh a known amount of sand at twc
reference points within the instruments
working range. The reproducibility of the
difference between the known sand bottle
-------
weight and that measured by the
instrument at the reference points would
have been a measure of the precision or
random error. The magnitude of the
difference would have been a measure of
bias or systematic error. The lack of
funding of this portion of the proposal
leaves the network with no means to
establish rain gauge performance in the
network. Similar rain gauge performance
tests were discontinued in 1983 due to
lack of funding.
Overall seasonal and regional esti-
mates of precision and accuracy of
NAOP/NTN rain and chemistry data will
have to be inferred from other monitoring
network data (e.g., Electric Power
Research Institute's daily Operational
Evaluation Network, National Weather
Service's climate center, etc.) or be
computed piece-meal from the volume of
agency information specific to each
network related study. The quality and
completeness of this latter approach has
never been assessed. Even if this latter
approach were successful, it is antici-
pated that much of the error estimates
used in computing overall error estimates
will have to come from 'related studies.'
Addition of a Second Laboratory
to the NADP/NTN Monitoring
Program
One of the more time consuming tasks
of the past contract period has been the
addition of a second laboratory to the
NADP/NTN monitoring program. Because
of contracting requirements, site liaison
and the chemical analysis of samples
from 18 NADP/NTN monitoring sites
administered by the National Park
Service were performed by a second
laboratory from March 24 through
September 29. 1987. During this time
period, the Quality Assurance Manager
was required to provide additional SOP
and QA documentation to the second
laboratory as well as coordinate the start
up of internal and external quality assur-
ance programs that would eventually
allow the evaluation of the second
laboratory's data in relation to the data
from the original laboratory. At the
request of the National Park Service, two
audits of the new laboratory were
conducted, one prior to the start of
operations and one approximately four
months into the contract. Completed data
and some documentation became
available in the early summer of 1988.
A number of inconsistencies with
previous NADP practices have prevented
a full evaluation of the data. Work is
currently scheduled, subject to funding,
for the fall of 1988 to reevaluate field data
in a manner consistent with prior
NADP/NTN practice. Chemistry data will
require more documentation before an
evaluation of comparability can be
initiated. Audit reports are available
through the National Park Service's Acid
Rain Coordinator in Washington, DC.
Briefly, no serious deficiencies were
found in the laboratory's operations.
Many of the 'problems' noted in the audit
were associated with data management
and site liaison activities. Most were
correctable. Some of the noted defi-
ciencies might have been more easily
correctable if better documentation of
NADP/NTN standard operating proce-
dures in these areas was available. The
quality of chemistry results simply lacked
documentation. This does not necessarily
indicate a lack of quality, only that quality
is undefined.
The NADP/NTN quality assurance
activities and their sponsoring agencies
are listed in Table 1.
Recommendations
• The Quality Assurance Manager's
participation in auditing, document
reviews and scientific meetings should
continue to be supported. These are
the windows through which the QA
Manager can view the network opera-
tions.
• Standard formats and transfer protocols
for external quality assurance data
needs to be established to eliminate
redundant data entry and to enable the
timely transfer of quality assurance
data from the agencies to the
NADP/NTN Coordination Office.
• Continue to evaluate and document
network data quality and network
procedures through the publication of
integrated quality assurance reports.
These reports should be inclusive of all
years of operation through current time.
• Studies like the field versus laboratory
pH evaluation should be increased to
further document the limitations and
strengths of the NADP/NTN data set.
• Establish the Quality Assurance Man-
ager and quality assurance technician
roles as full time positions. This should
help increase the frequency of inte-
grated quality assurance reports and
allow for continuity in the tracking of
network quality assurance activities.
The current three-quarter time staffing
level provides only enough resources
to manage the NADP/NTN quality
assurance program on a topical basis.
• The network should continue to
organize and gather information which
profiles each of its monitoring stations.
This information should be maintained
and organized in such a way as to
facilitate the selection of sites for
regional and topical data analysis.
• A quality assurance program specif-
ically designed to estimate network
precision and bias should be
developed and implemented as soon
as possible. Emphasis should be
placed on comparisons within NADP/
NTN and between peer network data.
• The comparability of NADP/NTN data
to that of other major networks
operating in North America should be
established.
-------
Table 1. NADP/NTN Monitoring Network Quality Assurance Activities and Their Sponsoring Agencies
Task No. Task Name Operating Agency"
Program Name
Field Site Operations:
1.
2.
3.
4.
Quality Control of Field Operations CAL
Quality Assurance of Field Operations CSU
Systems and Performance Audit of Field EPA (RTI)
Operations
Quality Assurance of Site Chemical USGS
Analysis
CAL Site Interactions
Across Site Network Analysis
NADPINTN Site Visitation Program
Intersite Comparisons
Laboratory Operations:
5.
6.
7.
Laboratory Quality Control
Laboratory Quality Assurance
Laboratory Systems Audit
CAL
CAL
NADP Subcommittee 2
CAL QA Program
Interlaboratory Comparisons
Data Management:
8.
8.
9.
10.
Quality Control of Site/Chemical Analysis
Data through Data Management
Quality Control of Site/Chemical Analysis
Data through Data Management
Quality Control of Data Management
Quality Assurance of Network Data
Management
CAL
CSU
PNL
QA Steering Committee
Data Screening and Coding
Data Screening, Coding and
Summary Reporting
Acid Deposition System (ADS)
for Statistical Reporting
Systems and Performance Audit
of (he Data Management Program
Network Operations:
11.
12.
13.
14.
15.
15.
Performance Audit of Field and
Laboratory Chemical Analysis
Laboratory Quality Assurance
Performance Audit of the Network
Special Studies which Assess Various
Components of Network Performance
Systems Review of the Deposition
Monitoring Network
Systems Review of the Deposition
Monitoring Network
USGS
USGS
CSU
ALL
TASK GROUP 4
QA Steering Committee
Blind Audit Program
Interlaboratory Comparisons
Network Intercomparison's
with CANSAP/CAPMoN
Published Scientific Research Results
NAPAP Annual Review
IR-7 Annual Review
•CAL-Central Analytical Laboratory, Illinois State Water Survey
CSU-Natural Resource Ecology Laboratory, Colorado State University
EPA-Environmental Monitoring Systems Laboratory, Environmental Protection Agency
PNL-Battelle's Pacific Northwest Laboratory
RTI—Research Triangle Institute, Research Triangle Park, NC
USGS-National Water Quality Laboratory, U.S. Geological Survey
Task Group 4—National Acid Precipitation Assessment Program (NAPAP)
QA Steering Committee—National Atmospheric Deposition Program
-------
David S. Bigelow is with the Colorado State University, Fort Collins, CO.
Berne I. Bennett is the EPA Project Officer (see below).
The complete report, entitled "Interim Report: Quality Assurance Support for the
National Atmospheric Deposition Program and National Trends Network
Monitoring Activities: 1987-1990" (Order No. PB 89-155 220/AS; Cost: $36.95,
subject to change) will be available only from:
National Technical Information Service
5285 Port Royal Road
Springfield, VA 22161
Telephone: 703-487-4650
The EPA Project Officer can be contacted at:
Atmospheric Research and Exposure Assessment Laboratory
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711
United States
Environmental Protection
Agency
Center for Environmental Research
Information
Cincinnati OH 45268
Official Business
Penalty for Private Use $300
EPA/600/S3-89/012
0000329 PS
U S *ȴIR PROTECTION AGENCY
CHlCftSO
*»- 60604
------- |