OFFICE OF WATER
PERFORMANCE EVALUATION STUDY PROJECT
Final Report
Prepared for:
The Acting Assistant Administrator for Water
Prepared by:
Communications and Information Management Staff
Policy and Resource Management Office
Office of Water
May, 1993
-------
-------
Acknowledgements
This document was prepared for the U.S. Environmental Protection Agency
(EPA), Office of Water (OW), under contract number 68-W1-0016. Ms. Wendy Blake-
Coleman, of the Policy and Resources Management Office and Martin Brossman of the
Office of Wetlands, Oceans, and Watersheds served as co-chairs for the project.
I would like to thank the following for their assistance in conducting the study:
The Design Team
Wendy Blake-Coleman, Quality Assurance Manager for the Office of Water
Martin Brossman, Quality Assurance Officer for the Office of Wetlands, Oceans, and
Watersheds
Robert Runyon, Quality Assurance Officer for Region II
Charles Jones, Quality Assurance Officer for Region III
Wade Knight, Quality Assurance Officer for Region IV
The Advisory Panel
James M. Conlon, Director
Drinking Water Standards Division
Geoffrey Grubbs, Director
Assessment and Watershed Protection Division
Michelle Killer, Chief
Communication and Information Management Staff
Richard Kozlowski, Director
Enforcement Division
William Telliard, Chief
Analytical Methods Staff
Ramona E. Trovato, Director
Ground Water Protection Division
Nancy Wentworth, Director
Quality Assurance Management Staff
-------
The Regional Quality Assurance Officers and Staff of the Environmental
Services Divisions
This group was instrumental in collecting and compiling use and benefit data as well as
reviewing the final report.
Carol Wood, Region I
Robert Rtmyon, Region II
Charles Jones, Region III
Wade Knight, Region IV
Charles Ritchie, Region VI
Jeff Wandke, Region VII
Barbara Daboll, Region VHI
Raleigh Farlow, Region X
Art Clark, Region I
Mark Winter, Region II
Jeff Dodd, Region III
George Shupp, Region V
Al Smith, Region VI
Rick Edmonds, Region VIII
Kent Kitchingman, Region IX
Barry Towns, Region X
A special thanks is extended to the staff of the Quality Assurance Research
Division of the Environmental Monitoring Systems Laboratory in Cincinnati, Ohio and
the staff of the Radioanalysis Branch of the Environmental Monitoring Systems
Laboratory in Las Vegas, Nevada for the background material they provided on the
design, administration, and funding of Water Laboratory PE Studies. I would also like to
thank the 50 other Office of Water, Water Management Division, and other EPA
Program Office staff who provided advice on the study approach, supplied information
on PE studies, and reviewed the final report.
Martha G. Prothro
Deputy Assistant Administrator
Office of Water
-------
TABLE OF CONTENTS
SECTION
PAGE NUMBER
EXECUTIVE SUMMARY
BACKGROUND
PERFORMANCE EVALUATION (PE) STUDY USES AND NEEDS
SCOPE AND COST OF THE CURRENT PE STUDY PROGRAM
16
FUNDING ISSUES AND OPTIONS
24
APPENDIX A - REGIONAL PE USE SURVEY
A-l
APPENDIX B - PRELIMINARY RESULTS FROM THE
REGIONAL PE USE SURVEY
B-l
APPENDIX C - FY 1992 RESOURCE REQUIREMENTS FOR
WATER PROGRAM PE STUDIES BY STUDY PHASE
C-l
APPENDIX D - ISSUES IDENTIFIED DURING STAFF INTERVIEWS D-l
APPENDIX E - SUMMARY OF ANALYTES BY PE STUDY
E-l
-------
-------
EXECUTIVE SUMMARY
Introduction
The quality and reliability of data used to implement water programs at the Federal,
Regional, and State levels is critical to the success of EPA's water programs. Using data of
inadequate quality or reliability creates the possibility of decision-making errors that lead to
unnecessary compliance expenditures of many millions of dollars or hi regulations that do not
sufficiently protect public health or the environment. In recognition of the critical importance
of data quality, the Office of Water (OW) and the Office of Research and Development (ORD)
have cooperated hi conducting a comprehensive laboratory performance evaluation (PE) study
program since the 1970s.
Laboratory PE studies are one of a suite of tools used to ensure the quality of analytical
data. PE studies are a relatively efficient and low cost way to determine whether laboratories
have the technical capability to accurately analyze samples. PE studies involve laboratory
analysis of samples which contain known concentrations of chemicals. The test samples are sent
to laboratories hi ampules which must be diluted to the appropriate volume, or as prepared
samples for analysis using specified analytical methods. Laboratories report the qualitative
and/or quantitative results of their analyses within a specified time period.
This report presents detailed qualitative and quantitative information concerning the uses
of and needs for water laboratory PE studies at the Federal, Regional, and State levels. It
reflects contributions from more than 75 staff and managers hi Headquarters OW programs,
Regional water programs the Regional Environmental Services Divisions, and QA representatives
from state programs. The report is the first comprehensive compilation of descriptive
information concerning water laboratory PE studies. As such, it will serve as a resource
document for continuing efforts to identify and implement long and short term solutions to the
problem of adequate funding for the water laboratory PE studies, improved administrative
processes, and effective technical approaches.
-------
11
Background
The joint OW/ORD water performance evaluation study program is summarized in
Exhibit ES-1. The program consists of three principal studies:
1. The Water Supply (WS) study, involving 4000 to 5000 laboratories annually,
which has chemistry, microbiology, and radiochemistry components and supports
implementation of the Safe Drinking Water Act;
2. The Water Pollution (WP) study, involving approximately 5000 laboratories
annually, which includes inorganic and organic analytes and tests laboratories'
abilities to analyze for common surface water quality parameters and pollutants;
and
3. The Discharge Monitoring Report Quality Assurance (DMRQA) study, distributed
to more than 7500 permit holders annually, which has chemistry and whole
effluent toxicity components and is used as one tool for ensuring the quality of
monitoring data submitted by National Pollutant Discharge Elimination System
(NPDES).
OW's Office of Science and Technology also conducts a laboratory PE study program in support
of effluent guideline development. PE testing is conducted for a limited number of laboratories
(nine laboratories were tested hi FY 1992) that provide analytical services for special projects
supporting effluent guideline development, evaluation, and revision.
In April of 1992, OW undertook a review of the laboratory performance evaluation (PE)
studies conducted for water programs by the Environmental Monitoring Systems Laboratories
in Cincinnati (EMSL-Ci) and Las Vegas (EMSL-LV) and by the OW Office of Science and
Technology (Engineering and Analysis Division). This effort was designed to achieve five
principal goals:
-------
Ill
EXHIBIT ES-1
WATER PERFORMANCE EVALUATION STUDIES
STUDY
Water Supply (WS)
Chemistry
Microbiology
Radiochemistry
KEY PURPOSE
Drinking Water Certification
Drinking Water Certification
Drinking Water Certification
LEAD ORGANIZATION
EMSL-Cincinnati
EMSL-Cincinnati
EMSL-Las Vegas
Water Pollution (WP)
Monitor performance of
laboratories that generate
ambient water quality monitoring
data.
EMSL-Cincinnati
Discharge Monitoring Report
Quality Assurance (DMRQA)
Chemistry
Toxicity Testing
Monitor quality of chemistry data
submitted by NPDES permittees in
Discharge Monitoring Reports.
Monitor quality of toxicity data
submitted by NPDES permittees
in Discharge Monitoring Reports.
EMSL-Cincinnati
EMSL-Cincinnati*
Effluent Guidelines Quality
Assurance
Monitor performance of contract
laboratories that analyze wastewater
samples to support effluent guidelines
development.
Analytical Methods Staff
The Office of Wastewater Enforcement and Compliance collaborates with
EMSL-Cincinnati on study design and implementation.
-------
IV
1. Determine the importance of laboratory PE studies for water
programs;
2. Identify current and future water program needs for laboratory PE
studies;
3. Evaluate the administrative efficiency and technical adequacy of
current water laboratory PE studies;
4. Determine current and future costs for water program laboratory
PE studies; and
5. Identify options for funding water program laboratory PE studies
in the future.
The study was conducted in response to concerns raised by the Environmental Monitoring
Management Council (EMMC) as well as by OW and ORD. In 1991, EMMC conducted an
initial examination of funding issues pertaining to laboratory PE studies conducted Agency-wide.
EMMC and OW agreed that OW would serve as a pilot for a comprehensive Agency-wide
review of current and future needs for laboratory PE studies and an evaluation of funding
alternatives. As a pilot, the OW project examines only PE study needs for OW programs and
alternatives for funding OW laboratory PE study needs in the future.
The OW Project
i
Data and information were gathered from six principal staff sources:
• EPA Headquarters OW,
• EPA Regional Water Management Divisions,
-------
. V
• EPA Regional Environmental Services Divisions (monitoring and quality
assurance),
• Office of Research and Development (both at EPA Headquarters and the EMSL
laboratories),
• Other EPA Program Offices at Headquarters, and
• Other Federal agencies.
The information collected focuses on uses and needs for PE studies hi water programs, including
those implemented under the Safe Drinking Water Act, the Clean Water Act, the Marine
Protection, Research, and Sanctuaries Act, and other programs implemented by OW. Limited
information was also collected on the extent to which the water PE studies are used by other
environmental programs at the Federal, Regional, and State levels.
The results of this study reveal that the water laboratory PE studies are a critical
component of quality assurance programs for data collection activities conducted in support of
water programs at the National, Regional, and State levels. Because of the consistent quality,
comprehensiveness, reliability, and availability of the water PE studies over the years, they are
the most widely used environmental laboratory performance monitoring tool hi the United States.
The majority of study participants expressed a belief that OW laboratory PE studies constitute
an important and cost-effective quality assurance tool for program planning, regulation
development, water quality assessment, compliance assessment, and enforcement activities.
They also support quality assurance for Regional and State ambient water quality monitoring and
discharge permitting programs.
The Water Supply (WS) laboratory PE studies are an integral and mandatory component
of EPA's Drinking Water Laboratory Certification Program required under the Safe Drinking
Water Act. All but two states use the WS studies to certify laboratories for drinking water
analysis. Many states use the WS results to certify environmental testing laboratories for other
purposes as well. The Water Pollution laboratory PE studies are critical to NPDES, ambient
water quality monitoring, and hazardous waste laboratory accreditation programs conducted by
-------
vi
the States. In some cases, State statutes and regulations include requirements for participation
in the Water Pollution studies as a condition for laboratory accreditation. The results from the
water laboratory PE studies conducted by EMSL-Ci and EMSL-LV are used not only to support
water program quality assurance programs, but also to support many other types of Federal and
State environmental programs (i.e., Resource Conservation and Recovery Act, State hazardous
waste programs, other State programs and programs in the Departments of Defense and Energy).
Current and Future Needs for Laboratory PE Studies
The study shows that there are current unmet or potential needs for laboratory PE study
data in nearly all OW programs.
Water Supply Studies
The Drinking Water Laboratory Certification Program will experience substantial new
requirements for PE studies over the next two to five years as a result of proposed and
anticipated regulatory changes. New monitoring requirements for dioxin and asbestos in
drinking water will result in additions to the chemistry component of the WS studies hi FY
1994. The WS radiochemistry component will require new studies to test performance in radon
measurement in FY 1994 also. Additional regulations requiring increased monitoring of
drinking water supplies for viruses (such as Giardia) and other waterborne pathogens (such as
Cryptosporidium) will result hi expansion of the microbiology studies hi the next two years. As
the drinking water program continues to develop over the next five years and additional
standards are developed, proposed and promulgated, the scope and size of all three components
of the WS studies are expected to continue to grow.
Water Pollution and DMROA Studies
As emphasis shifts in wastewater and surface water monitoring towards biological
indicators of ecosystem health and new programs for clean-up and control of contaminated
sediment and sludge are implemented, laboratory PE materials and studies will have to shift to
test laboratories' ability to measure a wide range of analytes hi complex matrices. These
matrices include fish and animal tissue, sediment, and sludge.
-------
Vll
Laboratory PE materials are needed now or in the future for many additional
environmental matrices including marine and estuarine water, wastewater, soil, sediment, sludge,
and plant and animal tissue. Numerous chemical and biological analytes and tests, not presently
included hi water laboratory PE studies, will be needed hi the next five years as a result of new
requirements for monitoring of surface water discharges. For example, new emphasis on
studying the sources of microbial chinking water contaminants found hi surface water sources
of drinking water may link surface water discharges to violations of drinking water standards.
The linkages may result hi imposition of microbiological monitoring requirements for NPDES
permittees with a concurrent need for inclusion of microbial agents hi the DMRQA PE studies.
Needs such as these will cause the scope of both the DMRQA and the WP studies to grow and
change.
Findings: Efficiency of the Current Laboratory PE Studies
The review identified numerous opportunities for achieving technical, administrative, and
cost efficiencies hi the water performance evaluation study program. They include the
folio whig:
• Reducing the scope of the current studies by making changes to the
technical design (e.g., by reducing the numbers of analytes hi each
study);
• Combining the WP and DMRQA studies into one study designed
to achieve the combined objectives of both;
• Combining the like components of all three studies into one set of
chemistry, microbiology, radiological, and toxicity testing studies
designed to meet all water program needs;
• Developing a system for distributing the DMRQA study test kits
directly to the laboratories involved rather than to the more than
7000 permittees;
• Examining opportunities for improving the efficiency of the current
studies by automating recordkeeping functions; and
-------
vm
• Investigating options for improving study timeliness by electronic
transfer of study results.
As a result of this study, OW is examining each of these opportunities in FY 1994 to determine
whether they represent potentially significant time and cost savings. In particular, in FY 1994,
OW is forming a technical work group to examine opportunities for improving the studies and
to develop an implementation plan. This effort will include examining options for combining
current studies. Any decision to implement changes in the program must consider the potential
impact on State programs and recommend an approach which allows adequate lead time.
Findings: Current and Potential Prosram Costs
As detailed in Exhibit ES-2, the water laboratory PE studies cost hi excess of $2 million
in FY 1993. The total resource requirement for the water PE studies in FY 1993 was $2.34
million in extramural expenditures and 16.12 FTE. Of this amount, the DMRQA studies
account for the largest share of the extramural expenses ($1.17 million) and the WS studies
account for the largest share of intramural resources (11.91 FTE). The total requirement
represents an increase of 11 percent over FY 1992 levels, with the largest increase experienced
in the WS chemistry study (25 percent). Because OW needs for laboratory PE study needs
continue to grow, costs will continue to increase over the next decade. Exhibit ES-3 identifies
major unmet current and projected future needs for new PE studies, changes in study size and
scope, and changes in PE materials and matrices.
Data show that, although laboratory PE studies are considered mission critical by most
OW, Environmental Services Division, and Office of Research and Development managers and
staff interviewed, the allocated budget is not sufficient. ORD reprograms end-of-year dollars
annually to cover budget shortfalls. Reliance on end-of-year funds jeopardizes the studies both
from the standpoint of funding and technical adequacy. As a result, OW and ORD have
maintained a minimal water PE study program that addresses only the most fundamental needs.
In addition, needed expansions in the program have been postponed and needs currently exist
that are not addressed by the present study designs. In fact, as costs have increased over time
-------
IX
EXHIBIT ES-2
COST OF THE WATER PE STUDIES
FY1993
STUDY
WS Chemistry
WS Microbiology
WS Radiochemistry
WP
DMRQA Chemistry
DMRQA Toxicity Testing
Effluent Guidelines
FY 1993 $
$493,400
260,900
0
488,200
681,300
339,600
80,000
FY1993FTE
4.22
.91
6.78
2.47
.89
.85
0
TOTALS
$2,343,400
16.12
-------
KJ
* K
3
li
5 i
a
*?
8 § ~
II
a *
I §
I g
§ *
5
I E
&
te
N
1
1
I
M
I
w§
a.
-------
XI
and availability of year-tend funding has decreased, changes hi study design have been necessary.
The program has actually eroded over time. Continued funding shortfalls may necessitate
additional changes hi study frequency or design. Such changes will have detrimental effects on
State certification programs as well as EPA quality assurance and programmatic oversight
activities.
The report emphasizes that finding short and long term funding approaches must be a top
priority for OW and ORD during FY 1994 and beyond. Another key recommendation is that,
in FY 1994, OW examine three long term funding options and make a commitment to
implementing a long term funding strategy for laboratory PE studies, hi conjunction with ORD
and other EPA programs.
Options for Funding Laboratory PE Studies
The Agency's laboratory PE study needs in general and OW needs hi particular, are
growing rapidly. A reliable approach to funding is required to ensure that all program needs
are met as efficiently and cost-effectively as possible and that costs are distributed equitably
among EPA programs that are study users. The study recommends that funding issues and
approaches be addressed at two levels: within OW and Agency-wide.
Funding Approaches for OW
The report recommends that the Office of Water work with the Office of Research and
Development and the Environmental Monitoring Management Council (EMMC) to examine all
possible alternatives for funding water performance evaluation studies and develop both a short
term and long term approach to funding the water studies.
In the near term, OW should assume greater responsibility for funding the water
laboratory PE studies until a permanent solution is implemented. There are two potential
sources for additional funds: (1) OW can redirect program funds to the PE study program at the
expense of other priorities and (2) the Assistant Administrator for Water can request that ORD
-------
Xll
place a higher priority on funding the water PE studies at the expense of other water research
priorities. ORD has suggested transferring the R&D base funding currently used to support the
PE program to the OW AC&C base budget to support a permanent line item in the OW budget.
OW should work with ORD to implement the transfer of resources. As part of this approach,
OW will have to decide whether to continue to use ORD staff to design and administer the
studies and analyze the results, have OW staff assume these responsibilities, or contract out the
activities to a third party. This approach has numerous resource implications, including the
possibility of transferring FTE from ORD to OW for purposes of administering the studies.
In addition to implementing strategies for securing additional funding for the water PE
studies, OW will pursue alternatives for improving the efficiency of the current program, as
recommended by the report. These include options for consolidating the studies and automating
information management aspects of the studies.
For OW, one long-range funding option is to generate user fees to support the water
laboratory PE study program. Currently there is no statutory authority for fees collected under
such a program to return to EPA. OW is investigating the merit of fees and addressing issues
pertaining to retiming fees to the Agency during the reauthorization of the Clean Water Act and
the Safe Drinking Water Act. Implementation of this option would be a 3 to 5 year process.
In addition to obtaining statutory authority for collecting and retaining fees, OW would have to
set up a fee-structure and collection process. EPA would also have to allow the States several
years to change their statutory and programmatic processes to accommodate the fee program.
Various alternatives also exist for funding laboratory PE studies through external sources.
These options should be fully explored and considered. For example, the EMMC is hi the third
year of an effort that, if successful, will lead to establishment of a national program for
environmental laboratory accreditation. This national accreditation program could be designed
as EPA's sole source of laboratory PE materials and studies. The EMMC anticipates, however,
that, once established, a national accreditation program would take at least 5 years to implement.
-------
Xlll
Agency-wide Approaches to Funding PE Studies
As a result of this pilot project, the Assistant Administrator for Water will recommend
to ORD and the Administrator that an Agency-wide initiative on quality assurance and laboratory
performance evaluation studies be undertaken. The initiative should identify budgetary and
statutory alternatives for establishing a comprehensive quality assurance program within EPA,
including a strong laboratory oversight component based on performance evaluation studies.
Other EPA program offices should conduct studies similar to OW's to document needs for PE
studies. The Agency-wide initiative on quality assurance should include a component to support
laboratory PE studies, as part of the FY 1996 budget development process. The initiative should
promote establishment of a permanent change in the Agency's budget development and
formulation processes to ensure funding to support quality assurance, including laboratory PE
studies. The QA budget appropriation could take the form of a centralized or decentralized
Agency-wide tap on the budget, a request for an increase in the Agency's base budget, or a
redirection of programmatic funds to provide sufficient funds for all of EPA's laboratory PE
study requirements. Part of this effort would include determining whether AC&C or R&D
dollars should be used to fund laboratory PE studies and other quality assurance activities and
whether ORD or the program offices should have principal responsibility for providing the
necessary staff support.
Next Steps
During FY 1994, the Office of Water and Office of Research and Development will work
to develop a short term and long term funding approach for water laboratory PE studies. In
addition, as a result of this study, OW is now working with EMMC and ORD to establish an
Agency-wide work group to address common technical, programmatic, and funding issues
associated with laboratory PE studies. The work group will examine opportunities for improving
the efficiency of the existing water PE study program, such as combining the various studies into
one or two studies designed to meet multiple program needs, and will develop an action plan for
implementing program improvements.
-------
-------
OFFICE OF WATER
PERFORMANCE EVALUATION STUDY PROJECT
Final Report
I. BACKGROUND
In April of 1992, the Office of Water undertook a review of the laboratory performance
evaluation (PE) studies conducted for water programs by the Environmental Monitoring Systems
Laboratories (EMSL) hi Cincinnati and Las Vegas and by the Office of Science and Technology,
Engineering and Analysis Division. The review was prompted by continuing funding shortages,
reported by EMSL, and a recognition that available resources for quality assurance hi general,
and PE studies in particular, have been eroding over time. The goals of the study were to: (1)
determine the current needs for PE studies within the Office of Water and uses by Regional and
State programs; (2) anticipate changes hi PE study needs hi the next five years; and (3) develop
options and recommendations to establish a permanent solution to the problem of funding PE
studies hi the future. The study was conducted hi response to requests made by several OW
Division Directors who recognized the importance of PE studies as a tool for ensuring the
quality of monitoring data used to make program decisions.
The Importance of Quality Assurance
Environmental monitoring data provide the basis for many of the policy, regulatory, and
planning decisions made within the Office of Water and hi water programs at the Regional and
State levels. To ensure that monitoring data are of sufficient quality to support effective decision
making, the Office of Water, with support from the Office of Research and Development,
implements a quality assurance program, which includes a PE component. This program has
-------
evolved to protect against deficiencies in data that could lead managers to make technical,
regulatory, or policy decisions that fail to protect human health or the environment; use program
funds inefficiently or incorrectly; take actions that create a negative public image for EPA; lead
to lawsuits; or prompt Congressional inquiries.
The quality of monitoring data used hi EPA programs has recently been an issue of
concern to various Congressional and other oversight organizations. Audits and investigations
by the General Accounting Office and the Inspector General have called into question various
aspects of EPA's quality assurance programs. As a result, in FY 1993, the Agency has
recommended that EPA's quality assurance program documentation be declared a weakness at
the Presidential level under the Federal Managers' Financial Integrity Act (FMFIA). This action
underscores the importance of quality assurance and data integrity to water programs.
Laboratory Performance Evaluation Studies
Laboratory performance evaluation (PE) studies are one of a suite of tools used for
ensuring the quality of analytical data. PE studies are a relatively efficient and low cost tool for
testing whether laboratories have the equipment and technical ability to accurately analyze
samples. PE studies may be designed as "single blind" or "double blind" studies. In single
blind studies, laboratories analyze samples identified as PE samples containing concentrations
of analytes known to the study designer but not known to the laboratory. Double blind studies
involve submitting PE samples disguised as real samples to the laboratory so that the identity
of the sample and its composition are unknown to the laboratory.
PE studies involve analysis of solutions of known concentrations of analytes. The
solutions are sent to laboratories, hi the form of ampules which must be diluted to the
appropriate volume or as prepared samples, for analysis using specified analytical methods.
Laboratories report back the qualitative and/or quantitative results of their analyses within a
specified time period. The water PE studies are designed to be "single blind" or quantitative
challenge studies. Laboratories know that the solutions they receive are performance evaluation
-------
3
samples and they know the identity of the analyte groups in the solutions. They must determine
only the quantitative concentrations of the analytes using specified analytical methods. The
results are scored against performance criteria that are statistically or empirically based.
PE studies do not provide a continuous indication of laboratory performance over time,
measure the quality of particular data sets generated by a laboratory, or prevent laboratory fraud.
However, PE study results are valuable as an indicator of basic laboratory capability and
competency. An accurate assessment of the value of PE studies cannot be conducted in an
independent, stand alone context. The value of PE studies is most effectively demonstrated as
a basic component in the framework of an overall program for assuring the quality of
environmental measurements.
The Office of Water PE studies support planning, regulation development, assessment,
compliance, and enforcement activities of the Drinking Water Program, Office of Wastewater
Enforcement and Compliance, and the Office of Science and Technology. The PE studies also
support EPA Regional and State ambient water quality programs under: Clean Water Act
sections 106, 205 (g and j), 305(b), 314, 319; and Federal programs such as the National
Estuary Program, the Ocean Dumping Program, and the Effluent Guidelines Program. Regional
staff indicate that Water PE studies are used to support RCRA, CERCLA and other State
programs. Exhibits B-4 through B-13 of Appendix B provide a detailed profile of the types of
programs, at the State level, that utilize the results of the water PE studies. Appendix B also
indicates the types of program decisions supported by PE study results at the State level.
Office of Water PE studies support methods development for the Office of Research and
Development. They support water quality monitoring programs and provide laboratory
performance evaluation information for programs at the Regional and State levels. Exhibit 1
identifies the different PE studies conducted each year, their principal purposes, and the
organization within EPA that has lead responsibility for executing each study.
-------
EXHIBIT 1
WATER PERFORMANCE EVALUATION STUDIES
STUDY
Water Supply (WS)
Chemistry
Microbiology
Radiochemistry
Water Pollution (WP)
KEY PURPOSE
Drinking Water Certification
Drinking Water Certification
Drinking Water Certification
Monitor performance of
laboratories that generate
ambient water quality monitoring
data.
LEAD ORGANIZATION
EMSL-Cmcinnati
EMSL-Cincinnati
EMSL-Las Vegas
EMSL-Cincinnati
Discharge Monitoring Report
Quality Assurance (DMRQA)
Chemistry
Toxicity Testing
Effluent Guidelines Quality
Assurance
Monitor quality of chemistry data
submitted by NPDES permittees in
Discharge Monitoring Reports.
Monitor quality of toxicity data
submitted by NPDES permittees
in Discharge Monitoring Reports.
Monitor performance of contract
laboratories that analyze wastewater
samples to support effluent guidelines
development.
EMSL-Cincinnati
EMSL-Cincinnati*
Analytical Methods Staff
The Office of Wastewater Enforcement and Compliance collaborates with
EMSL-Cincinnati on study design and implementation.
-------
Study Goals and Approach
The Office of Water Performance Evaluation Study project was designed to achieve three
principal goals:
1. Determine the importance of performance evaluation studies for
water programs as indicated by participation in the studies and use
of the study results.
2. Evaluate the efficiency of current performance evaluation studies.
3. Determine current and potential costs and identify funding
mechanisms for performance evaluation studies.
To achieve these goals, data and information were gathered from seven principal sources: EPA
Headquarters Office of Water, EPA Regional Water Management Divisions, EPA Regional
Environmental Services Divisions (monitoring and quality assurance staffs), State water and
quality assurance programs, the Office of Research and Development (both at EPA Headquarters
and the EMSL laboratories), other EPA Program Offices at Headquarters, and other Federal
agencies. The study involved three principal data collection steps: (1) a survey of performance
evaluation study users, conducted with the assistance of the Regional Quality Assurance
Managers, (2) interviews with staff at EPA Headquarters and at other Federal agencies that use
or conduct laboratory performance evaluation studies, and (3) interviews with staff at the
Environmental Monitoring Systems Laboratories (EMSL) hi Cincinnati and Las Vegas.
II. PE STUDY USES AND NEEDS
The results of PE studies (i.e., whether a laboratory is successful hi identifying and
quantifying the analytes of concern) are used in a number of ways by Headquarters, Regional,
and State water programs, other EPA Regional and State environmental programs such as RCRA
and State Hazardous Material Programs, and by the Office of Research and Development. They
-------
are also useful to the laboratories themselves, as a method for identifying and correcting
potential performance problems and as a tool for demonstrating their capability to prospective
clients. Exhibit 2 identifies users of PE study data and summarizes their principal uses.
Appendix B provides a summary of data collected through a survey of water PE study users at
the Regional level. Each of the ten Regional Quality Assurance Managers collected quantitative
and qualitative information concerning uses of PE study results within their Region. Information
sources consulted included Regional Water Management Division staff (including the Discharge
Monitoring Report Quality Assurance (DMRQA) Coordinators, Water Quality Monitoring
Coordinators, Regional Environmental Services Division (ESD) laboratory directors, and State
environmental and laboratory certification program representatives.
The survey requested information concerning:
• Numbers of laboratories in each Region participating hi the water
PE studies and their purpose for participating;
• The types of Regional and State programs that use the results of
the water PE studies;
• Regional and State practices concerning the number of times per
year that laboratories are requested or required to participated in
the water PE studies;
• • The types of program decisions supported at the Regional and State
levels by the results from water PE studies;
• The adequacy of the current water PE studies for meeting Regional
and State program needs;
• Additional analytes and matrices for which PE materials and
studies will be needed hi the future;
• States that have statutes or regulations requiring the use of the
water PE studies; and
-------
Kl
S
^
1
I
£
CO
h-
UJ
O
STATES
Q
O
(0
O
O
UJ
cc
w
cc
UJ
1—
cc
a
Q
^
UJ
X
>
O
o
Q
CO
0
CO
uj
(O
o
o
LU
5
n
o
o
5
Q
1
O
STUDY
|
'1^1 1
?|s 1
£- s« 1
ftfll-f
S | < •£ -g ? «
illil*5
«- ci ri •»
•X
Ifllll
^sq^tci
g
•II
u a
*- «
£
SI,
Hi!
g c o c S
-o £i£ tt
1 | | S |
55 E ir c o
*- c«i co
E
o
O
g
> f
7 - S
i s||
SfilM
il ;1 s|f
oslll^3
«- N n ^j- u>
^
S 5 S t « I
• 6 S S = «
2 .g 6 2. o 1
• o > a- » «
^s9Sll
C
O
p •
i <
u o
»- N
tl
tf |
I ^ |
1 .1 i J 1
llll I
CD p C c O
^ ci ri
2
u
I
W
f j||
Slllsii
T- 01 ri if ID
ca g
I-S 8 e« 1 . E
*• ci co"
C
O
U
ss
•- W
5
11
llfi
fill
r- M CO
f
1 ?
II
•- CM
Q
S
(O
?H Is.- 8
11 -5 1 •£ •! • i 1
nS6=J PS-8&-C
O-D-^C < 0 — 0 j
i|jj |ifi|
ijljgjljijl
•- c4
a
II Illl
$sq«,f£
g
111
«- CM
H s
S "i -5 S I «
Htii!
0.
8
So — i
i a S S
1 = = g S
. I^Sfi
iniM
llfSii
3IS53I
*- CM CO *<• ID
U
?l
i2
1]
"o
T^
2
~3 "
I if ll
•- CN ri «f
I g I
f flat
•£ a. J O »
^ ™ S"-
DMRQA
Chemical &
Whole Effluent
Toxicity
1
to <8
3 .fc
CD fc
> O
l2 Q
II
E I
•§•§
=. E
1 Effluent
Guidelines
o
a
c
o
u fl)
•aS1
O 0)
11
£5
u i'
-.
U
Si_
Q>
*- •w
II
-------
• Regional processes for managing participation in the water PE
studies by State and other laboratories.
The information collected through the survey is displayed in a series of summary tables and
graphics in Appendix B. These tables form the basis for conclusions regarding current uses of
PE study data and support the analysis of current and future PE study needs.
Current Uses ofPE Study Data
Overview
The principal use for the Water Supply (WS) and Water Pollution (WP) PE studies is
laboratory evaluation, accreditation or certification. States use the EPA water PE studies for
this purpose, because they are widely available, reliable, cover a wide range of regulated
analytes, and aid reciprocity among States. PE studies for laboratories are critical to EPA
because they are the only standard and unbiased tool we have available nationally to measure
laboratory capability for EPA-regulated analytes.
State Use
In order for a State to receive primacy for the State drinking water program they must
meet a national certification requirement. Successful completion of one PE study per year for
each analyte certified (the Water Supply PE study) is required for laboratories to be granted
certification under the program. In addition, certified laboratories must pass an on-site audit
conducted by State auditors at least once every three years. Nearly all States hold primacy for
the Safe Drinking Water Act program and consequently, operate the required Drinking Water
Certification Program. All but two primacy States (New York and Illinois) use the EPA Water
Supply PE Study for laboratory certification, identifying laboratories with performance problems,
and targeting auditing priorities. New York and Illinois have their own PE study programs.
The analytes and ranges covered by these State programs are virtually identical to those available
-------
through EMSL-Cincinnati.
State certification programs also constitute the single most important user community for
the WP studies. Exhibits B-9, B-10, and B-ll in Appendix B show that almost half of the States
use the Water Pollution (WP) Studies to certify laboratories analyzing samples for the National
Pollutant Discharge Elimination System Program and the State Ambient Water Quality
Monitoring Program. The Regional Quality Assurance Managers reported that discontinuing or
decreasing the frequency of the WP studies would have a significant detrimental effect on state
programs. Another 10 States use the Discharge Monitoring Report Quality Assurance
(DMRQA) PE study to certify laboratories analyzing samples for the National Pollutant
Discharge Elimination System (NPDES) Program. Many States not using the Water Pollution
or DMRQA studies for certification use the studies to identify laboratories with performance
problems and to set priorities for laboratory audits.
Exhibits B-8, B-ll, and B-26 through B-30 of Appendix B show that approximately 40
States currently operate environmental laboratory certification programs for analyses conducted
pursuant to environmental regulations and programs other than those generated for the Public
Water Supply Program, NPDES permitting, and ambient water quality monitoring (i.e.,
programs administered under authority of the Clean Air Act, Resource Conservation and
Recovery Act, and Superfund). Many of the analytes included in water PE studies are also of
concern to other programs. Moreover, the water PE studies are widely used in the commercial
and government laboratory communities. Other EPA programs have not developed, or are just
now developing, appropriate PE samples and studies to support their performance evaluation
needs. Consequently, the results from the WS, WP and the DMRQA are also used as the best
available tool for purposes of making certification decisions, for other environmental programs.
EPA Regions
At the EPA Regional level, water PE studies are used for several purposes. The Regional
BSD laboratories participate hi the Water Supply PE Study to monitor their own performance
-------
10
and maintain drinking water certification. Drinking Water Certification Officers monitor the
performance of State principal laboratories and use PE study data to grant or revoke their
certifications. In cases where the States do not have primacy, EPA Regional Drinking Water
Certification Officers certify commercial and municipal drinking water laboratories located in
the State. Results from Water Supply PE Studies are also used to set priorities for on-site
audits.
BSD laboratories participate in Water Pollution PE Studies to monitor their own
performance and so that they can evaluate the performance of State principal laboratories for
ambient and wastewater monitoring under the Clean Water Act. Study results are used to assess
ambient and wastewater methods for program grant requirements.
Regional DMRQA coordinators use the results from the DMRQA PE Studies to identify
laboratories with potential performance problems. The results are also used to set priorities for
conducting Performance Audit Inspections for NPDES permittees and enforcement actions.
ORD Laboratories
The EMSL laboratories use PE study results hi evaluating national laboratory
performance data, defining training needs, developing interlaboratory methods performance, and
method validation information/acceptance criteria. Other ORD laboratories request PE studies
on an as needed basis for quality assurance associated with special in-house projects and to
evaluate the performance of contract and grantee laboratories. The EMSL-Las Vegas laboratory
operates the drinking water certification program for radiochemistry and conducts the WS
Radiochemistry studies. EMSL-Las Vegas uses the WS Radiochemistry results to grant or
revoke drinking water certification for radiochemistry analytes.
EPA Headquarters
Office of Water Headquarters programs also use PE study data. Results from the WS
-------
11
studies are used in setting Maximum Contaminant Levels (MCLs) for drinking water,
particularly for carcinogens where the MCL must be set as close to zero as is practically
possible. Performance evaluation studies conducted by the Office of Science and Technology
as part of the Effluent Guidelines Quality Assurance Program are designed specifically to
monitor the performance of contract laboratories that supply data for development of effluent
guidelines. Although the results of the WP studies are not used by Headquarters program
presently, OWOW recognizes the important role of the studies as a tool for overseeing the
quality of data entered into STORET. OWOW is investigating approaches to making more
direct use of the WP studies in the ambient water quality monitoring program at the national
level.
The DMRQA chemistry and toxicity testing PE studies are designed specifically for
Office of Wastewater Enforcement and Compliance. The data generated are entered into the
Permit Compliance System (PCS) data base and support program planning and compliance
activities.
Current and Future PE Study Needs
There are potential applications for PE study data hi nearly all Office of Water programs.
Interviews with program staff identified numerous areas where new monitoring programs or
requirements are in development. As emphasis in monitoring shifts toward biological indicators
of ecosystem health and new programs for clean-up and control of contaminated sediment and
sludge are implemented, there will be needs for PE materials and studies that test laboratories'
ability to measure a wide range of analytes in more complex matrices, such as fish and animal
tissue, sediment, and sludge and other new biological PE needs. Exhibits 3, 4 and 5 summarize
needs for additional PE studies.
In general, needs for new PE study materials or analytes derive from statutory
requirements for new monitoring programs or requirements. Following passage of the 1987
amendments to the Clean Water Act, for example, OW began to design national programs in
-------
G n
I f
1 i it
§
§
e
t
*
5 -p S
J S
«
o
o
_*»
0>
«•»
5§
sS:
ft" f
Q i
i M
g |
8 «
! l
i -5
i*
£
Si<
S
|
5
i
^«
E
&* 1
5§ is f-
Il II I1
-------
Q
£:
I
CO
O s*
S< sa<
S 2*
Q AH
S to
I
§
|
5j
1
I
8
8-
2
1
1
1
o
2
|| Estuarine water
1
1
1 Wastewater
3
o
,0
o
1
£**
•"
«5
|J
§11
I
co
Soil, Sediment and
03
a
o
*£
.S
11
2 H
I
'S
!
PH
|| Flammable waste
g
§
Giardia
Cryptosporidi
Legionella
Viruses
Microbiologicals
Trace metals
Organics
Pesticides
Herbicides
I
Tissue:
- Fish (filets and )
- Invertebrates
- Mammals
- Plankton
•8
S
oo
•4-J
&
Vertebrate)
>^^
en
|| Biological material
-------
05
1
9
& a
CO
fej « 1
23 fcl co
3 5< s
« § €
8 IS
1
I
§
S, 8 ff-a .
•p to .2 ^, .
-------
15
areas such as contaminated sediments and sludge. In general, following promulgation of rules
or guidance establishing the national program, States incorporate the changes and, as permits
expire and are reissued, the new monitoring requirements are incorporated and the need for
laboratory oversight in the form of PE studies is realized. Because a period of years elapses
before new monitoring requirements begin to be incorporated into permits, many of the future
needs for PE materials cited by Office of Water staff could not be quantified. In order to fully
understand the extent to which new monitoring requirements will generate demand for PE studies
and contribute to expanding PE study costs, a quantitative predictive model or other tool may
be needed.
Water Supply
The most immediate needs for new PE materials and analytes derive from recent and
planned additions to drinking water regulations. PE studies for asbestos, dioxin, disinfection by-
products, and radon will be needed during FY 1993 and FY 1994. In addition, the Safe
Drinking Water Act Amendments require that MCLs be promulgated for 25 new contaminants
every three years. As standards are developed and promulgated, the WS studies will need to
be expanded.
Presently, the WS Chemistry study includes both regulated and unregulated analytes.
Unregulated analytes are those for which MCLs are proposed or expected to be proposed hi the
future. Such analytes are included in the study in order to generate data to support development
and validation of analytical methods. Analyses for unregulated analytes are not mandatory for
certification. In order to maintain the WS as a source of method validation data, expansion of
the study to account for planned MCLs will need to occur one to two years hi advance of final
rulemakings.
OGWDW staff and representatives from EMSL-Cincinnati, the Regions and the States
also reported a need for expanding the number of laboratories included hi the WS Microbiology
studies for total and fecal coliform bacteria. The study is now distributed to the EPA Regional
-------
16
and State principal laboratories (approximately 250 laboratories). Local laboratories, which
number between 3,000 and 5,000 and include municipal, utility-owned, and commercial
laboratories, are not now included in the study. Several States reported that they require local
laboratories seeking certification for microbiological analytes to purchase and analyze
microbiological PE materials from commercial sources. New York and Iowa reported that they
manufacture and distribute their own PE materials for this purpose. The Regional Quality
Assurance Managers noted that the importance of microbiological parameters from a public
health standpoint makes this aspect of the program important. They recommended that the
Office of Water and EMSL-Cincinnati take steps to expand the WS Microbiology study to
include local laboratories in the future.
Discharge Monitoring Report Quality Assurance (DMRQA)
The most recent DMRQA PE study contained 15 metals, 5 nutrients, and 10 other water
chemistry analytes. OWEC and EMSL-Cincinnati representatives both noted that emphasis on
toxic organic pollutants, particularly the primary pollutants, hi permit monitoring requirements
is increasing. Consequently, they recommended that the DMRQA chemistry study be expanded
in the future to cover toxic organic analytes. The shift from technology-based effluent limits to
water quality-based effluent limits creates a need to provide PE samples with lower
concentrations of analytes currently included as well as to expand coverage to new analytes.
The DMRQA toxicity testing PE study is a relatively new study which is still hi
development. In FY 1992, the study tested only a portion of the toxicity testing methods
presently hi use by the regulated community. Both OWEC and EMSL-Cincinnati staff noted that
considerable additional work will be needed to expand the studies to address additional methods
presently allowed hi NPDES permits.
//I. SCOPE AND COST OF THE CURRENT PE STUDY PROGRAM
In FY 1992, the water PE studies cost an estimated $2.63 million. Exhibit 6 provides
-------
EXHIBIT 6
SIZE, SCOPE AND COST OF WATER PE STUDIES
FY1992
17
STUDY NO. LABS PER YEAR
PER YEAR ANNUAL COST
WS Chemistry 4000
$395K
WS Microbiology 520
$244K
WS Radiochemistry 355
$516K**
WP 5000
$456K
DMRQA Chemistry 7500***
$637K
DMRQA Toxicity Testing 800
$295K
Effluent Guidelines 9
$83K
TOTAL $2.63M
NO. ANALYTES NO. AMPULES
196 116,000
4 7,800
33 4,500*
152 190,000
30 75,000
16$ 400
NA NA
**
***
WS Radiochemistry study uses cubitainers rather than ampules.
Based on an average cost of $76,106 per FTE (for 6.78 FTE). This average dollar value
includes labor and other program costs that are not tracked independently, such as
shipping, stockroom supplies, word processing, graphics support, copying and mailing.
For DMRQA Chemistry, test kits are distributed to permittees rather than to laboratories.
For DMRQA Toxicity Testing Study, laboratories measure the toxicity of only one
analyte using only one of 16 possible test methods.
-------
18
a summary of total costs for each of the studies. More detailed FY 1992 cost data, for the
studies conducted by EMSL-Cincinnati and by EMSL-Las Vegas, are provided in Appendix C.
Exhibit 7 summarizes FY 1992 costs (in dollars and FTE) and provides an estimate of FY 1993
costs for each study, broken out by study phase. The largest projected increase in cost (25
percent) is expected to be for WS Chemistry. This increase is attributable to the increased
number of analytes in the study, hi response to expanded regulations, as discussed previously.
The DMRQA Toxicity Testing study is expected to experience a 15 percent increase in costs hi
FY 1993. This increase is also attributable to inflation and changes hi the study. Other studies
will experience smaller cost increases attributable to inflation.
Exhibit 8 shows the funds available in FY 1992 and FY 1993 to EMSL-Cincinnati for
conducting the WS Chemistry and Microbiology, the WP, and the DMRQA studies and
compares the percent change hi funding to the projected increase hi funding requirements. In
all cases, funds available hi FY 1993 are expected to be less than funds available hi FY 1992.
As stated previously, the largest increase hi needs will be for the WS Chemistry and the
DMRQA Toxicity Testing studies.
Exhibits 9 and 10 provide estimates of costs associated with the principal new needs for
PE studies identified by Headquarters staff during interviews. Exhibit 9 shows those that can
be quantified based on available information, all of which will become critical hi FY 1994.
Exhibit 10 provides cost estimates for needs that will be realized hi approximately FY 1998 or
later. Information concerning the scope and extent of the needs shown in Exhibit 10 was not
available. Consequently, projected costs could not be accurately quantified.
Opportunities for Achieving Cost Efficiencies
Throughout the hiformation collection phase of the project, representatives from OW
programs, EMSL-Chichmati, EMSL-Las Vegas, and the Regions made numerous suggestions
for changing the PE studies hi order to make them more efficient and/or cost-effective.
Interviewees also suggested that options for covering the cost of PE studies through external
-------
I
I
13
§
s
§
03
-------
i
§
gj
S;
g
§
tC(
9
3
^5
1
,j
S
^3
00 §£
&5 |S
flQ S^ 5
5 *3
N cR §
»*j ^ >•—
te
fa
**4
'^j
"xj
|
••"<
£ .
i
**5
1
o
t>
S § ^
fe i§ «!
s *^ s
S o s
fell
Os
5 faj
^ Kj .w
^ ^ "^
^ ^ §
s ^ g
^ fe: ^
1"
b]
ai
2j 2 $2
N "^ ^
i
^
a
S\ S o
? CO §
1
CO
^
r5
6?
t~
^
M
**J
9
&
en
t-
»-^
5
I
GO
^
c-
o
o
0
Microbiology
oo
^
a
Z
o
I
%
0
a
%
o
%
p
!-H
3
Radiochemistry*
C/3
^
[7
O;
'
VO
s
T—t
t>
g
CU|
^
+
ON t^
1 '
•* O.
SJ §
01
"S^s
_> g
Q> CO
T3 O
w" g
H g
TT ^»
ti, g
* . .
W *4^
£< °
§1
\o u
<*H «>
O U
ts«
80)
"Jrt
a^
w ^
aJ *o
S 3
9 *
a >
i^
Crl ^
N
oo - ri
t^ g ON
vo f os
** O rt
§5£
1 f5
•° i 13
b.2 3
•I S 8
Is *
•s» a
i2S
« . o\
ft} fM -•
BS|
trf .4— * >S
.2 S «
^ 8 »
2
-------
EXHIBIT 9
ESTIMATED COSTS FOR MEETING
NEAR-TERM WATER PROGRAM PE NEEDS
FY1994
EST. TIME FRAME PROGRAM/NEED
FY 1994
FY 1994
FY 1994
FY 1994
Drinking Water/Asbestos
Drinking Water/Dioxin
Drinking Water/Micro
Expansion
Drinking Water/Radon
EST. ANNUAL COST
$ 50,000 - $150,000
$ 60,000 - $100,000
$240,000 - $350,000
$250,000 - $400,000
$519,000 - $1,050,000
EXHIBIT10
ESTIMATED COSTS FOR MEETING
LONG-TERM WATER PROGRAM PE STUDY NEEDS
EST. TIME FRAME
FY 1998
FY 1998
FY 1998
FY 1998
Undetermined
PROGRAM/NEED
Metals in Sludge
Sediment Chemistry
Sediment Toxicity
Biodiversity
Pathogens in Sludge
and Drinking Water
EST. ANNUAL COST
$450,000 - $650,000
> $500,000
Unqualified
Unqualified
Unqualified
-------
22
sources (e.g., through user fee systems similar to those used by State laboratory certification
programs) should also be investigated. Many also noted a link between the PE studies and the
Environmental Monitoring Management Council (EMMC) initiative on national laboratory
accreditation. Incorporating OW's PE study requirements into a national environmental
laboratory accreditation program would also be an option for covering the PE study costs
through external sources. A summary listing of the design, administration and funding issues
identified during the project appears hi Appendix D.
Regarding the WS Chemistry study, EMSL-Cincinnati representatives made two principal
suggestions for reducing study costs: (1) reducing the number of analytes tested hi each study
and (2) reducing the frequency of the study from twice each year to once annually. Reducing
the number of analytes in the study could be used to make the study both a qualitative challenge
and a quantitative challenge to laboratories, if the identity of the analytes included hi the study
were unknown to laboratories. Laboratories would then be required to determine which of the
regulated analytes appear hi the ampules and make a quantitative determination of the
concentration for each analyte.
The qualitative challenge aspect of the study design would have the further advantage of
improving the extent to which the study tests laboratory capability. Presently, for the most part,
laboratories know which regulated analytes are present in each of the ampules for a study. They
need only determine the quantitative concentrations. If the study were changed to be a
qualitative challenge also, laboratories would still need to run the full set of analytical methods
to determine which analytes are and are not present. Adding the qualitative challenge aspect
would require a re-examination of the requirements for drinking water certification, however,
since certification is now dependent on passing the quantitative challenge samples for each
analyte.
It is not clear that making the WS Chemistry study a qualitative and quantitative
challenge would result hi significant cost savings. Such a change would reduce the number of
analytes tested and could potentially reduce the number of ampules manufactured and distributed
-------
23
for each study. The data returned to EMSL-Cincinnati by the laboratories would also change
and modifications to data reporting sheets and the scoring process would be needed.
EMSL-Cincinnati has given consideration to reducing the frequency of the WS study in
recent years. Presently, most States and Regions require that laboratories participate in the first
study of each year to maintain certification. Laboratories participate in the second study if they
did not participate in the first study or if they fail to meet any of the performance criteria in the
first study. Only those analytes missed hi the first study are required to be analyzed in the
second. Subsequent failure for the same analytes hi the second study can result hi a laboratory
losing its drinking water certification. Decreasing the frequency of the WS study would have
the effect of requiring laboratories to wait for a full year before having the opportunity to
demonstrate proficiency on analytes missed hi a particular study. States and Regions would need
to develop revised certification policies and requirements if the frequency were changed.
However, reducing the WS to one study per year would result hi a significant cost savings.
Two principal opportunities for increasing the cost-effectiveness of the DMRQA
Chemistry study were cited by Office of Water and EMSL representatives: (1) combining the
WP and the DMRQA Chemistry study into one study designed to meet both wastewater and
ambient water monitoring needs; and (2) developing a system for requiring NPDES permittees
to designate laboratories for receipt of then: PE study test kits as an alternative to the present
system which distributes test kits to all designated permittees. The potential for combining the
WP and DMRQA Chemistry studies has been examined previously by a technical work group
of OW, ORD, and Regional representatives. The most often-cited barrier to combining the two
studies is that the concentration ranges of interest to the two programs (ambient water quality
monitoring and NPDES) are significantly different. A comparison of the analytes and
concentrations tested hi the two studies shows considerable overlap hi the analytes tested by all
three chemistry studies (see Appendix E). Moreover, the concentrations tested hi the WP and
DMRQA studies are essentially the same (as demonstrated by the comparison analysis and
confirmed by EMSL-Cincinnati). Combining the studies therefore has significant potential as
a cost-savings alternative.
-------
24
In the DMRQA Toxicity Testing study, participating NPDES permittees are contacted
and asked to designate a laboratory for receipt of their test kit. Kits are then distributed to the
laboratories directly and the laboratories report the results back to their client(s). The permittees
report the results to EPA. OWEC estimates that this system allows EPA to send out 60 percent
fewer test kits than would be needed if the kits were sent to all participating permittees. A
similar system could be developed for the DMRQA Chemistry study to reduce the number of
kits distributed (approximately 7300 in the FY 1992 study). Such a system would require
collection of additional information from the permittees (i.e., each would have to designate a
laboratory or laboratories for receipt of the kits) and the information would have to be verified
and updated annually. OWEC is presently working with EMSL-Cincinnati to determine whether
such a system would result in meaningful cost savings or program efficiencies.
During the EMSI^Cincmnati site visit, numerous opportunities for automating the PE
study records and information management system were identified that would benefit all of the
studies. Potential opportunities exist for using machine-readable data reporting forms and for
electronic transfer of test results and summary reports to Regions and States, for example. A
review of the present information management system used to store PE study results could be
useful for identifying opportunities for cost and time savings.
TV. FUNDING ISSUES AND OPTIONS
The water PE studies will cost hi excess of $2 million hi FY 1993. Because the Office
of Water PE study needs continue to grow, it is anticipated that the cost of the studies will
continue to increase over the next decade.
Although PE studies are considered mission critical by most Office of Water,
Environmental Services Division, and Office of Research and Development managers and staff
interviewed, the Agency budget is not sufficient to cover the entire cost of the studies.
Consequently, ORD reprograms end-of-year dollars annually to cover budget shortfalls for the
studies. This reliance on end-of-year funds to supplement PE study budgets jeopardizes the
-------
25
studies because funding cannot be guaranteed from year to year. A funding shortfall would
necessitate either changes in study frequency or design and would have detrimental effects on
State certification programs as well as EPA quality assurance and programmatic oversight
activities.
InFY 1991, EPA's Environmental Monitoring Management Council (EMMC), the Office
of Water and the Office of Research and Development concluded that, because of the importance
of PE studies, there should be an Agency-wide initiative to establish a stable approach to funding
this important component of environmental quality assurance programs. The findings of this
study support the EMMC conclusion. It is therefore recommended that finding stable short and
long term funding sources be a top priority for the Office of Water and the Office of Research
and Development during FY 1993 and FY 1994. It is further recommended that funding issues
and approaches be addressed at two levels: within OW and Agency-wide.
Funding Approaches for OW
The Office of Water should continue working with the Office of Research and
Development and the Environmental Monitoring Management Council (EMMC) to examine all
possible alternatives for funding water performance evaluation studies and develop both a short
term and long term approach to funding the water studies.
In the near term, OW should assume greater responsibility for funding the water
laboratory PE studies until a permanent solution is implemented. There are two potential
sources for additional funds: (1) OW can redirect program funds to the PE study program at the
expense of other priorities and (2) the Assistant Administrator for Water can request that ORD
place a higher priority on funding the water PE studies at the expense of other water research
priorities. ORD has suggested transferring the R&D base funding currently used to support the
PE program to the OW AC&C base budget to support a permanent line item in the OW budget.
OW should work with ORD to implement the transfer of resources. As part of this approach,
OW will have to decide whether to continue to use ORD staff to design and administer the
-------
26
studies and analyze the results, nave OW staff assume these responsibilities, or contract out the
activities to a third party. This approach has numerous resource implications, including the
possMity of transferring FTE from ORD to OW for purposes of administering the studies.
In addition to implementing strategies for securing additional funding for the water PE
studies, OW will pursue alternatives for improving the efficiency of the current program, as
recommended by the report. These include options for consolidating the studies and automating
information management aspects of the studies.
For OW, one long-range funding option is to generate user fees to support the water
laboratory PE study program. Currently there is no statutory authority for fees collected under
such a program to return to EPA. OW is investigating the merit of fees and addressing issues
pertaining to returning fees to the Agency during the reauthorization of the Clean Water Act and
the Safe Drinking Water Act. Implementation of this option would be a 3 to 5 year process.
In addition to obtaining statutory authority for collecting and retaining fees, OW would have to
set up a fee structure and collection process. EPA would also have to allow the States several
years to change their statutory and programmatic processes to accommodate the fee program.
Various alternatives also exist for funding laboratory PE studies through external sources.
These options should be fully explored and considered. For example, the National Institute of
Standards and Technology (NIST) has the capability to implement a self-supporting program for
PE studies. OPPTS has established a Memorandum of Understanding with NIST to operate an
accreditation program for laboratories that conduct asbestos analysis. OPPTS provides limited
funding to NIST for research needed to develop and test analytical methods and new PE
materials. All other program funds are generated through fees paid by applicant laboratories.
NIST also operates a PE testing program for the National Oceanic and Atmospheric
Administration (NOAA) to support its Status and Trends and Mussel Watch marine monitoring
programs.
The Agency's PE study program could also be operated independently in the context of
-------
27
a national laboratory accreditation program. In October of 1992, the EMMC recommended to
the Deputy Administrator that steps be taken to design a national program for accrediting
environmental testing laboratories. Since then, an EPA/State Operating Group has been
convened to plan and establish a national conference on environmental laboratory accreditation.
The program model currently under consideration would incorporate all existing State
environmental laboratory accreditation programs and includes provisions for ensuring uniform
national performance evaluation testing. The model assumes that all accreditation costs would
be borne by the participating laboratories.
The Agency has not yet decided whether the development and distribution of PE samples
to support national laboratory accreditation would be operated by EPA's Office of Research and
Development or by a private sector organization. Implementation of a national program will
involve development of consensus uniform national standards for laboratory accreditation,
coordination of over 30 existing State environmental laboratory accreditation programs, and
establishment of new programs hi states that do not presently have them. Consequently, EMMC
estimates that a national program could not be operational for at least 5 years. It is critical that
OW staff continue to play a major role in the development of the national laboratory
accreditation program to ensure that the program is adequate to meet OW programmatic
laboratory oversight needs.
Agency-wide Approaches to Funding PE Studies
As a result of this pilot project, the Assistant Administrator for Water will recommend
to ORD and the Administrator that an Agency-wide initiative on quality assurance and laboratory
performance evaluation studies be undertaken. The initiative should identify budgetary and
statutory alternatives for establishing a comprehensive quality assurance program within EPA,
including a strong laboratory oversight component based on performance evaluation studies.
Other EPA program offices should conduct studies similar to OW's to document needs for PE
studies. The Agency-wide initiative on quality assurance should include a component to support
laboratory PE studies, as part of the FY 1996 budget development process. The initiative should
-------
28
promote establishment of a permanent change hi the Agency's budget development and
formulation processes to ensure funding to support quality assurance, including laboratory PE
studies. The QA budget appropriation could take the form of a centralized or decentralized
Agency-wide tap on the budget, a request for an increase in the Agency's base budget, or a
redirection of programmatic funds to provide sufficient funds for all of EPA's laboratory PE
study requirements. Part of this effort would include determining whether AC&C or R&D
dollars should be used to fund laboratory PE studies and other quality assurance activities and
whether ORD or the program offices should have principal responsibility for providing the
necessary staff support.
-------
APPENDIX A
THE WATER PROGRAM
LABORATORY PERFORMANCE EVALUATION STUDY
REGIONAL USE SURVEY
-------
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, DC. 20460
JL.,'-. - ]
OFFICE OF
WATER
MEMORANDUM
SUBJECT:
FROM:
TO:
Analysis of Performance Evaluation Studies Used by the
Water Program
Martha G. Prothro
Deputy Assistant Administrator
Environmental Services Division Directors
Water Management Division Directors
Office of Water Office Directors
The purpose of this memorandum is to request your
assistance, over the next two months, in an Office of Water (OW)
effort to assess laboratory performance evaluation (PE) studies.
PE studies are used to determine and ensure the technical
competency of laboratories analyzing monitoring samples. OW will
be documenting how PE studies are used in the water programs, the
benefits being derived, and current and future PE study needs.
The information we collect will be the basis for determining the
best way to design and administer water-related PE studies in the
future so that OW program needs are met in a cost-effective
manner .
BACKGROUND
In FY 1992, the cost of PE studies supporting water programs
will be between $2-3 million. The Office of Research and
Development (ORD) currently funds between 70-80% of the water
program PE study costs through the Research Committee process.
In the summer of 1991, ORD requested that the Office of Water
assume full responsibility for funding PE studies. Other Agency
program offices are also being asked to fund their PE studies.
Resources saved will be shifted to research.
In the winter of 1991, several OW Division Directors
requested that the Immediate Office take the lead in responding
to ORD's request on funding. They emphasized that lack of
funding for PE studies would have serious impacts throughout the
water programs. PE studies are critical for certifying EPA
Regional, State, and commercial drinking water laboratories;
certifying State wastewater and water quality laboratory
programs; developing and validating new analytical
17: Pnmec o* flecvc-ea
-------
-2-
raethods; providing data .for the drinking water regulatory_
development process; assuring that data received in the Discharge
Monitoring Reports is of sufficient quality to use in oversight
and enforcement actions; and ensuring the quality of water
monitoring data used in national information systems and national
reports.
The Division Directors suggested that the appropriate
approach would be to evaluate all existing and future needs for
PE studies within the water programs. The information gathered
would be the basis for design, administrative, and funding
decisions. ORD senior management, as well as the Environmental
Monitoring Management Council (EMMC) support this approach. It
is anticipated that the OW effort will be a pilot for other
Agency program offices.
SCOPE OP THE PROJECT
Over the last 2-3 months, OW staff has worked with both
Headquarters and Regional staff to design three tools to obtain
the information necessary for this study:
• A questionnaire on the design, administration, and
funding of PE studies for the Environmental
Monitoring Systems Laboratories (EMSL) in
Cincinnati and Las Vegas;
• A PE Study Use Questionnaire for Headquarters and
Regional staff to determine how States,
Environmental Services Divisions (ESD),
Headquarters and Regional Water Program Offices,
and EPA Regional and ORD Laboratories use water-
related PE studies in their programs; and
• A "guide" to use during interviews with selected
staff in Office of Water, the Water Management
Divisions, the ESDs, Regional Laboratories, and
the ORD Laboratories to determine if there are
program needs not met by current PE studies and/or
new monitoring requirements that will require
additional PE studies in the future.
All ten Regional Quality Assurance Officers (RQAOs) have
participated in designing and reviewing draft versions of the PE
Study Use Questionnaire and have approved the final version
(Attachment 1). Discharge Monitoring Report (DMR) Quality
Assurance Coordinators and Regional ESD laboratory staff also
assisted. Selected Regional and Headquarters staff were involved
'in developing the other two tools. The project will build upon
previous studies and will be coordinated with related activities
-------
-3-
such as the lab certification and the methods integration efforts
sponsored by the Environmental Monitoring Management Council.
Attachment 2 provides more detailed information on the scope of
the overall project.
NEXT STEPS
The RQAOs have 'been asked to coordinate with the Regional
Water Management Divisions and Laboratory staff to collect and
compile responses to the PE Study Use Questionnaire. All
indicated a willingness to participate and believe the effort
will take between 4-6 weeks.
Our discussions with the RQAOs indicate that the ESDs
maintain most of the information needed. Some telephone calls to
State program representatives may also be required. The RQAOs
will consult with representatives of the various Regional Water
Management Divisions and the Laboratory Branch Chiefs in
responding to the questionnaires. We would like to have the
completed questionnaires returned to Wendy Blake-Coleman, of my
staff, by July 15, 1992.
During the same time frame, OW will contact Headquarters and
Regional staff to arrange interviews. We will also be working
with EMSL Cincinnati and Las Vegas to document PE study design,
administration and funding information. We will analyze the
results in late July, report the findings, and make preliminary
recommendations to senior Water Managers and the EMMC in August.
This study is a first step in resolving long-term and
Agency-wide issues associated with the most efficient and
effective way to conduct PE studies. Please contact Wendy Blake-
Coleman at 202-260-5680 for further information.
Attachments
cc: LaJuana Wilcher
Alan Fox
Robert Pavlik
Cynthia Puskar
Kathi Payne
Gary McKee
Tom Clark
J. Gareth Pearson
Regional Quality Assurance Officers
-------
-------
.ATTACHMENT 2
REVIEW OF WATER PROGRAM PERFORMANCE EVALUATION STUDIES
BACKGROUND
EPA's principal tool for evaluating laboratory performance is the performance
evaluation (PE) study program presently operated by the Environmental Monitoring Systems
Laboratories (EMSLs) in Cincinnati and Las Vegas. PE studies involve sending manufactured
samples to laboratories identified by the regions and states once, twice, or four times
annually. Participating laboratories are asked to analyze the samples in accordance with
certain specified analytical methods and to identify and quantify the pollutants present in the
samples. The laboratories' analytical results are submitted to EPA and are evaluated
statistically. Laboratories that do not perform at or above the statistical average may be
required to participate in additional PE studies and/or to report on corrective actions taken.
The Office of Water is undertaking a study to evaluate the current water PE studies to
determine the extent to which they are meeting the current needs of OW quality assurance
programs, to characterize changing needs for PE studies in water programs, and to develop
options for administering the water PE studies in the future. The study involves collecting
information on current program administration from the EMSLs, information on current uses from
the Regional Environmental Services and Water Management Divisions, and information on
anticipated future needs from staff at Headquarters and in the Regions.
STUDY PLAN
OW will work with the EMSLs to develop an administrative profile of the current studies.
This component will characterize the extent of the national program and provide information
for developing future program options. It will also provide managers at Headquarters and in
the regions with an understanding of how the current program is operated and managed.
In conjunction with the Regional Quality Assurance. Officers (RQAOs). the Regional
Laboratory Branch Chiefs, and the Regional DMRQA Coordinators, OW has developed a
questionnaire for use in developing and inventory of current uses of water PE studies. The
inventory will be used to:
Evaluate the present uses of PE study data by Headquarters,
regional, and state/local programs;
Identify present needs for PE information that are not being
serviced by current studies;
Identify additional uses of PE study data that could benefit
current programs; and
Identify new areas where additional or different types of PE
studies will be needed.
The analysis of future requirements for PE studies will involve interviews with
Headquarters, regional program and regional ESD staff. The focus of these interviews will be
anticipated program directions and changes in monitoring requirements. New program
requirements in areas such as biological monitoring and contaminated sediments management
-------
will result in needs for additional PE studies or changes to existing studies. This information
will be used to develop strategic options for administering PE studies in the future such that
they adequately support the quality assurance aspects' of new monitoring requirements, from
logistical and budgetary standpoints. OW will begin conducting interviews for the requirements
analysis.at Headquarters in May of 1992 and will contact each Region to ask their participation
in this aspect of the study.
The inventory of present uses will be used in conjunction with the analysis of future
requirements and an evaluation of potential funding options to develop recommendations for
future program configurations. OW will also determine whether there are alternative sources,
external to EPA, for meeting the PE study needs of water programs.
-------
1.
WATER PROGRAM PERFORMANCE EVALUATION STUDIES:
CURRENT USES
NUMBERS OF LABORATORIES PARTICIPATING IN PE STUDIES BY TYPE-
Record the number of laboratories of the following types that participate in each of the Water
Program Performance Evaluation (PE) studies indicated. In cases where one laboratory falls
into more than one category, include it in all applicable categories (this will appear to result
in "double counting"). For example, a private/permittee laboratory that also analyzes samples
for commercial clients should be included in both the Private/Permittee and Private/Commercial
categories for all applicable studies.
For the DMRQA study, be sure to verify and report the number of laboratories (rather than the
number of permittees) participating, based on information from the DMRQA report forms. Note
that the number of laboratories for DMRQA may be smaller than the number of permittees.
Information for 'other studies may be obtained from regional records, reports from EMSL-
Cincinnati, or both.
Provide data for the following studies, as indicated on the table: WP 026 and WP 027, WS 028 and
WS 029, DMRQA 11, and any other studies used. Complete one table for the Regional Office and
one for each state in the Region. The Office of Water will obtain records concerning
radiological performance evaluation studies from EMSL-Las Vegas. Regions should provide only
additional or unique regional or state information pertaining to those studies. In the space
below, indicate the information sources used and identify the data provided by each.
DATA SOURCE
INFORMATION PROVIDED
-------
g
Cd
g
g
(£ 5
o o
s! cu
5 S
*
c<
Cd
O
£
W
W
>-<
FEDERAL:
NON-EPA
1
Q <
Cd CU
to cd
a.
0
3
S
Cd
§
Cd
£S
ss
si
a. cu
r . O*?
< w
OS O
cu o
£
03
a
1
CO
CO
o
*
*
a
OS
1-1
cy
Q
Cd
O
CO
CO
cu
o
cu
cu
cu
CO
cu
1o
CO
Qi
cu
CO
CO
o
CO
•o
(U
•p
o
f-t
a
•a
co
cu
•a
iJ
CO
O
O
0)
D*
•r-1
3
CO
I
c
cd
0)
J-c
cu
03
cu
Cd
cu
CD
CO
CO
i
Cd
•O
CU
•a
0
t-,
a<
cu
JO
cu
1 I
O CO
-------
CO
#
Si
Cd
X
E-
O
g
00
«
ta
•z,
J <
W
^g
H E-j
S W
Cu CU
[VATE/
HMERCIAL
a: o
Cu CO
cu
?J
w
CQ
• a
e.
CO
CO
o
Cu
o
*
^
cu
o_
CO
CJ
o
3
e
cu
§
0
CO
s
g
oo
CO
o
s
s_
C7>
CJ
o
31
*
a
oi
rt
a
•o
L<
(0
CO
CO
cu
o
cu
CO
cu
CO
!
a>
eo
(0
o
CO
(0
•a
CD
1°
1
a*
"O
CO
c
CO
•a
o
cu
co
3
cr
LI
o
la
o
•a
O (0
-------
2.
SUMMARY OF USES BY PROGRAM
On the table provided, for each PE study, indicate the programs that use the study, the number
of laboratories participating for each program that uses the study results, and the type of use
or uses each program makes of the data. Use the program and use codes provided. If other
programs or uses are needed, please identify and define on a separate sheet. Include
Headquarters, regional, and state/local programs. State/local programs and uses should be
verified with state program representatives.
For example, in a case where WS Chemistry has 50 laboratories participating for drinking water
certification, 10 laboratories participating for state laboratory certification for the
parameters and methods tested, 20 laboratories participating to demonstrate their performance
on the method for purposes of RCRA program work, and 5 laboratories participating to
demonstrate proficiency for ambient ground water monitoring for a state program, report the
following:
WS Chemistry dw 50 labs Ic
cp 10 labs Ic
rp 20 labs mp
sa 5 labs Ic
Complete one table for the Regional Office and one for each state in the Region.
Program Abbreviations
dw = federal or delegated state drinking water program
np - federal or delegated state wastewater permitting program
en = federal or state enforcement program (identify statute and explain use)
cp - state laboratory certification/accreditation program (explain scope and purpose
on a separate sheet)
sa ~ state ambient monitoring program
rp = federal or delegated state RCRA permitting program
re * federal or delegated state RCRA compliance program
fp = regional Superfund program
sp = state program for clean-up of hazardous materials sites
op = other programs (identify on a separate sheet)
no = not program-specific
Use Abbreviations
Ic » laboratory certification/accreditation/acceptability determination
mp » method performance
md = method development
es = enforcement support
mv = validation for alternative procedure
ip = improve laboratory performance (lab's own internal use)
ep - improve laboratory performance at EPA/state's request
rf = reference materials/standards
la - laboratory audit/audit -targeting
It = laboratory training
o « other (identify on separate sheet)
cu = contact unaware of use
-------
STUDY
WP 026
WP 027
WS 028 (Chem)
WS 029 (Chem)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
PROGRAM
SUMMARY OF USES BY PROGRAM
REGIONAL OFFICE NO.:
NO. LABS
* Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
WP 027
WS 028 (Chem)
WS 029 (Chera)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
SUMMARY OF USES BY PROGRAM
STATE:
STUDY
WP 026
PROGRAM
NO. LABS
* Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
3. FREQUENCY OF USE BY PROGRAM
For each program that uses PE studies, indicate the number of times per year participants are
required to analyze samples for each study. Also provide the reason for requiring analysis
more than once per year (where appropriate). For example, if laboratories are required to
analyze samples for the odd-number study only when unacceptable results were achieved for the
previous even-numbered study, the reason for analyzing more than once per year would be to
ensure that corrective action has been taken following unacceptable performance. Odd-
numbered studies may also be required, for example, when a new laboratory participates for the
first time. In cases where laboratories analyze samples for odd-numbered studies although not
required by state or Federal programs, indicate "voluntary" or "for lab's own use".
Include information for Headquarters, regional, and state/local programs. Information
concerning state/local programs should be verified with state program representatives, as
appropriate. Complete one table for the Regional Office and one for each state.
-------
WP 027
WS 028 (Chem)
WS 029 (Chem)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
FREQUENCY OF USE BY PROGRAM
REGIONAL OFFICE NO.:
STUDY
WP 026
PROGRAM
NO. TIMES/YR. PURPOSE OF MORE THAN 1/YR.
* Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
WP 027
WS 028 (Chem)
WS 029 (Chem)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
FREQUENCY OF USE BY PROGRAM
STATE:
STUDY
WP 026
PROGRAM
NO. T3MES/YR. PURPOSE OF MORE THAN 1/YR.
* Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
4. PROGRAM DECISIONS SUPPORTED BY PE STUDIES
For each program that uses each study, identify the types of decisions are supported by the
study results. Use the Program Abbreviations and the Application Abbreviations provided
below. If necessary, identify and define any other Application Abbreviations needed. Include
information on Headquarters, regional, and state/local programs. Address all uses by all
regional and state programs, including those not related to water. For example, include state
programs for certifying coal mining laboratories and methods validation applications by state
RCRA programs. Information concerning state/local programs should be verified with state
program representatives, as appropriate. Complete one table for the Regional Office and one
for each state in the Region.
Proeram Abbreviations
dw - federal or delegated state drinking water program
np - federal or delegated state wastewater permitting program
en = federal or state enforcement program (identify statute and explain use)
cp = state laboratory certification/accreditation program (explain scope and purpose
on a separate sheet)
sa - state ambient monitoring program
rp « federal or delegated state RCRA permitting program
re = federal or delegated state RCRA compliance program
fp » regional Superfund program
sp - state program for clean-up of hazardous materials sites
op * other programs (identify on a separate sheet)
no = not program-specific
Application Abbreviations
a = requiring corrective action
t - targeting audits
p = measuring compliance with laboratory performance requirements
d ~ determining validity/acceptability of data
c - making certification/accreditation determinations
-------
PROGRAM DECISIONS SUPPORTED BY PE STUDIES
REGIONAL OFFICE NO.:
STUDY
WP 026
PROGRAM
APPLICATIONS
WP 027
WS 028 (Chem)
WS 029 (Chem)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
PROGRAM DECISIONS SUPPORTED BY PE STUDIES
STATE:
STUDY
WP 026
PROGRAM
APPLICATIONS
WP 027
WS 028 (Chem)
WS 029 (Chem)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
* Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
o.
TECHNICAL ADEQUACY OF PE STUDIES
On the tables provided, for each program that uses each study, characterize whether the
current study is adequate to meet program quality assurance goals (i.e., sufficient quantity,
frequency, analyte coverage, concentrations, matrices). Use the Program-Abbreviations and the
Technical Adequacy Codes provided below, or define others, as needed. Any inadequacy noted
should include an accompanying narrative explanation to further define the inadequacy noted.
For example, if inadequate analyte coverage is noted for the RCRA program participants in the
WP study due to the absence of furans in the PE samples, the entry should be:
WP
rp ic (PE samples do not contain furans)
The information provided should reflect all relevant perspectives, including state program and
laboratory representatives and Regional program and laboratory representatives. Complete one
table for the Regional Office and one for each state in the Region.
Program Abbreviations
dw
np
en
cp
sa
rp
re
fp
sp
op
no '
federal or delegated state drinking water program
federal or delegated state wastewater permitting program
federal or state enforcement program (identify statute and explain use)
state laboratory certification/accreditation program (explain scope and purpose
on a separate sheet)
state ambient monitoring program
federal or delegated state RCRA permitting program
federal or delegated state RCRA compliance program
regional Superfund program
state program for clean-up of hazardous materials sites
other programs (identify on a separate sheet)
not program-specific
Technical Adecruacy Codes
ad = adequate study design
id = inadequate study design
aq = adequate number of performance evaluation samples
iq = inadequate number of performance evaluation samples
ac = adequate analyte coverage
ic = inadequate analyte coverage
ak = adequate analyte concentrations
ik = inadequate analyte concentrations
am = adequate matrices tested
im = inadequate matrices tested (please identify recommended additional matrices and
analytes on a separate sheet)
In the space below or on a separate sheet, discuss any administrative inadequacies
opportunities for improving the administration or efficiency of the studies:
or
-------
WP 027
WS 028 (Chem)
WS 029 (Chem)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
TECHNICAL ADEQUACY OF PE STUDIES
REGIONAL OFFICE NO.:
STUDY
WP 026
PROGRAM
ADEQUACY OF STUDY TO MEET NEED
* Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
WP 027
WS 028 (Chem)
WS 029 (Chem)
WS 028 (M)
WS 029 (M)
RAD*
DMRQA 11
Other (identify)
TECHNICAL ADEQUACY OF PE STUDIES
STATE:
STUDY
WP 026
PROGRAM
ADEQUACY OF STUDY TO MEET NEED
* Information to be provided by EMSL-Las Vegas for PE A and PE B. Provide any additional
or unique regional or state data, if available.
-------
6.
STATE STATUTORY AND REGULATORY REQUIREMENTS FOR PE STUDIES
For each state program, are the studies used required by state regulation or statute (Y or N)
and what is the legal citation? Please use the Program Abbreviations.
Program Abbreviations
dw » delegated state drinking water program
np = delegated state wastewater permitting program
en » state enforcement program (identify statute and explain use)
cp - state laboratory certification/accreditation program (explain scope and purpose
on a separate sheet)
sa = state ambient monitoring program
rp - delegated state RCRA permitting program
re * delegated.state RCRA compliance program
fp = regional Superfund program
sp = state program for clean-up of hazardous materials sites
op = other programs (identify on a separate sheet)
no - not program-specific
STATE
PROGRAM
REQUIRED BY:
STATUTE? REG.?
CITATION
WS Chem.
WS M.
RAD
DMRQA
Other
(identify)
_
-------
7. Describe the process by which the Region manages, oversees, administers, or organizes
participation in the PE studies. Indicate points of contact with the laboratories, the
states, within the Region, and with EMSL-Cincinnati. Do the states in the region make
requests directly from EMSL-Cincinnati, or are all requests conveyed through a -single
regional point of contact with Cincinnati? Also, indicate whether laboratories can be
included at their own request and whether such requests are honored by the Region or
must be forwarded to the Region by the state contact. Provide a brief description of
records maintained by the Region, including tracking systems or other documentation
showing which laboratories receive samples and which return results of what type.
8. How and by whom are decisions made concerning which laboratories participate and how
often? Describe Regional and state policies for including laboratories in the studies.
If ho policies exist (i.e., all requestors are allowed to participate, regardless of
need/application), please note this. -Do you receive "extra" or surplus PE samples and,
if so, how are they used? .
-------
9. ALTERNATIVE APPROACHES TO EVALUATING LABORATORY PERFORMANCE/DATA QUALITY
Based on current Regional quality assurance needs, are PE studies the best way to acquire the
information needed or are there other approaches that could be used? If PE studies are not
the best alternative, explain why and state what you would recommend be done differently.
-------
10. QUALITY/RELIABILITY OF INFORMATION
On the table provided below, estimate the relative reliability of the information provided for
questions 1-8 above, based on the sources consulted and your judgement. Use the following
scale:
3 = Very reliable information, based on complete, comprehensive records
2 = Somewhat reliable information, based on complete but not comprehensive
records
1 = Somewhat unreliable information, based on incomplete records
0 = Information based on judgements of regional/state representative
QUALITY/RELIABILITY OF INFORMATION
REGIONAL OFFICE NO.:
QUESTIONNSTUDY
Ql
Q2
Q3
Q4
Q5
Q7
Q8
WP
WS CHEM
WS M
RAD
DMRQA
OTHER
-------
QUALITY/RELIABILITY OF INFORMATION
STATE:
QUESTIONXSTUDY
01
Q2
Q3
04
Q5
•Q6
Q8
WP
WS CHEM
WS M
RAD
DMRQA
OTHER
QUALITY/RELIABILITY OF INFORMATION
STATE:
QUESTIONXSTUDY
Ql
Q2
Q3
Q4
Q5
Q6'
Q8
WP
WS CHEM
WS M
RAD
DMRQA
OTHER
-------
QUALITY/RELIABILnY OF INFORMATION
STATE:
QUESTIONXSTUDY
Ql
Q2
Q3
Q4
Q5
Q6
Q8
WP
WS CHEM
WS M
RAD
DMRQA
OTHER
QUALITY/RELIABILITY OF INFORMATION
STATE:
QUESTIQNXSTUDY
Ql
Q2
Q3
Q4
Q5
Q6
Q8
WP
WS CHEM
WS M
RAD
DMRQA
OTHER
-------
-------
APPENDIX B
i
PRELIMINARY RESULTS FROM THE WATER PROGRAM
LABORATORY PERFORMANCE EVALUATION STUDY
REGIONAL USE SURVEY
-------
-------
APPENDIX B
PRELIMINARY RESULTS FROM THE WATER PROGRAM
LABORATORY PERFORMANCE EVALUATION STUDY
REGIONAL USE SURVEY
This Appendix provides graphical and tabular summaries of the results from the Water
Program Laboratory Performance Evaluation Study Regional Use Survey. The survey form (see
Appendix A) was completed by each of the ten Regional Quality Assurance Managers (RQAMs)
in EPA's Regional Offices using information available in the Regional Offices and based on their
understanding of State programs in their Regions.
Responses to the questions posed by the survey were varied. Due to differences hi
recordkeeping and information management practices in the Regions, some were unable to
provide definitive quantitative responses to certain of the questions. In addition, one Region was
not able to respond to most of the questions posed. The summary graphics and tables displayed
in this Appendix include all of the information that was provided. Each indicates the level of
response received. Cases where significantly low response rates were experienced are noted.
As a result of this variability, the data provided in this Appendix should be interpreted to be an
indication in trends related to the use of the water PE studies nationwide. Specific information
regarding individual State or Regional programs should be verified with the State or Region
before being used hi any other context.
-------
-------
CO
CD
(/) UJ
iy < t
LJ II i "frt
§.£
.
1
& 1«
O q>£
CO
_co _co
Q §
i
UL ^ ^
Oco •»-•
.^•co
QC
HI
CO
co
Q. CD
13 LLJ
CD
s: CD
Q. U.
00
CM1?
•.
QCM
si
-------
r
CO
o
^
Q
<
CC
to
CD
'•a
•»-•
CO
111
DL
CD
CD
CM
O>
OJ
.9 ^
.0. CD
Oca .S-
^
CDS
O"> m
CD .
B^m •*•* ^—
Hco o
^> I*
CC "§.§
CO JOB
'o
CD
CD
CO
•S0!
O.P-
CD
DC
LU
O'E
|=L
"12
3 III
C C
CD o
2i CD
Q. U.
-------
CO
CD
UJ
V)
LLJ
Q_
£S
LLJ .CO
I
g |ts
O
cc
Q. CD
"
E-o
> CD
II
w ]o
ll
.Q- co
•
J2 "«" "8
GQ *p z
O
CC
UJ
CQ
co ^
•^1
Q. CD
B-l
CD =
=3 LLJ
JP. O
col
-------
LL
DC
UJ
DQ
CO
CD
CO
HI
Q-
0)
ill
s
H
O 1
a CD
•a
•S a>
1 ^
5 :tl
CD
3.E
E-a
gts
•2-5
co .52
- "C
is.
I?
CD £
.Q. CO
^ CD
±2 fc-
CO O
O
z
I
o
DC
£
(0
UJ
p il
5 If
sll
CD
2 CD
a. u_
CD
-------
CO
UJ
Q
i
UJ
CO
ROG
I
O)
2
Q.
1
2"g>
1 f
"j "g
< ""S
^* co
Q.
CO
CD
8 I
11
CC "b-
£ I
CC c
uj i
Q. =
CD
S £
UJ >;
I -o
O -5
CO
CD
CD
o ii
£s co -Q
CO £ CD
Z) co £
CO «B
1 ?!
tt Q.CO
.
ll
-^ CO D>
Q. go
uj S5-
f|
. "^ CD
fe £ =
CD"®
CC 1 CD
UJ S Q.
CO r- ^
i- -a
£§
1
S
O)
i
rrt
1
P
2
»
1
Q-
*«
-° •§
JIiiHl
I
Itll i^li
z 5i3
" u
fera
£o>
6s
a.
-------
to -£
UJ ^
Q 2
D ^
H D>
00 I
__ CO
Z B-
O i|
H"™" co
DL
ID CD
-I "Q
UJ
UJ
O
CM
CO
CD
I
C
DC j?
O ~
II >»
•"2 c5 •
CC -3 CO.
UJ «|j
O- Jo
op «5
ar
CA ^
ii
«H
—1
e
,
Laboratory
Audits
,
'
§
f
*t
§
improving Lab -
performance
i-
i
i
1
Jltuoqn-
1
i
Ijojoq
ill
ni
•
mini
s .
iiiiiiiiii
•
i i
Hill
Hi
i
iunN
ilil
imini
7
••
muni1
s
lllll
•d
i t
iiiiiiliili
iiiiiiiiiiiii
s
,1!
^ i 1
I ||l
1 fit f I
< IP iLJl! § o
i
o.
o
&
CL
U.
g
i
g
S
3
CL
i
a
(O
ffl
-------
(0
LU
"§
0
i *
H Q.
^ •-
< t
> CO
UJ .g
s 1
re
£ 1
ffi I
CL CD
j=
CO *t
CM >>
O JjS
QC co cfl
1^ ^ CD
jj» 3 *-
S ^> co
5* "go
0^2
O) £
p o>
CD -
~
O *8
£ II
UJ "2 S
f— J= Q
5 ^00.
P LU^1
(/) 0. 0
>- x:
LL, £> **
OcO ^
^r _ i^
g €£"
CQ m £
SJ D)
Jr o
—* O C
5 LU Q.
Q.O
EC—
Og.
ocE
£
" r
I1*'
2 2,-g
II, II
2
f
«
a,
O)
<££
O J3 £
g55
— S e
ill
5 ! J
n
Reference
Lab Audit
LabTralnIn
ti
3
i
s
3
&
CQ
a.
cr
-------
CO
CD
CM o
q> •*-
o> D)
2J to
= Q.
*o
0) '€
5 a
D to
3 -g
CO |i
CO
o
.CO
"S
Q
••" C
to I
CO £
•
i*
73
^•S
^, CO
CC
O
O
CC
a,
z
o
1
LL
g
111
° 1 S
iiJ I S
H ^ =
to f 8
^A r\
LL
O
Q.
CD
111
Q.
i_
3
^J
TJ
> ^i
S o£
D co co
CO * o>
o S
LJL Q.
CD
-------
CO
UJ
to
uj
UJ
o
cc
s
cc
UJ
Q.
o
oc
O
S
i
DC
O
u.
CO
t
co
®
O 05
S
a.
a.
a
UJ
O
Q.
uj -a
ts
ll
- 0)
•S 2
CO Q-
O CO
LL. CD
0)
0'C =
+3 'V.TJ
€^§
UJ
O)
CD
3s
ti
ll
is.
-------
m
z s
rfS-i CL
o 5
a
8
> 1
LU o
S*
11
I i
cc i
^^ CD
£ e
DC
£§!
S; € co
O ~ E
Z 5 E
<0 |i
§ II"
*^ O CTJ
rf Q)
o 5 s
o ^s ^
£11
S s"
^ to 5=
5zr LU "o
CO Q. c
oil
a— D>
-C O
LU g S.
CD o .c
o!
II
ii
cuS
l!
1U
ft
O)
: £
&
•8
O)
nil
js O «2 £
2 §*£ O
2 s "5 "5
CO iX O O
llll
i
£§>
si
i
I
°z*
<&&
133
S5 Q. O. -
£££
ll
33
» ii
35
o
T~
CQ
-------
I
CD
SI o
o> •*-
O) O>
Z '"§
— 9-
"
CO
LLj
Q
i
(/>
UJ
QC
CO
CL
CO
CD
I
I
CD
^3 3 CO
Or -S tl
•"* tn CO
o £"§
o £ J2
£ I?
§ II
g II
< co E
Oi=. CO
_ ^* *~
til5
CC
UJ
o
CD
HI =0
CO
CD
X -£ =
5 ^°
S Sg
CO Q.
LL uj >•
O Q. CD
^ flj "•"•
1 =
> CO
"I I
CO ® »
-------
CO
Ul
H ra
CO .§
ZCQ
Q.
O S
I— CO
5 a
3 s
< 1
m .§
•MM Qj
Ul ^
o 2
cc
o
li.
ct
Ul
CL
(O
CM
or
CD
^ 5fi
if
to c
£= co
IS1
2 II
CO S"CD
251
b IS'B
-------
V)
UJ
Q o
WHk CO
^j CD
fe £
ZO)
_c
O to
< --F
UJ o
Z S
< ?
2 CD
oc f
si
DC 2
UJ *=
O CD
DC £
C5
*= CO
II
CO C
o>2
is
-------
CO £
UJ ra
5 -I
S S.
tT '5
CO -|
5? CO
1 »
P .2
3 "i
in *-
ui 2
0|
1 !.
0 S-|
U. -0*
DC =J .c
LLI w *"
Q_ CD "S
MM Ja; *"'
< f 8
Cf CO "rrt
DC ^ E
oil
MMI • ty
O 2 Q.
H D) -.
co US
ZQJ
Q)
0 5 |
O .a *o
B- ^ §:
f^ .^» pT>
-. *
oil
Q. > ™ •
LU J= o
Ko *-
co Q-
u.3
s:
0)
II
>— 'Q.
IS
U*
(
t
'
.
1
1
1 i 1
•Q
°;l
H—
p
(
1
(
'
8 S -
'fll
>0
O '*
*!•£ i^
i|
P-
•
i
,
i i ;
ErS
StS
2 .0
o !£
•a«
q 0)
-IQ
•
1
.•••i
topowioqen
q
Lilffllll
i !
s . •
i i
.mmmmmi
k i
jojoqumn
j
j
H
iiiiiini
I i
•
-
i
-
-
r.
L
• <
'
C
•
-
'
i
•
D
m
.«
s
z
5
>
o.
a.
z
0
>
&
:
&
>
&
s
*
s
0
M
3
|||
W ^£
i
6
IU1
s
°
o
CO
-------
UJ
QC
O
•3
CO
UJ
•••
Q
3
O
ALUATI
g
UJ
o
•
S WHY WATER LABORATORIES USE PERR
THAN ONCE PER
2"
O
CO
UJ
QC
Q
"CO 0)
it
"5 ^
^"fe"
§1
CO ^*
£ 0
C Q.
S.8
!" c
CO CZ
O. CO
^ £
S CD
- i
CD C
°-to
8'i.
f= ^»
rogram that uses PE studies, indicate the number of tin
r each study. Also provide the reason for requiring ana
0.0
CD "o.
£§
•
CO
CD
• ^
0)
0)
1
CD
0)
9=
O
CD
CO
3
to
O
DC
Q
3
•
CO
CD
CO
O
|
to
^^v
_Q.
4^
^^^
CO
Q.
CO
J5
CO
CO
J)
0)
to
o
•
l^n
H
- First test is manciaior
C
_O
O
CO
B>
o
£
o
o
0)
CO
•2 o.
HM 3
8 *
HH* ^^
T3 —
C o
o "8-
0 -o
$ i
1
£
5
to
Q.
^^
• •
Most labs voluntarily parl
•
to
o
•o
c
o
o
o
CO
.c
JC
^^B
c
performance improvemei
•o
0)
"5
0)
o
c
CO
• ^•^H
even if corrective action i
0)
£
0)
75
CO
CO
•"••
- •
Some States indicate thai
•
CD
£
_c
CD
4-»
CO
Q.
"o
"•E
Q.
O
^j
"5
o-
0)
0)
tol
® co
i?
O C
10
T-
m
-------
CO
LU
a
^^
i— «
CO
HI
D.
GQ
O
HI
£
o
a,
Q.
=>
•
\fj
O
CO
o
HI
Q
§
DC
0
O
CC
CL
CO
LLJ
Q
CO
ULI
X
o
CO
•
JaquTnN
II
0)O)
o;
O)
OE
COO)
S.E;
o
Q
a.
o
o
o.
o
o
ft.
O
o
a.
E £
2 2
MS
5fj,H
J5i5^<
E O Q g O)
H
s
CLQiS
I II tl
< i- a a o
a.a>
*-Q.
O
(0
po
CO
o
o
a.
o
o
a
i-
(O
GQ
u
a
Q.
sasuodsau to jaqufnN
3 sasuodsay jo jaquInN
-------
CO
5
=>
CO
LU
Q.
ffl
Q
UJ M
tffi
co
CO
o
UJ
Q
DC
O
O
QC
a.
3
CO
£
CD
O
Q.
Q.
CO
CO
CO
g
"co
'8
T3
°-w §
Q. w f
3O £
COS £•
o
O
2
(0
CO
I
D)
g
a.
1
CD
s °
5 §>
: in oc
•e
o
»_ o o
B BE (C
g •&£•
&s»f|
" •"** o g.
II
•I S^ § £§
I S H"e S r
3 O ffi O O O
Z O 1= O. Q O
II II II II II It
n < H- a. o o
OQ
cn
sasuo
-------
C/>
UJ
ill
o.
m
Q
LLJ
DCfT)
O1"
QUO
2i
(O
o
ui
o
<
DC
o
o
DC
a.
CO
CD
•i
B
a
P
co
.
'cj
•§
o
TJ
•i
.C
CO
CD
CO
=3
I
D>
CD
£
og
tare
II-
Q.
U
a
i:
.11
c c c
H
I
i u i
< H a. o o
U
O
C
E
E
00
I—
CO
I O
Q
sasuodsoy Jo JaquinN
-------
(0
LLj
Q
:D
HI
0.
m
o
HI
o§
CL
2
Q.
0
£
sasuodsej/jo jaqiunN
O)
II
State
§,E£
«— Q. °r
O. n « T=
'
I'll
•?r' ^ ^
0|
li
2
cno
(0
£
0)
5^1
:-o •»
» o •
fill
I
n n u n
HQ.QU
u
Q
a.
O)
r~
CQ
u
Q
a
i-
u
a
a
sasuodsau *o jaqmnw
-------
UJ
Q
H
CO
UJ
Q.
CD
O
UJ
CC
OCO
CLUJ
20
C0CC
UjD
Q
DC
O
O
tr
a.
£
=3
CD
TJ
CD
CO
£
CO
•s
'to
5
1
f
i
i
0
CO
CD
CO
CO
1
D)
g
Q.
1
8£.
§.§ » 5
W *3 ™
i 2 is
(D w *»
!*!
Ill
££fIlll
> B) (0 '£ IS
"IS ill
a o v, »
n H ii
Q. Q O
sesuodsay *o JaquinN
-------
1
cr
o
o
-------
CO
111
uu
Q.
o
z
o
CT
CO
O)
O
o
CD
CM
C\l
ffl
-------
UJ
Q
D
CO
UJ
Q.
111
O
CO
O"
$
E^
o co
S-g
CO -J±
"i £
co .2
« 8
f{S
CD o
3 CD"
O O)
ffi CO
® 8
•*—» m
CD •*-*
5 "f5
& s
If
O
z
O
UJ
--
CD C
CO .2
CD O
si
5= CD
E ^
2 D)
Q. CD
CO
CD
CC T-CJ
So
Q.
I'
-------
CO
111
111
a.
u.
O
\lt
LLJ
CO
cr
o>
S
a.
•s
CD
CD «
73 CO
CO C
II
to §
|i
S*~
CD 2
O
Z
O
LLl
CD
O
0§
LU §
o i
o Q
2 §
co cr
"52
<5
CO •»-'
CD C
eg •§
si
to «
£ CD
li
S D)
Q- CD
•gs
CO CO
U_ CO
tn
ui
co
ui
a
LL
O
1
s
a
ol
Ul D.
II
CO
Ul
O
£
a
55
Ul
a
5
i
LL
O
» gg|
O o 0*0
111 Q.CC
si I
CO HBP
ffl
-------
CO
3'
cr
£
CO
O)
Z .2"
55 $
< E~
UJ o S
CC ~ o
O -2 "£
•^ CO CO
i o
^ .CO *3
ft ^1
5 S|
13 =8
UJ « §
O_ 0 ir
••• M Qj
tSL ~ £
0 |8
u_ £ ®
wj Jr -^
Q ^ g
III CD CO
g !$
fc *il C
a 9 CD
5; 2 =
*^^fc CO Slk
^^^ «• Qj
1 ft
<5 ^ E
1 If
i 11
z 2 "•
i sf-
£ f|
CO gB
CD ^
Jr CO
O CO
U_ CO
(/)
UJ
z
(/}
MATRICE
Not Specified
CD
co
Not Specified
B
Estuarine wa
Not Specified
Wastewater
PCBs
Total metals
BTex and other
HCs
1
CD
•6
QJ
CO TO
"5 3.
CO CO
A
•2 a
5
aj
1
1
a.
cc
f£
CD
1
Flammable w
Giardia
Cryptosporidium
Legionella
Viruses
CO
m
Microbiologici
Trace metals
Organics
Pesticides
Herbicides
"co"
1
co
CO
CD
CO
.CO
CO
LL.
CD
1
M
CD
CO
Invertebrates
CD
5
.52
Same as fish 1
Plankton
CD
1
J= '
i
CD
CO
CO
'co"
CO
E
E
TO
.£.
tn
0
Biological mat
A
CQ
-------
X
cc
cc
HI
g
0)
UI
o
s
Q
CO
cr
i
I
CD
-2 §
l|
cr c
11
CO 3=
5! w £2
> t; S
Q
HI
d o
t_ x:
eo *"
5 £
8
CD"
CD 8
CD
•5g
0 CO
is
S?
3 C
W £S
s§
cS'a
l
"•5
CO
>»
Q.
Q.
(/)
1.
0)
ro
£
ZT
CO
.2
3
c
Jl
"o
Q.
<5
1
CO
CO
e>
o
o °
g it
*- CD -5
O) ^
4: E co S ^ E?
j?!c llif ff
CO Q) -~ c CJ) ff CO ^2 o *™ C O
CQ ^} 55 c ^ 5 ^™ •"- ® ^v •"" .2 cO O
O w .2 o.£ — ^S^c-cZo'o o o
0.
-------
(0
UJ
CO
HI
£L
O
o
m
cc
O(0
UJ
ceo
<
CO
UJ
CO
X
i
CO
UJ
CO
03
"«
o
O)
£
o
§
CO
CD
CO
i
O)
2
a.
CD
1
I
CD
-------
CO
LU
5
UJ
°- <£
O I
DC
o
LU
CC
CO
ffi
o
•8
.2
w
>,
CO
12
1
CO
X
CO
UJ
1
2
CO
I
D)
fi
5
CO
I
(D
£
00
*
ID
-------
(0
1U
CO
111
DL
O
z
c
o
UJ
DC
CO
y co
C ui
Q i
<
CO
UJ
CO
UJ
I
1
to
o
O)
I
w
TJ
3
CT
Q CD
LU< |
Qu o
CO S
CO
s
D)
S
Q.
0)
CO
•5)
o>
CM
-------
CO
sy
a
D
CO
UJ
o.
o
z
DC
5
a
LU
CC
CO
z
o
UJQ.
CO
p
CO
CO
HI
CO
o
g
JS
3
D>
2
1
"S5
•a
2
a*
CD
CO
I
T3
i
2
CO
JO
fi
I
o
£
o
«?
CD
-------
CO
LU
CO
LU
CL
O
O
UJ
or
s Q
CO
UJ
CO
CO
LU
CO
g
3
D)
B
ceo $
Q E
(0
(D
£
£
CO
2
O)
g
Q.
(1)
£
-------
CD
.Q
Q_ JS
CD Q
f 1
O o
II
SB IS
CD «I
co S-S
C/> SI.
O._ Ui.
§s
0 s>"§
O 2 §-
£- o £
MMBI Q}
°- s|
-t. co
rn i—
^•». •!= o
*^*» cr xj
EEjco
TJ r- tO
*l\ CO 0 .2
W ,n E "D
JC CD 53
Si III
O CD C
. o —
/^/\ w c: w
f/J CD O o •=
3> g CD S
^§"22
E E£
«*tit
o*5f
SlfB
V^^, _ CD
^pV> jr St. »-
"^ .a « o
_ . -i £ «
™^3 O CD
S ^"il
o 8*i
c §?1
CD £L"«
Olll
ficol
o •— °
CO *C3 *D)
CD = CD
D w CC
CO
CD
(/) CO
o c cn
•S O —
2- S. s ,
^ _ £ co cn
C co o m CD C Q
£ J _l CO .0 EC 5CC
m > ^^ 1-^ Q p o) I
CD S -J »
a: g o:
\. e
° * S
o°—
|||
>. CO
= co f "j >co
W E J §
JT UJ f
F
^•—
3c?I
" T3 TJ S
•J5 O 0> •?
^ ^2 .— 2"
*~~*j *•*** ">^ *^ C
ggcn o"> C S^J O co
(XJ 2 pr i _O DC_j J| CO
|— *l 'cT~'|~H
\le
1||
I ^ ^ »
E to co Q.
s==** ">*- ">*^ — c
a S_*j? o *" c o *" CJ) co
j& p^« 2 cc" .2 cc -J j co
O) • ^ JO '••"- fr O ~^ CO """
t/ * s
¥
111
cn «M
1 "S » I
i- -S- •§- - I
C £.*2 Q) cn c o> *" C3 co
O ^ j ® E-J ° CJ _l CO
— ""^^ •; k.^. ». IS
D) •— r ,g • r *** F co
j£ CO rr* ^
cc T\ u» m
\ /
* ^J
1=
ill
311
cn
j2 3 ^
~ cn m
= S_3
1A-|/^ u-
™ s\i
CO
co 00
£J a 3 ja oejfg H- °?o
3 3 S * Ij'X — » -~ '
co — in ,£•»— ' t*-C/5 a. CO > ui
S p o , ty*^ ||| — z ^ ^ „ .. 3 3 z •
Orr*iCr o> o> - - ~
°~ O EircciriraQcz>z
M CO
3 3
S<» -
C CD ^^
w c^.2cc xi ^J
>2 gi >_«
w oc
CO
S 2
c cn cn 3 .Q
g. 3 3 8_«
8 c S «3 S = ^ f \
•° ^ .5 ct. j c K-§^^ ^^
J cr § cc N^S
CO
cn
CD
c co « w <«
g. 3 3 3 3
Se:SuScSa£ «y^
^DCOCClcCOBCSECjs ^J
k QJ k p>— ^ CO~~~~T O>- ^ S ^ <0 ^^^m
T-J o 2 ® co -1
cc m tt
CO
CD
CO
§222
Q. 3 3 3
m O 0) C 0} ^. Q> ^^^T
a E 3 S 5 tr^S cc^a ^
-" 2 5 co J
£ oc
OQ
-------
CO "S
€l
3 CO
UJ-§,
D_ CD
O CC
£• CD
O 0)
•£ ®
CO *•;
Q. 0
0) P
Jl •- ^
CO 11
CO °S
o
jlir O •=
CC „-•§»
S Iff
•J F S CD
£2 £1
^^^H O *""*
*w -— ° c
(O O) g) ~w
VM r»" "2 ®
^.k LL. CO -r-
CD CD E 0
JHH £ co co
0) -g.i.8
O -p CO CO
•S^o
>>T3 .=
•° CD "5
« « -2
CO CO cj
§E-£
O k—
*— r™ O
a.-|-i=
CM CO
S^-g
-S>
CD 0 CO
QX OT
00
aBW 0 CO < < *?
!*• S ' ' ' ' O
(0 ^ ca co ^ a> T-
C Q C C C C C
.2 Q. .2 .2 .2 .2 .2
"in (/) O) O) O) O) O)
QJ QJ Q) 0> 0) 0) CD
3 cr uC GC OL QC Cm co co
° CO | 3 a,
§• i. 3 3 c »-i
•g E co co co /^ E
5 Ji £ o 3 o / °- ^ ^
CC "^ fl> ff "^ • fr ^^\
* UJ n ill
CO
£
si-§
0 0*0
1" 8 £
Co S •«
C "B 03 w -3 flj
"c o5 ^ ™~ o *•* ^* fl>
~j — 2- Q. 3 3 a: >=
oT* >•« c co co co / E
2. o CD^ _ a Q) o CD a> ,/ t:
c &••»= CC J o°5a>CCr-CCoCC c / JP ^^.
.2 £ ^ S ° Ji .2 / a flO
£ I
8gg
C ®»S
'§§•3
5 gc>
BLBC ^
h^ Mhi ^»
1" 8 £
!«!! |.ls£ 1|
^~! ">is;'>is! £ S" « « X^"E
> o 5— JSJ~ —w^o ®^» / E
c ^*- C_J e IE J oCOuOCr-CCfiOC «- / CD -
5 & ^o 3 £ o 3 & / °" ^
1 ^A *^
k. <*M
o CO
2g>o
IP
w
i
CO
-------
-------
APPENDIX C
FY1992 RESOURCE REQUIREMENTS FOR
WATER PROGRAM PERFORMANCE EVALUATION STUDIES
BY STUDY PHASE
-------
-------
o
Q
SI "1
a 1
o
+
§
I
<
z
<
is
w
g
o
w
i
D
O
CO
§
S
o
aJ
e
S
c>
8
S
c5
S
o
tl, P-I
°g
GO
.S
i
EC
u
?3
a
I
S
o
I
«•>
r-1
O
S
o
CO
eu
o
c
c
El
>
5:
W
I
P-
Cu
U
ie
-
.
OiaHOeaz
Jff
<-S 2 ^ S.- o-o « g.2
3 d rgS
i
-
JS MT3i
'-*
« 5
§1
< J
Oca
,l
« •=
I,
§"8 g
SS<|
zopS
-------
I
I
s
8
I
S
&
"S
a
c
ca
PL.
>
V.
oi
S
O
en
CO
o\
f^
s
S
8
o
*
CO
s
o
01
s
0
s
! «
60 C.
CO rsi
CO l
< <
2 2
T~t ^N
S S
g g-
hi W
a S
c>
M M
^y en
1 g
^ w
r~ *^
« ft
S $
c>
2 3
< bi .2 co .2
OO a u a
9. Produce final partici-
pants shipping lists.
10. Modify all data handling
to reflect study design.
•D
CO "O
£.5
% s
o •*
s °_
•^i CO
'Dased on cost of average con
"Resources for regions, state
-------
o
co O
PJ P4
W
0°
8|
o2
S
Q
0
fc
p
y
p
g
*•
C
P
c
c
£
>•
C
p
^
|
£
c
.2
"3
.0
•s
Q
•o
03
: Production
PHASE II
o
c
2
Q
CO
S
•o
a
CO
T-<
2
H
W
^
W
w
o
<<
0
•3 P^
r ^5
3 o
_^t
b.
ca
^*
>
a
55
S
ft
w
s
4
tf
5 w
1
(Xjj j i/^ j( ^« wi ^*
O22 22 '§ 2
•»
H
«5
i
fcrf fe4 td ^ --* .-*
S 0*1 ^ c*>
r-c r-l
H -^ o $ y S S
V,\ *• ^i f. f. do
tl
i
D
U
fe
C"» O »O f*t ^« *^!
s^ si •» s 2 a
^ o • i-1 <" y
z d A <=! ci "^
*
J
D
S
^ W g W <
m rr *** ^r
1
CO
t
^3
CU
O
PARTI
2 S * * is
w
0
"5
§
SM W to o "** « «
i! I ti !I !|i lift
«J2 gj .2 "« "5. "« .S2 ^ ^c^ o. S ^ — H- ^ a.-— — o^04^
a.
t^
S
t*
I
e
o
ost of average c
o
c
o
1
*
-------
"x
Ui
CO
o
o
Di
•o
+
3
o
1
EC
O
Z
T3
a
53
T-l
W
5;
"
E
e«5
Sj
«>J
a
|
1
00
s
x a
^
T.
s
s
o
"3 S
§ 8 **
< £ o
O O S
S
i
S
mf
-!
-------
u
a
i
w
CO
X
g
I
s
Q
1
1
I
55
1
i
CO
s ^
2 o
«~< ^
0 5
c> •*•
< «
2 V3
3 |
0 /i
_
S 8
•o
"S ^ S J «
" s co re to -:
£.« -o . = S §
I!!. HI
:§ « ! J S S §
s-g £ E « §.-§
c c u o "O ?r
6 «S U9 CL *™ ^
«> ^ «-T^ S 5
. 2 S £ . 8 «
c-i -— >• = n — a>
^
10
S
i
g
t<
S
C7\
§
^
S
1
Ox
I
9
s
01
il
TOTALS:
FTE TOTAL = 9.34
RAMURAL TOTAL
H
i-i
.
"2
S
.0
ts
?Q
1
1
IM
1
i
I
J2
O.
1
^3^
OO
«O
U9*
O
•s
a
8
t>
2
O
1*
wT
c
5^
"3
c
-a
73
1
I
e
CO
|
f
§
w*
o
s
2
S
ea
"""
^3
il
5*
_s
a.
:r WS Chcm Study: A
•
o
S
(A
1
I
'f
•6"
u
_G
S
O
"o
•o
u
•^
c:
CO
1
"§
a.
J
*S
C*
o
1
1
&
cu
!<
S
a
VI
»-i
=1
(3i
S
t/1
(A
"e
g.
o
s
'i
1
13
C
5
.O
|
<^
i
c
ca
•§
8
C
•>
&
M
s
«>
LO-
V
•§
1
JH
i
•5*
CO
_S
a.
a.
:r WS Micro Study: A
rmed as fecal coliform,
ft. •£
-: P
"g
S
i
.S3
•o
^j
c
to
*o
i
j>
TJ"
8
1
p.
D.
C
CO
§
ST
So"
|
!
2
CO
O,
en
?«
e
00
JS
w
«>
1
S
1
i
s
jj.
CO
s
1
d,
ft.
t«
ft.
_ r
•O
O
~5
.0
^
g
1
•c
?
*T^
8
•o
§,
42
"3
0,
1
^o
3
"3
M
O
S
CB
Q.
S
O
T™<
if
s>
CO
s
if
0
i
1
S
«^N
t>
cs
i
I
S*>
^3
sr DMR-QA Cheni Sli
B,
„«
w
g
=5
"e
S
•g
•o
c
CO
a>
o
CO
1
*fc-
J2
o
§
ii
•3
t*
i
e
o
f
u
3
1
g
g
"a
•o
"3
.2*
i
a
•o
1
if
ss.
E?
CO
'•i
S
«r
8
jq
^
c
1
0
o
!
i
J:>
**>
To
.§
"x
!
^
:r DMR-QA Tox Stud]
ft.
...
i4
C3
t
S
I
G
0
u
1
1
8
u
c:
O
•o
8
ca
-------
-------
APPENDIX D
OFFICE OF WATER PERFORMANCE EVALUATION STUDY PROJECT
ISSUES IDENTIFIED DURING STAFF INTERVIEWS
-------
-------
ISSUE
OFFICE OF WATER PE STUDY PROJECT
SUMMARY OF ISSUES BY TYPE, PROGRAM AND STUDY
July 9, 1992
PROGRAM
STUDY
DESIGN ISSUES
1. PEs do not measure routine lab performance OGWDW
OST
A. Directions unclear
D. Samples not "double blind'
E, Linkage to performance standards/
certification status leads to "over
performance"
Regions
OGWDW
Regions
B. No enforcement mechanism for directions OGWDW
Regions
C. Concentration ranges known in advance OGWDW
Regions
OGWDW
Regions
OGWDW
Regions
WS
WS
WP
DMRQA
OST
WS
WP
DMRQA
WS
WS
WP
DMRQA
WS
WS
WP
DMRQA
WS
WS
WP
DMRQA
WS
WS
WP
DMRQA
WS
WS
WP
-------
ISSUE
PROGRAM
STUDY
DESIGN ISSUES (CONT)
2. Considerable duplication across studies
OST
OWEC
Regions
WS
WP
DMRQA
WP
DMRQA
WS
WP
DMRQA
ADMINISTRATION ISSUES
1. Criteria for including laboratories are
inconsistent/unclear
2. Number of studies overloads Regional and
state laboratories
3. Inefficiencies in sample distribution exist
A. Laboratories receive all samples
regardless of need
B. Laboratories receive multiple sets of
samples
Studies too infrequent for effective
data quality control
Regions
OGWDW
Regions
WS
WP
WS
WP
WS
WP
OWEC
Regions
OWEC
Regions
Regions
DMRQA
WP
DMRQA
DMRQA
DMRQA
WS
DMRQA
USE ISSUES
1. Study data inappropriately used for numeric
standard-setting
2. Used for state certification programs
OGWDW
OWEC
Regions
WS
DMRQA
WS
WP
DMRQA
-------
ISSUE
USE ISSUES (CONT)
PROGRAM
STUDY
3. Used for non-water program applications
(e.g., RCRA method validation, coal mining
laboratory certification)
Regions
WP
FUNDING ISSUES
1. Users do not contribute to funding OWOW
accordingly
2. Privatization may be appropriate OWEC
3. Effect of privatization on state programs Regions
WP
DMRQA
WS
WP
DMRQA
-------
ISSUE
OFFICE OF WATER PE STUDY PROJECT
SUMMARY OF ISSUES FROM EMSL-CINCINNATI INTERVIEWS
DRINKING WATER PROGRAM/WATER SUPPLY STUDY
August 5, 1992
PROGRAM
STUDY
DESIGN ISSUES
1. Study design is cumbersome and expensive.
A. Rapidly increasing number of analytes
adds cost and increases design
complexity.
2. PEs do not measure routine laboratory
performance.
A. Laboratories anticipate concentration
ranges based' on their experience and
knowledge of MCLs and acceptance levels.
B. Samples are not "double blind."
C. Linkage to certification status
leads to "over performance."
3. Micro PEs not required by national program
for local Micro laboratories.
A. Inconsistent QA policy (vs. Chem and
Rad).
B. Some states have implemented programs
on their own.
C. Expanding micro PEs would be expensive;
current production at capacity.
4. Allowable analytical methods are not
evolving with technological advances.
A. PE analyte concentrations based on least
sensitive allowable method.
B. Laboratories may "re-tool" for PE
studies, or, in some "cases, must
maintain old, special equipment for PEs
(e.g., packed column method for THMs).
EMSL
EMSL
EMSL
TSD
EMSL
EMSL
EMSL
EMSL
TSD
EMSL
TSD
EMSL
Regions
EMSL
Bionetics
EMSL
TSD
EMSL
TSD
TSD
WS
WS
WS
ALL
ALL
ALL
WS
WS
WS
WS
WS
WS
WS
-------
ISSUE
PROGRAM
2
STUDY
DESIGN ISSUES (CONT)
5. Fixed acceptance limits create technical EMSL
performance problems at concentrations near
method detection limits.
A. Initial high failure rate due to EMSL
narrow acceptance range.
B. Assumes linearity in non-linear range; EMSL
scientifically invalid.
6. Acceptance range limits set at 95% EMSL
level; acceptable range narrower
than WP and DMRQA (barrier to
consolidation).
7. Conversion of analytes from unregulated EMSL
to regulated status should be timed to
allow for certification prior to effective
date of rules.
8. Utility of requiring analysis of a EMSL
high and low concentration ampul for
each analyte.
WS
WS
WS
WS
WS
WS
USE ISSUES
1. Using PE study data for standard-setting TSD
and regulation development. EMSL
A. PE studies are an efficient way to EMSL
generate method performance database.
B. Separate program to test method TSD
performance and generate data needed
standard-setting should be established.
C. PE study data do not represent EMSL
typical or average laboratory TSD
conditions and are therefore not
appropriate for standard-setting.
D. Unregulated chemicals add to expense EMSL
and complexity of study design.
2. Micro PEs used for ambient and wastewater EMSL
monitoring laboratories (especially states).
WS
WS
WS
WS
WS
-------
ISSUE
ADMINISTRATION ISSUES
1. Report distribution process is
cumbersome and time consuming.
A. Minimum of 4 copies per laboratory
must be distributed.
B. EMSL distributes to many sources
(Regions, states, others).
2. Study results often not timely.
Considerable delays reported.
3. Modernization
A. Automated information handling and
data access.
B. Linkage to other water databases.
C. Automating the production process.
PROGRAM
EMSL
EMSL
EMSL
TSD
Regions
EMSL
EMSL
EMSL
EMSL
3
STUDY
ALL
ALL
ALL
WS
ALL
ALL
ALL
ALL
FUNDING ISSUES
1. Study growing in scope and complexity;
costs increasing; funding steady/eroding.
2. Asbestos PE material requires funding
for further development (stability
study).
3. Funding required to expand coverage
of Micro PE study.
EMSL
EMSL
EMSL
WS
WS
WS
-------
OFFICE OF WATER PE STUDY PROJECT
SUMMARY OF ISSUES RESULTING FROM EMSL-CI VISIT/WP
August 10, 1992
ISSUE
PROGRAM
STUDY
DESIGN ISSUES
1. Analytes "frozen" in early 1970s. EMSL
A. No guidance on analytes or EMSL
concentrations from
Headquarters.
B. Design does not necessarily EMSL
reflect current uses of study
results.
2. Analytes overlap with DMRQA Chem. EMSL
and WS.
Microbiological PE needed
surface water monitoring.
for EMSL
WP
WP
WP
WP
WP
-------
OFFICE OF WATER PE STUDY PROJECT
SUMMARY OF ISSUES RESULTING FROM EMSL-CI VISIT/DMRQA
August 10, 1992
ISSUE
DESIGN ISSUES
1. WET study methods
A. Number , of
variations
complicated.
PROGRAM
STUDY
methods and
makes study
B. Methods not necessarily
representative of monitoring
requirements.
EMSL
EMS I,
EMSL
DMRQA/WET
DMRQA/WET
DMRQA/WET
ADMINISTRATION ISSUES
1.
2.
3.
Test kits for Chemistry sent to
permitees rather than to
laboratories
A.
B.
Identity and numbers of
laboratories not known; sending
kits to laboratories designated
by permittees could save
resources.
Sending kits directly to
designated laboratories
requires additional resources
for information
management/tracking.
List of recipients based on
often out of date.
Requirement for technical assistance
to laboratories is significant
(Bionetics receives ca. 1000 calls
per month during the study; EMSL,
regions and states also receive
calls).
A. Agency contact should be
identified in study
instructions.
EMSL
EMSL
DMRQA/Chem
DMRQA/Chem
Bionetics
DMRQA
PCS; Bionetics
Bionetics
EMSL
Regions
EMSL
DMRQA
ALL
ALL
-------
ISSUE
PROGRAM
STUDY
ADMINISTRATION ISSUES (CONT)
B. Policy needed for replacement
of "lost" and/or "broken"
ampuls.
4.
Automation requirements
A. Handling of Announcement
Letters should be automated.
B. Numbers of reports distributed
for each laboratory (6) is
unnecessary. Sending hard
copies is costly and
cumbersome.
USE ISSUES
1. WET data used for method validation
only; not applicable for enforcement
because of method complications.
FUNDING ISSUES
1. Resources required for technical
assistance to laboratories.
2. Sending Chemistry test kits to
laboratories designated by permitees
may significantly decrease study
costs.
3. WET should be expanded to include
significant minor permittees.
EMSL
EMSL
EMSL
EMSL
EMSL
EMSL
EMSL
EMSL
ALL
DMRQA
ALL
ALL
DMRQA/Chem
ALL
DMRQA/Chem
DMRQA/WET
-------
-------
APPENDIX E
OFFICE OF WATER PERFORMANCE EVALUATION STUDY PROJECT
SUMMARY OF ANALYTES BY STUDY
-------
-------
OFFICE OF WATER PE STUDY PROJECT
COMPARISON OF ANALYTES AND HISTORICAL CONCENTRATION RANGES
ANALYTE
Trace Metals
Aluminum
Antimony
Arsenic
Barium
Beryllium
Boron
Cadmium
Chromium
Colbalt
Copper
Iron
Lead
Manganese
Mercury
Molybdenum
Nickel
Selenium
Silver
Strontium
Thallium
Titanium
Vanadium
Zinc
Nutrients
N-Nitrate
N-Nitrite
N-Ammonia
WS
.046 - 1 .5 mg/L
.006 -.01 5
.025 -.15
.5 - 1 .0
.0004 - .01
*
.0025 - .05
.05 - .2
NT
.1-1.0
NT
.0025 - .1
NT
.0005 - .008
.003 - .07
.15- 1.9
.005 - .1
.025- .12
NT
.002 - .01
NT
.022 - 1 .9
.013-1.3
.1-10 mg/i
.05 - 2
NT
WP
.04 - 3.5 mg/L
.01 5 -.2
.007 - .45
NT
.006 - .9
NT
.002 - .45
.006 - .9
.008 - .9
.008 - .9
.01 - 1 .9
.016-1.4
.015- 1.0
.0003 - .05
.003 - .08
.01 2 - 2.2
.01 - .2
.0005- .01 7
.001 5 -.085
.003 - .1
.035 •• .3
.022- 10.0
.012-1.9
.25 - 1 5 mg/L
:NT
.25 - 20
DMRQA
.95 - 4 mg/L
NT
.004 - .5
NT
.002 - .1
NT
.003 - .225
.05 - 1 .0
.1 -.5
.026 - 1 .0
.3 - 4.O
.025 - .6
.1 - 4.0
.0002 - .005
NT
.071 - 2.92
.01 - .1
NT
NT
NT
NT
6.0 - 20.0
.095 - 2.0
2-40 mg/L
NT
2- 14.5
-------
ANALYTE
N-TKN
P-Orthophosphate
P-Total Phosphate
Demand
BOD
CBOD
TOC
COD
Cyanide
Residue
TSS/Non-filterable
Turbidity
Total Dissolved Solids
Other Chemistry
CaCO3/Ca+ +
CaCOS/Alkalinity
Calicum
Chloride
Chlorine (Total Residual)
Conductivity
Corrositiviy (Agressive
Index)
Corrositivity (Langlier
Index)
Fluoride
Hardness
Magnesium
Oil and Grease
PH
Potassium
WS
NT
NT
NT
NT
NT
NT
NT
.02-5.0**
NT
.5 - 8.0 NTU
200-400**
90-250**
25-50**
NT
NT
.25- 1.8**
NT
«•
*
.1 -8
NT
NT
NT
6.0-9.3 (pH Units)**
NT
WP
.35 - 35 mg/L
.05 - 5.5
.15- 10
10-225
10-200
5- 100
12-250
.02- 1
20 - 100
NT
30 - 650
NT
4.5- 120
1.3-110
8.0 - 250
.1 - 5.0
50.0- 1050
NT
NT
.1 -4
12-350
.6-40
5-50
4.0 - 9.5 pH units
.8-40
DMROA
3 - 36 mg/L
.3-1.5
0.8- 3
15-45
14.8-40
5-70
50 - 200
0.01 - 1
23-50
NT
NT
NT
NT
NT
NT
0.018- 1
NT
NT
NT
NT
NT
NT
10-20
6 - 9 pH Units
NT
-------
ANALYTE
Sodium
Sulfate
Total Phenolics (4-AAP)
Herbicides
2,4 -D
2,4,5 - TP (Silvex)
Bentazon
Dalapon
DCPA Acid Metabolites
Dicamba
Oinoseb
Picloram
Pentachlorophenol
Pesticides
Alachlor
Aldrin
Atrazine
Bromocil
Chlordane
DDT
DDE
DDD
Dieldrin
Diquat
Endothall
Endrin
Glyphosate
Heptachlor
WS
12-24mg/L*»
5.0-40**
NT
.001 - .15
.004 - .025
1 -20**
.0075- .1**
.1-2**
.32-2**
.001 -.1**
.0075- .1**
.0004- .1
.001 - .02
NT
.003 - .03
.01 25 -.050**
.0007 - .005
NT
NT
NT
NT
.0025 -.05**
.OS -.5**
.0001 - .0015
.03- .5**
.00005 - .0025
WP
5 - 1 25 mg/L
5-125
.02-5
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
.0001 - .001
NT
NT
.001 - .015
.0001 -.0015
.0001 - .001
.0001 -.0015
.0001 - .001
NT
NT
NT
NT
.00008 - .001
DMRQA
NT
NT
0.003 - 0.6
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
-------
AIMALYTE
Heptaohlor Epoxide
Hexachlorocyclopentadiene
Hexachlorobenzene
Undane
Methoxychlor
Metolachlor
Metribuzin
Prometon
Simazine
Toxaphene
Trifluralin
PCBo
Aroclor 1016/1242
Aroclor 1 248
Aroclor 1 260
Aroclor 1232
Aroclor 1254
Decachlorobiphenyt
PCBs in on
Aroclor 1254
Aroclor 1 260
Aroclor 101 6/1 242
Volatile Organics
Arbon Tetrachloride
Benzene
Bromobenzene
Bromochloromethane
Bromodichloromethane
WS
.000075- .0015mg/L
.00065 - .005
.00005 - .0025
.000075- .0015
.005 - .1
.00375 -.030**
.00075 - .020**
.0015- .030**
.02 - .2
.0025 -.01 5
.15-10
NT
NT
NT
NT
NT
.0005 -.005**
NT
NT
NT
.0025 - .02
.0025 - .02
.005- .02**
.002- .02**
.01 - .03
WP
.00008 - .001 mg/L
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
.001 -.015
.001 - .01 5
.001 -.015
.001 -.015
.001 -.015
NT
10-50
7-50
10-50
NT
.004 - .1
NT
NT
.005 - .065
DMROA
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
-------
ANALYTE
Bromoform
Bromomethane
n-Butylbenzene
sec-Butyibenzene
tert-Butylbenzene
Carbontetrachloride
Chlorobenzene
Chlorodibromoethane
Chloroethane
Chloroform
Chloromethane
o-Chlorotoluene
p-ChlorotoIuene
Dibromochloromethane
1,2-Dibromo-3-chloropropane
Dibromomethane
m-Dichlorobenzene
o-Dichlorobenzene
p-Dichlorobenzene
Dichlorodifluoromethane
1,1-Dichloroethane
1,2-Dichloroethane
1,1 -Dichloroethene
cis-1,2-Dichloroethytene
trans-1,2-Dichloroethylene
Dichl.oromethane
1,2-Dichloropropane
1,3-Dichloropropane
2,2-Dichloropropane
1,1 -Dichloropropene
ws
.01 - .04 mg/L
.005- .02**
.002 -.02**
.002- .02**
.002- .02**
NT
.002 - .2
.002 -.02**
.002 - .02**
.01 - .06
.005 -.02**
.002 -.02**
.002- .02**
.01 - .03
.0001 - .02
.002 -.02**
.005 -.03**
.005 - .02
.0025 - .02
.002 -.02**
.002- .02**
.002 - .02
.002 - .02
.002 - .2
.002 - .2
.005- .02**
.0025 - .02
.0025- .02**
.005- .02**
.002- .02**
WE
.003 - .07 mg/L
NT
NT
NT
NT
.002 - .08
.005 - .07
NT
NT
.005 - .075
NT
NT
NT
.003 - .065
NT
NT
.005 - .1
.0025 - .1
.005 - .1
NT
NT
.002 - .07
NT
NT
NT
NT
NT
NT
NT
NT
DMRQA
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
. NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
-------
ANALYTE
cis-1,3-DichIoropropene
trans-1,3-DichIoropropene
Ethylbenzene
Ethylene Dibromide
Fluorotrichloromethane
Hexachlorobutadiene
Isopropylbenzene
p-lsopropyitoluene
Methylene Chloride
n-Propylbenzene
Styrene
1,1,1,2-Tetrachloroethane
1,1,2,2-TetrachIoroethane
Tetrachloroethylene
Toluene
Total Trihalomethanes
Total Xylenes
1,2,3-Trichlorobenzene
1,2,4-Triohlorobenzene
1,1,1-Trich!oroethane
1,1,2-Trichloroethane
Trichloroethene
1,2,3-Trichloropropane
1,2,4-Trimethyibenzene
1,3,5-Trimethylbenzene
Vinyl Chloride
m-Xyiene
o-Xytene
p-Xyiene
ws
.002- .02mg/U**
.002- .O2»*
.002 - .02
.000025 - .02
.002- .02 **
.002- .02**
.002- .02**
.002- .02**
NT
.002 -.02**
.002 - .2
.002- .02**
.002- .02**
.002 - .02
.002 - .02
.0275 - .25
.002 - .05
.002 -.02**
.002 -.02**
.002 - .02
.002- .02**
.002 - .02
.002 -.02**
.002 -.02**
.002 -.02**
.001 - .005
.002 - .02
.002 - .02
.002 - .02
WP
NT
NT
.004 - .1
NT
NT
NT
NT
NT
.004 - .075
NT
NT
NT
NT
.0022 - .075
.003 - .1
NT
NT
NT
NT
.002 - .07
NT
.004 - .065
NT
NT
NT
NT
NT
NT
NT
DMRQA
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
Nt
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
-------
ANALYTE
Carbamates
Aldicarb
Aldicarb Sulfoxide
Aldicarb Sulfone
Carbofuran
Methomyl
Vydate
Adipate/Phthalate
Bis-(2-ethylhexyl) Phthalate
Butyl Benzyl Phthalate
Di-(2-ethylhexyI) Adipate
Oiethyl Phthalate
Dimethyl Phthalate
Di-n-butyl Phthalate
WS
WP
DMRQA
PAHs
Acenaphthylene
Anthracene
Benzo (a) Anthracene
Benzo (k) Fluoranthene
Benzo (b) Huoranthrene
Benzo (g,h,i) Perylene
Benzo (a) Pyrene
Chrysene
Dibenzo (a,h) Anthracene
Fluoranthene
Fluorene
Indeno (1,2,3-c,d) Pyrene
Naphthalene
.005- .05 mg/L**
.005- .05**
.01 - .1**
.0075 -.15"
.0025 -.025**
.01 - .1**
.004- .08**
.0025- .05**
.003- .06**
.004- .08**
.0015- .03**
.02 - .2**
.0005- .01**
.0005- .01**
.001 - .02**
.001 5 -.03**
.0015 -.03**
.005- .01**
.005-.!**
.0015 -.03**
.0005- .01**
.0015 -.03**
.001 - .02**
.0005 -.01**
.015- .05**
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
-------
ANALYTE
Phenanthrene
Pyrene
Organic Disinfection By-Products
Bromochloroacetonitrile
Chloral Hydrate
Dibromoacetic Acid
Dibromoacetonitrile
Dichloroacetic Acid
Dichloroacetonitrile
1 ,1-Dichloropropanone
Monobromoacetic Acid
Monochloroacetic Acid
Trichloroacetic Acid
Trichloroacetonitrile
1 ,1 ,1-Trichloropropanone
Inorganic Disinfection By-Produts
Bromate
Chlorate
Chlorite
VIS
.001 -.02mg/L**
.0005 -.01**
.0005 - .01
.005 - .03
.005 -.01 5
.0005 - .01
.005 - .5
.0005 - .01
.0005 - .01
.005 -.01 5
.005 -.01 5
.005 - .05
.0005 - .01
.0005 - .01
.02 - .1
.03 - .2
.03 - .6
WP
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
DMRQA
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NT
NJ Not Tested
• Indicates analyte listed by EMSL-Ci, but no concentration range quoted.
*• Unregulated analyte.
-------
OFFICE OF WATER PE STUDY PROJECT
COMPARISON OFANALYTES LISTED IN TABLES II - V WITH PE STUDY ANALYTES
(4O CFR, PART 122, APPENDIX D)
TABLE II (40 CFR PART 122. APPENDIX D)
ANALYTE WS
Volatiles
Benzene X
Bromoform X
Carbon Tetrachloride
Chlorobenzene X
Chlorodibromomethane X
Chloroethane X
Chloroform X
1,1-Dichloroethane X
1,2-Dichloroethane X
1,1 -Dichloroethylene X
1,2-Dichloropropane X
1,3-Dichloropropylene X
Ethylbenzene X
Methylene Chloride
1,1,2,2-Tetrachloroethane X
Tetrachloroethylene X
Toluene X
1,2-trans-Dichloroethylene X
1,1,1-Trichloroethane X
1,1,2-Triohloroethane X
Trichloroethylene X
Vinyl Chloride X
Base/Neutrals
Acenaphthylene X
Anthracene X
WP
X
X
X
X
X
X
X
X
-------
ANALYTE WS
Benzo (a) Anthracene X
Benzo (k) Ruoranthene X
Benzo (g,h,i) Perylene X
Benzo (a) Pyrene X
Bis (2-ethylhexyI) Phthalate X
Butyl Benzyl Phthalate X
Chrysene X
Dibenzo (a,h) Anthracene X
1,2-Dichlorobenzene X
1,3-Dichlorobenzene X
1,4-Diehlorobenzene X
Diethyl Phthalate X
Dimethyl Phthalate X
Di-n-butyi Phthalate X
Ruoranthene X
Fluorene X
Hexachtorobenzene X
Hexachlorobutadiene X
Hexachlorooyclopentadiene X
Indeno (1,2,3-c,d) Pyrene X
Napthalene X
Phenanthrene X
Pyrene X
1,2,4-Trichlorobenzene X
WP
DMROA
Pesticides
Aldrin
Chlordane
DDT
DDE
DDD
X
X
X
X
X
X
X
X
-------
ANALYTE
Dieldrin
Endrin
Heptaclor
Heptachlor Epoxide
PCB-1 242
PCB-1254
PCB-1 232
PCB-1 248
PCB-1 260
PCB-1 01 6
Toxaphene
TABLE 111 (40 CFR, PART 122, APPENDIX D)
ANALYTE
Antimony
Arsenic
Beryllium
Cadmium
Chromium
Copper
Lead
Mercury
Nickel
Selenium
Silver
Thallium
Zinc
Cyanide
Total Phenols
ws
X
X
X
X
ws
X
X
X
X
X
X
X
X
X
X
X
X
X
X
WP
X
X
X
X
X
X
X
X
X
WP
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
DMRQA
DMRQA
X
X
X
X
X
X
X
X
X
X
X
X
-------
TABLE IV (40 CFR, PART 122, APPENDIX D)
ANALYTE
Chlorine, Total Residual
Fluoride
Nitrate-Nitrite
Oil and Grease
Sulfate
Aluminum
Barium
Boron
Cobalt
Iron
Magnesium
Molydenum
Manganese
Titanium
X
X
X
X
X
X
X
WP
X
X
X
X
X
X
X
X
X
X
X
X
DMRQA
X
X
X
X
X
TABLE V {40 CFR. PART 122, APPENDIX D)
ANALVTE W§
2-D X
Dicamba X
Diquat X
Ethylene Dibromide ' X
Methoxychlor X
Strontium
Styrene X
2,4,5-TP (Silvex) X
Vanadium X
Xyiene X
WP
DMRQA
-------
ANAL YTES LISTED IN TABLES II - V WITH NO PE STUD Y
(40 CFR, PART 122, APPENDIX D)
TABLE II (40 CFR, PART 122, APPENDIX D)
ANALYTES
Volatiles
Acrolein
Acrylonitrile
2-Chloroethylvinyl Ether
Dichlorobromomethane
Methyl Bromide
Methyl Chloride
Acid Compounds
2-Chlorophenol
2,4-Dichlorophenol
2,4-Dimethylphenol
4,6-Dinitro-o-cresol
2,4-Dinitrophenol
2-Nitropheno!
4-Nitrophenol
p-Chloro-m-Cresol
Pentaohlorophenol
Phenol
2,4,6-Trichlorophenol
Base/Neutrals
Acenaphthene
Benzidene
3,4-Benzofluoranthene
Bis-(2-chloroethoxy)methane
Bis-(2-ohloroethyl)ether
-------
ANALVTES
Bis-(2-chloroisopropyl)ether
4-BromophanyI Phenyl Ether
2-ChIoronaphthalene
4-ChlorophenyI Phenyl Ether
3,3'-Dichlorobenzidine
2,4-Dinitrotoluene
2,6-DinitrotoIuene
DJ-n-Ootyl Phthalate
1,2-Diphenylhydrazine (as azobenzene)
HexaoMoroethane
Isophorone
Nitrobenzene
N-nitrosodimethylamine
N-nitrosodi-n-propylamine
N-nitrosodiphenylamine
Pesticides
alpha-BHC
beta-BHC
gamma-BHC
delta-BHC
alpha-Endosutfan
beta-Endosulfan
Endosulfan Sulfate
Endrin Aldehyde
PCB-1221
-------
TABLE IV (40 CFR, PART 122. APPENDIX D)
ANALTYE
Bromide
Color
Fecal Coliform
Nitrogen, Total Organic
Phosphorous
Radioactivity
Sulfide
Sulfite
Surfactants
Tin
-------
TABLE V (CFR 40. PART 122, APPENDIX D)
Toxic Pollutants
Abestos
Hazardous Waste
Acetaldehyde
Ally) Alcohol
Ally! Chloride
Amyl Acetate
Aniline
Benzonitrile
Benzyl Chloride
Butyl Acetate
Butyiamine
Captan
Carbaryl
Carbofuran
Carbon Disulfide
Chlorpyrifos
Coumaphos
Cresol
Crotonaldehyde
Cyclohexane
Dimethylamine
Diazinon
Disulfoton
Dichlobenil
Dichlone
2,2-Dichloropropionic Acid
Dichlorvos
Diethy) amine
Dintrobenzene
Diuron
Epichlorohydrin
Ethion
Ethylene diamine
Formaldehyde
Furfural
Guthion
Isoprene
Isopropanolamine
Kelthane
Kepone
Malathion
Mercaptodimethur
Dimethyl Amine
Methyl Mercaptan
Methyl Methacryiate
Methyl Parathion
Mevinphos
Mexacarbate
Monoethyl Amine
Monomethyl Amine
Naled
Napthenic Acid
Nitrotoluene
Parathion
Phenolsulfanate
Phosgene
Propargite
Propylene oxide
Pyrethrins
Quinoline
Resorcinol
Strychrine
Disulfoton
2.4,5-T
TDE
Trichlorofan
Triethanolamine
Dodecylbenzenesulfonate
Triethylamine
Trimethyl amine
Uranium
Vinyl Acetate
Xylenol
Zirconium
------- |