I'PTl !H:T ''"""! "M'l
-------
Inspector General Division
Conducting the Audit: Northern Audit Division
Region Covered: Region 5
Program Offices Involved: Region 5 Water Division
Chicago, IL
Office of Ground Wat^r
and Drinking Water
Washington, D.C.
-------
905R94028
» UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
*- OFFICE OF THE INSPECTOR GENERAL
NORTHERN DIVISION
77 WEST JACKSON BOULEVARD
CHICAGO, IL 60604-3590
September 30, 1994
MEMORANDUM
SUBJECT: Region 5 Procedures to Ensure
Drinking Water Data Integrity
Audit Report No. E1HWC4-23-0009-4100540
FROM: Anthony C. Carrollo ^,
Divisional Inspector 6eneral"for Audit
Northern Audit Division
TO: Valdas V. Adamkus
Regional Administrator, Region 5
Attached is the final audit report titled Region 5
Procedures to Ensure Drinking Water Data Integrity. This audit
report contains findings that describe the results of our audit
of Region 5 and its states' procedures for reviewing the validity
of drinking water test data.
We appreciate the cooperation we received from your staff in
conducting this review. Your staff's professionalism and
dedication resulted in a cooperative effort to resolve the issues
in this report. We hope that our report will be useful to you as
you continue your work to assure a safe drinking water supply
throughout the Region.
This report represents the opinion of the Office of the
Inspector General. Final determinations on matters in this
report will be made by EPA managers in accordance with
established EPA audit resolution procedures. Accordingly, the
findings described in this audit report do not necessarily
represent the final EPA position and are not binding upon EPA in
any enforcement proceeding brought by EPA or the Department of
Justice.
Your response to our draft report is included as
appendix 1. Based on this response and discussions at the exit
conference, we made appropriate changes to this final report,
including the addition, deletion, and revision of text and the
revision of recommendations made in the draft report.
Action Required
In response to the draft report, your office provided
responsive action plans and milestone dates for correcting the
findings. As a result, and in accordance with EPA Order 2750, we
Printed on Recycled Paper
-------
find your response to the report acceptable. Therefore, we are
closing this report in our tracking system as of this date.
Please track all planned actions and milestone dates in the
Management Audit Tracking System.
We have no objections to the further release of this report
to the public. Should you or your staff have any questions
regarding this report, please contact Kimberly O'Lone, Audit
Manager, Northern Audit Division, at (312) 886-318G.
Attachment
-------
DRINKING WATER DATA INTEGRITY
EXECUTIVE SUMMARY
PURPOSE
Office of Ground Water and Drinking Water (OGWDW) officials
requested that the Office of the Inspector General (OIG)
conduct this audit: (1) because of their concern that
falsified data may be going undetected and could result in
potential threats to human health and (2) to aid them in
planning how to use limited resources efficiently and
effectively.
If water system operators falsify test results, serious
health risks to the public could result. Moreover, if a
state does not detect questionable test data and investigate
the situation, the state's ability to take proactive measures
to prevent contamination of the water supply is negated.
The Centers for Disease Control and Prevention (CDC)
estimated that over 940,000 people become ill every year from
drinking contaminated water. Of those, CDC estimated 900
people die each year.
The objectives of the audit were to determine:
• whether cases of invalid or falsified data have gone
undetected, and
• what procedures Region 5 and its states are using to
detect invalid or falsified data.
BACKGROUND
The Environmental Protection Agency's (EPA) OGWDW, the
regions, and the states are responsible for ensuring that the
water the public drinks is safe and free of harmful
contaminants. However, over the last decade statutory
monitoring and testing requirements increased. According to
Region 5 officials, limited EPA and state resources were
available for reviewing the validity of reported drinking
water test data. Falsified drinking water test data could
result in serious health risks to the public.
OGWDW guidance published in 1991 to assist regions and states
in detecting falsified data stated that the problem of data
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
falsification may be larger than originally suspected.1 It
also stated that
...with the promulgation of new regulations under
the Safe Drinking Water Act, there is an increased
incentive for water systems to purposely submit
invalid compliance data. This is due to the cost
of compliance with the new regulations and an
increased complexity in the monitoring and
reporting requirements.
During preliminary work conducted in three Region 5 states
from October to December 1993, we found several cases of
questionable drinking water test data that the states did not
detect. Moreover, we found a general lack of EPA and state
resources committed to detecting falsified data.
RESULTS-IN-BRIEF
Operators at public water systems (PWSs) reported invalid and
potentially falsified data, which went undetected by their
states. Such inaccurate reporting lessens the integrity of
the drinking water data which, in turn, could result in
increased risks to public health.
Changes are needed to (1) raise the awareness of Region 5 and
state officials regarding data falsification issues and (2)
improve states' capabilities for detecting questionable
drinking water test data. We found that state officials (1)
generally trusted that the data reported to them was valid
and (2) were unaware of the potential for data falsification.
We visited and inspected seven PWSs and found problems at
three of them that indicated that data might have been
falsified in the past.
State and Regional officials placed a low priority on
reviewing the integrity of drinking water test data. States
did not have formal procedures for, and spent very few
resources on, reviewing operational drinking water test data
for falsification. In addition, Region 5's oversight of
states' efforts to detect questionable data needed
improvement. Regional and state officials told us that they
did not believe data falsification occurred very often and,
therefore, could not justify spending limited resources on
•"•The relevant section of this guidance is referred to as
Appendix G. See chapter 1.
11
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
the detection of falsified data given other program
priorities.
PRINCIPAL FINDINGS
PWS Operators Reported Questionable Test Data
All PWS operators are required, under the Safe Drinking Water
Act and EPA's regulations, to ensure that all appropriate
drinking water tests are conducted and test results are
accurately reported. Accurate test data are necessary to
assure that the quality of drinking water provided to the
public meets all drinking water standards.
Files for about 10 percent (51) of the 495 PWSs we reviewed
contained test data that were questionable in accordance with
EPA guidance. Based on further Regional and state
investigations, we determined that operators at about 4
percent of the PWSs we reviewed reported questionable test
data, without their states detecting it. Most of these cases
were at community PWSs.
Undetected questionable test data--whether invalid or
potentially falsified--could (1) result in contamination of
the water supply not being promptly detected and (2) lead to
serious public health problems. PWS operators may have
reported invalid test data because they lacked training on
proper testing and reporting procedures, or had
malfunctioning equipment. Operators may have reported
falsified data to cover-up problems at the PWS or to avoid
additional requirements or corrective actions. We provided a
copy of this report to the OIG Office of Investigations for
its review.
States' Efforts To Detect
Questionable Test Data Need Improvement
State officials placed low priority on reviewing records for
invalid or falsified data primarily because they did not
believe that falsification was widespread. State officials
generally trusted that data reported to them was valid and
seemed to be unaware of the potential for data falsification.
In addition, according to officials in some states, limited
resources do not allow for any detailed reviews. As a
result, invalid or potentially falsified data has gone
undetected. In accordance with criteria in 40 Code of
Federal Regulations (CFR) Part 142, primacy states are
responsible for administering and enforcing drinking water
iii
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
regulations and related requirements, including general
review and oversight of PWS-reported test data. In addition,
according to OGWDW guidance, states should try to identify
and prosecute PWSs suspected of falsifying data. However,
according to state officials, past experience has shown that:
developing successful prosecutions of data falsification
cases is very time consuming and difficult.
Region 5 Oversight Needs Improvement
Region 5's oversight of drinking water data falsification
issues, including states' efforts to detect falsification,
needs improvement. According to Region 5 officials,
reviewing data for falsification was not a high priority
because they: (1) had limited program resources to manage
increasing drinking water program responsibilities, (2) did
not believe data falsification was widespread, and (3) relied
on their states to detect questionable data.
The Region did not closely monitor states' efforts to review
the validity of reported data. Region 5 officials did not
review states' efforts to detect questionable data as part of
their annual reviews of state drinking water programs.
Region 5 also did not conduct any comprehensive falsification
reviews of state-reported PWS data. In addition, Region 5
needs to reemphasize the importance of OGWDWs guidance on
detecting invalid and falsified data to its states.
Because the Region did not emphasize the importance of
reviewing data for falsification, the states also did not
consider it a priority. As a result, cases of potentially
falsified data have gone undetected. According to OGWDWs
guidance, EPA regions and states should try to identify and
prosecute PWSs suspected of falsifying data.
RE COMMENDATIONS
We recommend that the Regional Administrator direct the
Region 5 Water Division to advise states to:
1. provide training to, and/or consider taking
enforcement actions, as appropriate, against the 19
PWSs, or operators, which we found reported invalid
or potentially falsified test data;
2. develop a certification block to be included on
monthly operational reports (MORs) to ensure that
reported data is reviewed for reasonableness; and
iv
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
3. add documented steps to their sanitary survey
procedures to verify the validity of reported data.
These steps should, at a minimum, include a sample
review and comparison of MORs and on-site PWS
records.
We recommend that the Regional Administrator direct the
Region 5 Water Division to:
1. evaluate states' efforts in detecting invalid or
falsified data during Region 5's annual review of
each state's drinking water program;
2. redistribute OGWDW's Appendix G to each of the
states in the Region and reemphasize its
importance; and
3. incorporate techniques for identifying questionable
data into sanitary survey training. This training
should be given to both state and Regional staff.
On July 26, 1994, the Associate Assistant Inspector General
for Internal and Performance Audits wrote the Director of the
Office of Ground Water and Drinking Water recommending that
his office: (1) develop national guidance on sanitary
surveys that incorporates methods to detect invalid and
falsified test data and (2) encourage regions to distribute
future sanitary survey guidance to their states so that
appropriate oversight can be performed. The July 26, 1994,
memorandum is attached as exhibit 2.
On September 29, 1994, the Director of the Office of Ground
Water and Drinking Water responded. In his response, he
stated that his office has included data falsification issues
in sanitary survey training courses. His office also hopes
to either update its training manual for the sanitary survey
course or develop separate sanitary survey guidance in fiscal
year 1995. The September 29, 1994, response is attached as
exhibit 3.
REGION 5 COMMENTS AND ACTIONS
The Region 5 Administrator provided a written response to our
draft report on September 23, 1994. The Administrator agreed
with all recommendations in this report.
v
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
The Region plans to advise the states in writing by October
31, 1994, to take action to address problems at the PWSs we
identified as having reported invalid or potentially
falsified data. Moreover, the Region will also request that
the states (1) develop a certification block to be included
on MORs and (2) add documented steps to their sanitary survey
procedures to verify the validity of reported data.
The Region issued program guidance to the states in April
1994 which requested that the states screen PWS-submitted
data for evidence of data falsification, and take follow-up
enforcement action as appropriate. The states were also
asked to report on the number of falsification cases
identified, and the type of follow-up taken. The Region
plans to evaluate states' efforts in these two areas by
September 30, 1995.
The Region has agreed to redistribute Appendix G to the
states by October 31, 1994. Moreover, the Region will be
holding two sanitary survey training classes, one in October
and one in November 1994, for state officials. The Region
will also request that state officials report on the
usefulness of the training.
Although the Regional Administrator agreed with our
recommendations, he had two remaining concerns with our
report. First, the Administrator did not agree that the
percentage of potentially falsified data the report
identified was significant, given the total amount of test
data reported. Second, the Administrator did not agree that
the examples included in the report of Indiana, Minnesota,
and Illinois were valid examples of a lack of Regional
oversight.
We have included a summary of Region 5's comments in
appropriate sections throughout the report. Appendix 1
contains a copy of Region 5's written response to our draft
report.
PIG EVALUATION
The Region's actions, when completed, will address the
findings and recommendations in this report.
VI
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
TABLE OF CONTENTS
EXECUTIVE SUMMARY i
CHAPTERS
1 INTRODUCTION 1
PURPOSE 1
BACKGROUND 2
SCOPE AND METHODOLOGY 10
PRIOR AUDIT COVERAGE 13
2 PWS OPERATORS REPORTED QUESTIONABLE TEST DATA .... 15
POTENTIAL HEALTH RISKS 15
ABOUT TEN PERCENT OF REGION 5 PWSs
REPORTED TEST DATA THAT APPEARED QUESTIONABLE ... 17
SITE VISITS CONFIRMED
PROBLEMS WITH SOME PWSs 20
CONCLUSION 24
RECOMMENDATION 25
REGION 5 COMMENTS AND ACTIONS 25
OIG EVALUATION 25
3 STATES' EFFORTS TO DETECT
QUESTIONABLE TEST DATA NEED IMPROVEMENT 27
STATES DID NOT HAVE FORMAL
PROCEDURES FOR DETECTING QUESTIONABLE DATA 27
STATES HAVE NOT ESTABLISHED CONTROLS
TO ENSURE VALID TEST DATA ARE REPORTED 29
CURRENT STATE SANITARY SURVEY PROCEDURES
ARE NOT DESIGNED TO IDENTIFY QUESTIONABLE DATA ... 30
CONCLUSION 31
vii
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
RECOMMENDATIONS 32
REGION 5 COMMENTS AND ACTIONS 32
OIG EVALUATION 32
4 REGION 5 OVERSIGHT NEEDS IMPROVEMENT 33
REGION 5 DID NOT CLOSELY MONITOR STATE EFFORTS ... 33
REGION 5 DID NOT REVIEW DATA 34
REGION 5 NEEDS TO REEMPHASIZE OGWDW GUIDANCE .... 34
CONCLUSION 35
RECOMMENDATIONS 35
REGION 5 COMMENTS AND ACTIONS 36
OIG EVALUATION 36
EXHIBITS
EXHIBIT 1 PERCENTAGE OF NONCOMMUNITY
SURFACE WATER SYSTEMS
THAT SUBMITTED QUESTIONABLE DATA 37
EXHIBIT 2 OIG MEMO TO OGWDW DIRECTOR 39
EXHIBIT 3 OGWDW DIRECTOR'S RESPONSE 43
APPENDICES
APPENDIX 1 REGION 5 ADMINISTRATOR'S
RESPONSE TO THE DRAFT REPORT 45
APPENDIX 2 ABBREVIATIONS 50
APPENDIX 3 DISTRIBUTION 51
VI11
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
CHAPTER 1
INTRODUCTION
PURPOSE
In January 1994, U.S. Environmental Protection Agency (EPA)
Office of Ground Water and Drinking Water (OGWDW) officials
requested that the Office of Inspector General (OIG) conduct
this audit: (1) because of their concern that falsified
drinking water data may be going undetected resulting in
potential health threats, and (2) to aid them in planning how
to use limited resources efficiently and effectively. If
water system operators report invalid or falsified test
results, serious health risks could go undetected. Moreover,
if a state does not detect invalid or falsified test data and
investigate the situation, the state's ability to take
proactive measures to prevent contamination of the water
supply is negated.
During preliminary work conducted in three Region 5 states
from October to December 1993, we found several cases of
questionable drinking water test data that the states did not
detect. Moreover, we found few EPA and state resources
committed to ensuring the integrity of reported test data.
Americans drink over one billion glasses of water a day. The
Centers for Disease Control and Prevention (CDC) estimated
that over 940,000 people become ill every year from drinking
contaminated water. Of those, CDC estimated 900 people die
each year. Moreover, controversy has recently been generated
amongst the public regarding the overall quality of our
nation's drinking water supply and the technology of the
systems that provide it.
The objectives of this audit were to determine:
• whether cases of invalid or falsified data have gone
undetected, and
• what procedures Region 5 and its states are using to
detect invalid or falsified data.
1
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
BACKGROUND
Responsibilities of EPA and States
The Safe Drinking Water Act of 1974 (the Act), as amended,
required that EPA ensure that public water systems (PWSs)
monitor and test the quality of drinking water and distribute
water to consumers that is safe to drink. The EPA's OGWDW,
the regions, and the states are all responsible for achieving
the goals of the Act.2 However, over the last decade
statutory monitoring and testing requirements increased.
According to Region 5 officials, limited EPA and state
resources were available for reviewing the validity of
reported drinking water test data.
Under the current safe drinking water program, all states arid
Federal territories, except Wyoming and the District of
Columbia, are responsible for ensuring that the water within
their jurisdiction is safe for consumption. EPA delegates
this authority, called "primacy", to states meeting the Act
and its implementing regulations' requirements. To maintain
primacy, states must adopt regulations at least as stringent
as the Federal Government's and demonstrate that they can
effectively execute program requirements.
In 1991, OGWDW's Enforcement and Program Implementation
Division published its Public Water System Supervision Data
Verification Guidance. Appendix G of this guidance, entitled
"Guidance on Detecting Invalid or Fraudulent Data" presents
ways to identify systems that may be submitting invalid or
falsified data and outlines the correct procedures for
investigating and prosecuting systems suspected of falsifying
data. This guidance states that EPA regions and states with
primacy should identify and prosecute systems suspected of
falsifying data, as part of their oversight of drinking water
programs.
According to the guidance, the problem of data falsification
may be larger than originally suspected. It explains that:
...with the promulgation of new regulations under
the Safe Drinking Water Act, there is an increased
incentive for water systems to purposely submit
invalid compliance data. This is due to the cost
of compliance with the new regulations and an
2The Act applies to all 50 states, the District of Columbia, Puerto
Rico, Indian lands, the Virgin Islands, American Samoa, Guam, the
Commonwealth of the Northern Mariana Islands, and the Republic of Palau.
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
increased complexity in the monitoring and
reporting requirements.
Responsibilities of Public Water Systems
The majority of PWSs must treat the drinking water they
distribute to the public to ensure that the water customers
drink in their homes, places of business, restaurants, etc.,
is safe. Some sources of drinking water contamination
include turbidity (or "cloudiness") of water due to suspended
matter such as clay, silt, and microscopic organisms;
bacteria from human or animal sources; pesticide run-off;
underground injections of hazardous wastes; disinfection by-
products; and lead, copper, and asbestos from corroding
pipes.
This is part of a
settling tank at a small
PWS. At this stage, the
water is at the beginning
of the treatment process.
The brown foamy residue
in the middle section is
a mixture of dirt and
other matter, and
chemical additives to
treat the water.
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
PWSs must test for various contaminants in their water supply
and report those results to their states. Generally, the
larger the population a system serves, the more often that
system is required to test and report. The type of PWS
(community or noncommunity) and the water source (surface or
groundwater) determine what contaminants must be monitored.
The four basic contaminant groups PWSs must test for and
report on are: (1) inorganic chemicals, (2) organic
chemicals, (3) radionuclides, and (4) microbiological
contaminants.3
PWSs generally either contract with an independent certified
laboratory or a state-owned laboratory to analyze water
samples from their systems involving complex tests, such as
those for bacteria, most inorganic and organic chemicals, and
radionuclides. Many states require independent labs to
report the results of their tests directly to the state
before reporting the results to the water system, their
client.
While PWSs generally contract out their complex tests, they
conduct other tests in-house. Water system personnel
conducting these operational tests are normally required to
report the results of these tests to their state once per
month. Water operators submit this data on monthly
operational report (MOR) forms.
At least three measures are important indicators of water
quality: turbidity, chlorine residual, and pH.
• Turbidity measurements indicate how well filtration
systems are removing particulates from drinking water.
As turbidity levels increase, the efficiency of a water
treatment plant in filtering particulates in water
decreases. If turbidity levels are too high, health
risks from microbes increase.
• Chlorine measurements, including the time chlorine is in
contact with water in the distribution system, are very
important also. Although chlorine can kill microbes arid
prevent the regrowth of any bacteria that make it
through the filtration system, microbes may not be
inactivated by chlorine if the time chlorine is in
contact with the microbes is too short.
3OGWDW, in its FY 1993 National Compliance Report, included turbidity
within the microbiological contaminant group.
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Improper pH levels affect the efficiency of
disinfectants and also affect the types and levels of
disinfection by-products generated. In addition, pH
levels that are too acidic increase the leaching of
metals, such as lead, from within the distribution
system into the water supply.
This is a laboratory located at a small PWS. Tests for
turbidity, chlorine residual, and pH are conducted at
this facility.
Many PWS operators are certified to perform operational-type
tests on drinking water samples, including those for
turbidity, chlorine residual, and pH (a measure of the
acidity or alkalinity of a liquid). According to Region 5
officials, while community PWSs usually have certified
operators, smaller systems--particularly noncommunity PWSs--
might not. Each state has an operator certification program,
which ensures that operators are properly trained on PWS
monitoring requirements. According to Regional officials,
state training is available for all operators, certified or
not. The requirements for certified operators vary by state.
These requirements are based on the type and size of the PWS,
its water source and quality, and the population affected.
If operators do not continue to meet these requirements,
their certifications can be revoked. This could result in
the loss of their jobs. Falsification of test data is also
grounds for revocation of an operator's certification.
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Potential for Invalid or Falsified Data
PWS operators are in the business of providing drinking water
to the public. Thus, according to the Region, it is in the
best interest of operators to provide a safe, reliable, and
high quality drinking water product to their customers.
However, the increased cost of new regulations and the
increased complexity in monitoring and reporting requirements
might give some operators an incentive to submit falsified
data.
Because PWS operators and staff are directly involved with
recording and reporting results of operational tests, there
is a greater chance that these results may be falsified.
Operators could report invalid test data because of
malfunctioning equipment, improper testing procedures, or a
misunderstanding of how to properly record and report test
readings. PWS personnel could falsify test data in several
different ways. For example, PWS personnel could: (1)
report fabricated data without taking required tests; (2)
boil or microwave samples thereby killing harmful organisms;
(3) submit samples from sources outside the PWS; or (4)
collect all samples from one location within the PWS.
Types, Sizes, and Number of PWSs
A PWS, as defined by EPA, provides piped water for human
consumption to at least 15 service connections or serves an
average of at least 25 people for at least 60 days each year.
PWSs are classified as either community or noncommunity
systems. Community systems provide water generally to the
same population year-round, such as in residential areas.
Noncommunity systems are either institutions such as schools,
hospitals, or work places that regularly serve at least 25 of
the same people at least 6 months of the year, or
establishments such as campgrounds and motels that supply
water to people traveling through non-residential areas.
OGWDW categorizes the size of PWSs by the number of people
each serves. There are five size categories as shown in
figure 1.
6
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Figure 1: PWS Size Categories
Population Served Size Category
25 - 500 Very Small
501 - 3,300 Small
3,301 - 10,000 Medium
_0,001 - 100,000 Large
.ore than 100,000 Very Large
As of March 1994, there were about 191,000 PWSs in the United
States and its territories. Community systems served over 90
percent of the U.S. population, even though they comprised
only 30 percent of all PWSs. Noncommunity systems, on the
other hand, served only about 8 percent of the U.S.
population, but constituted the remaining 70 percent of PWSs.
PWSs obtain the water that they distribute to the public from
either: (1) groundwater sources--such as springs and
aquifers4, or (2) surface water sources--such as rivers,
lakes, and reservoirs. Both types of water sources are
subject to contamination by various substances. However,
because surface water sources are exposed to the air and are
accessible to human and animal use, these water sources
require more extensive treatment before they can be used for
human consumption.
4An aquifer is an underground geological formation containing usable
amounts of groundwater that can supply wells and springs.
7
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
This shows two chambers
of a water treatment
plant for a small PWS.
The chamber in the
background is where
chemicals are added. In
the foreground is the
settling chamber. The
third, and final, section
is not shown. In that
chamber, the water is
filtered and pumped to a
well.
Both nationwide and in Region 5, surface water systems make
up a small percentage of the PWSs, but serve the majority of
the population. As of March 1994, nationally there were
13,678 surface water systems. While surface systems
comprised only 7 percent of all PWSs, those same systems
served about 57 percent of the U.S. population. Similarly,
in Region 5, only 3 percent of the PWSs were surface systems
(figure 2), yet they served about 58 percent of the
population (figure 3).
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Figure 2: Number of Region 5 Surface Systems
REGION 5: GROUNDWATER VS. SURFACE SYSTEMS
TOTAL NUMBERS
Groundwater
60,058
Surface Water
1,581
TOTAL PWSs = 61,639 (SOURCE:FRDS)
Figure 3: Number Served by Region 5 Surface Systems
REGION 5: GROUNDWATER VS. SURFACE SYSTEMS
TOTAL SERVED BY SOURCE
Groundwater
18.246.OOO
Surface Water
a4.733.OOO
TOTAL SERVED = 42,979,000 (SOURCE: FRDS)
Both nationally and in Region 5, community surface systems
make up the bulk of surface systems. Of the 13,678 surface
systems nationwide, 78 percent are community systems and 22
percent are noncommunity systems. Of the 1,581 Region 5
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
surface systems, 82 percent are community systems and 18
percent are noncommunity systems (figure 4).
Figure 4: Number of Region 5 Community Systems
REGION 5: SURFACE PWSs
TOTAL NUMBERS
Community
1,303
Noncommunity
278
TOTAL SYSTEMS = 1,581 (SOURCE: FRDS)
SCOPE AND METHODOLOGY
Our audit focused primarily on the efforts of Region 5 and
its six states. We interviewed OGWDW officials in
Washington, B.C., and obtained information regarding OGWDW's
budget and data falsification policy. We also performed on-
site audit work which focused on identifying state and Region
5 efforts to detect questionable test results reported by
surface water systems only. Because we only focused on
surface systems, throughout this report "PWS" refers only to
those systems with surface water sources.
We performed audit work, including reviewing PWS files and
interviewing officials, at the following locations:
• Illinois Environmental Protection Agency (Illinois) and
the Illinois Department of Public Health in Springfield;
• Indiana Department of Environmental Management (Indiana)
in Indianapolis;
• Michigan Department of Public Health (Michigan) in
Lansing and Escanaba;
10
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
• Minnesota Department of Health (Minnesota) in
Minneapolis;
• Ohio Environmental Protection Agency (Ohio), district
offices in Columbus, Dayton, Bowling Green, Twinsburg,
and Logan;
• Wisconsin Department of Natural Resources (Wisconsin) in
Madison; and
• EPA Region 5 Water Division in Chicago, IL.
Because OGWDW and Region 5 officials requested that we
concentrate our efforts on community systems, we reviewed
MORs for 100 percent of the community surface water systems
within Region 5 that did not purchase their water from
another system. We also reviewed MORs for all noncommunity
surface systems in Ohio, Indiana, and Michigan. We did not
review MORs for noncommunity surface systems in Illinois,
Minnesota, or Wisconsin because sufficient documentation was
unavailable for our review or, in the case of Wisconsin,
there were no such systems.
We did not review the Federal Reporting Data System (FRDS) to
verify its accuracy. We did, however, check the data against
state lists. Region 5 identified the PWSs in our review
through FRDS. However, if there were discrepancies between
FRDS and state officials' lists, we used the states' lists of
surface PWSs.
We reviewed MORs for 495 surface water systems--452 community
systems and 43 noncommunity systems. We focused our review
on MORs from January 1991 through January 1994. However, if
data for a particular PWS looked questionable for this time
period, we did review data further back than 1991.
While reviewing MORs, we attempted to answer the following
question: How probable was it that the test values reported
by this surface water system appeared valid and were based on
actual tests? We looked specifically for operational test
readings--such as those for turbidity, chlorine residual, and
pH--that did not fluctuate at all or fluctuated very little
over the course of several consecutive months. We also
looked for any other obvious patterns in reported data over
time. We considered such data to be questionable and
suspect, in accordance with OGWDW's Appendix G. We also
looked for additional indicators of potential data
falsification, such as correction fluid on MORs or the use of
pre-printed forms to record results. Because we looked for
11
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
obvious cases of invalid or potentially falsified data, there
may be other cases of such data that we did not identify.
Accompanied by state officials, we conducted unannounced site
visits at 7 of the 51 PWSs we identified as having reported
questionable test data. We judgmentally selected these seven
PWSs taking the size of the systems into consideration.
Three systems were in Ohio, two in Indiana, and two in
Illinois. The PWSs served varying population sizes,- there
was one very small, three small, one medium, one large, and
one very large PWS. During these site visits, we: (1)
interviewed PWS personnel, (2) examined original test
records, (3) inspected the equipment and overall condition of
the facility, and (4) observed water plant operators conduct:
certain water tests we requested.
We obtained information on the potential health effects if
drinking water test data were invalid or falsified by: (1)
interviewing the Associate Director of EPA's Health Effects
Research Laboratory at Research Triangle Park, North
Carolina, and (2) reading recent reports detailing the
importance of turbidity measurements in preventing outbreaks
of disease.
We conducted our fieldwork from January 25, 1994, to June 6,
1994. We provided our position papers to the Region 5
Administrator and the directors of the drinking water
programs of the six states in the Region during June 1994.
Comments to the position papers were included in the draft
report.
On August 12, 1994, we issued our draft report. We held an
exit conference with Region 5 Water Division officials, and
received the Region 5 Administrator's response to the draft
report on September 23, 1994. After reviewing the response,
we finalized the report by making changes, where appropriate.
Region 5's response is included as appendix 1.
We performed our audit in accordance with the Government
Auditing Standards issued by the Comptroller General (1988
Revision). As part of our audit, we also reviewed OGWDW's
and Region 5's 1993 Federal Managers' Financial Integrity Act
reports. Neither of these offices' reports cited data
integrity (potential falsification) as a material weakness.
We did not detect any significant internal control weaknesses
during our audit.
12
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
PRIOR AUDIT COVERAGE
The OIG issued a report on July 30, 1993, which addressed
Region 1's enforcement of the Act, including its data
falsification efforts. According to this report, Region 1
and its states did not routinely conduct data falsification
reviews or follow-up on cases of questionable data that were
identified.5
The U.S. General Accounting Office (GAO) recently issued two
reports related to drinking water quality and data integrity.
According to a June 1990 report, most EPA and state officials
GAO interviewed did not believe data falsification was
extensive. These officials also told GAO that falsifying
test results was relatively easy and incentives for doing so
would increase in the future.6
An April 1993 GAO report focused on the importance of
sanitary surveys as a way to ensure the quality of drinking
water distributed to the public. A sanitary survey is an on-
site review of the water sources, facilities, equipment, and
operation and maintenance of a PWS to evaluate the adequacy
of those elements for producing and distributing safe
drinking water. The report states that: (1) the frequency
of sanitary surveys by the states has declined and that many
states are not conducting surveys as often as EPA recommends,
(2) inadequate training of state inspectors may contribute to
survey flaws, and (3) EPA places limited emphasis on sanitary
surveys.7
5Audit Report of Region I's Enforcement of The Safe Drinking Water Act
(SDWA) (EPA Report No. #3100291).
6Drinking Water: Compliance Problems Undermine EPA Program as New
Challenges Emerge, (GAO/RCED-90-127).
7Drinking Water: Key Quality Assurance Program is Flawed and
Underfunded, (GAO/RCED-93-97).
13
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
[This page intentionally left blank.]
14
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
CHAPTER 2
PWS OPERATORS REPORTED QUESTIONABLE TEST DATA
All PWS operators are required, under the Act and EPA's Safe
Drinking Water regulations8, to ensure that all appropriate
drinking water tests are conducted and test results are
accurately reported. Accurate test data are necessary to
assure that the quality of drinking water provided to the
public meets all drinking water standards.
Files for about 10 percent (51) of the 495 PWSs we reviewed
contained test data that were questionable in accordance with
EPA guidance. Based on further Regional and state
investigations, we determined that operators at about 4
percent of the PWSs we reviewed reported questionable test
data, without their states detecting it. Most of these cases
were at community PWSs.
Undetected questionable test data--whether invalid or
potentially falsified--could (1) result in contamination of
the water supply not being promptly detected and (2) lead to
serious public health problems. PWS operators may have
reported invalid test data because they lacked training on
proper testing and reporting procedures, or had
malfunctioning equipment. Operators may have reported
falsified data to cover-up problems at the PWS or to avoid
additional requirements or corrective actions. We provided a
copy of this report to the OIG Office of Investigations for
its review.
POTENTIAL HEALTH RISKS
If PWS operators report invalid or falsified test results,
serious health threats could go undetected. Moreover, if a
state does not detect and investigate questionable test data,
the state's ability to take proactive measures to prevent
contamination of the water supply is negated.
Drinking water regulations combined with improved water
treatment practices over the past twenty years have
eliminated many waterborne diseases. As a result, water in
the U.S. is considered very safe to drink. However, despite
increased monitoring and testing requirements, contaminated
drinking water still remains a serious health threat. The
40 CFR Part 141.
15
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
CDC estimated that over 940,000 people in the U.S. become ill
every year from drinking contaminated water. Of those, they
estimated 900 people die each year.
These are rapid sand filters at a very large PWS.
The water at this stage has gone through the entire
treatment process, and is ready to be distributed
to the public. The reflections in the water
indicate its clarity.
According to the Associate Director of EPA's Health Effects
Research Laboratory in Research Triangle Park, North
Carolina, testing for the presence of microbes in drinking
water is very important in ensuring harmful microbes that can
cause sickness are not present. Microbes include parasites,
bacteria, and viruses. Such organisms, if ingested, can
result in varying degrees of sickness from mild
gastrointestinal illness to death. He further stated that a
recent outbreak of cryptosporidiosis in Milwaukee, Wisconsin,
shows the significance of turbidity measurements.9 He said
'According to EPA, in 1993, over 370,000 people in the Milwaukee area
were affected by the parasitic microorganism Cryptosporidium in the water
supply. Reports indicated that at least 100 people in the Milwaukee area
died as a result of the parasite. Although the Milwaukee PWS operators did
not report invalid or falsified data, the outbreak highlighted the potential
health effects associated with turbidity violations.
16
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
that the high turbidity measurements that were reported were
good indicators that microbes may have passed through the
filtration system and into the water supply.
ABOUT TEN PERCENT OF REGION 5 PWSs
REPORTED TEST DATA THAT APPEARED QUESTIONABLE
Files for about 10 percent (51) of the 495 PWSs we reviewed
contained test data that was questionable in accordance with
Appendix G. We referred these cases to Region 5 which, in
turn, required the states to follow-up. We accepted PWS data
as valid if state officials (1) visited or contacted the PWSs
and (2) provided reasonable explanations for the data
patterns we identified. As a result, states determined, and
we agreed, that 32 PWSs likely reported accurate test
readings.
Four Percent of PWSs Reported
Invalid or Potentially Falsified Data
Data for the remaining 19 PWSs, however, continued to be
questionable. Of these, operators at 13 PWSs (2.6 percent)
reported invalid data, while operators at 6 PWSs (1.2
percent) reported potentially falsified data. According to
FRDS, these 19 PWSs served about 61,300 people. Small and
very small PWSs reported questionable data most often.
However, medium and large PWSs also reported questionable
data.
Table 1, on the next page, shows details about the 4 percent
(19) of Region 5 PWSs that reported questionable (invalid or
potentially falsified) data. PWSs in all Region 5 states,
except Minnesota, submitted questionable data. The total
percentages among the states ranged from a low of zero in
Minnesota to a high of 7 percent in Indiana.
17
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Table 1
PERCENTAGE OF SURFACE WATER SYSTEMS
THAT SUBMITTED QUESTIONABLE DATA10
STATE/
REGION
IL
IN
MI
MN
OH
WI
R5
TOTAL
TOTAL
SYSTEMS
167
71
77
115
171
20
I11
62212
REVIEWED
129
71
77
26
171
20
1
495
INVALID
3
2
3
0
4
1
0
1312
POTENTIALLY
FALSIFIED
1
3
0
0
2
0
0
613
TOTAL
PERCENT
3.1
7.0
3 .9
0.0
3 .5
5.0
0.0
3.814
10This table, along with table 2 and exhibit 1, should not be
interpreted as a comparison of states' efforts to detect questionable data.
None of the states in Region 5 had procedures to detect or identify "red
flags" which could indicate invalid or fraudulent drinking water test data.
(See chapter 3.)
l:LRegion 5 is directly responsible for oversight of PWSs on Indian
lands. There was only one surface PWS on Indian lands, according to Region
5 officials.
12This total represents all Region 5 surface PWSs which did not
purchase water from another PWS.
13We further examined PWSs which reported data that exhibited obvious
indications of being invalid or potentially falsified. However, we did not
conduct any additional research into PWSs which reported data free of such
indicators. As a result, we were unable to assess the validity of data from
those other PWSs.
14Total percent refers to the percentage of PWSs reviewed which
reported invalid or potentially falsified data. This percentage was derived
by adding the number of PWSs that reported invalid data to the number that
reported potentially falsified data and dividing by the number of PWSs
reviewed.
18
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Community PWSs More Often
Reported Questionable Data
Of the 19 PWSs reporting questionable data, 14 (74 percent)
were community PWSs. The remaining five PWSs (26 percent)
were noncommunity systems.15 As shown below, 9 of the 14
community systems were either very small or small PWSs. The
size category of each of the 14 community PWSs follows:
COMMUNITY PWSs REPORTING
QUESTIONABLE DATA BY SIZE
Size Category
Very Small
Small
Medium
Large
Very Large
Total
Potentially
Invalid Falsified Visited/(States!
2
3
1
2
0
0
4
1
I
0
0
2
0
1
Q_
1
(--)
(IL &
( - - )
(IN)
(--)
OH)
This is a water
treatment facility
at a small PW3.
The blue upright
structure to the
left is a tank in
which solids in
water settle to
the bottom and are
subsequently
removed as sludge.
The building to
the right houses
the PWS's
treatment and
filtering
facilities, as
well as the
laboratory. The
round concrete
structure to the
far right is a
well containing
treated water.
15
'All of the five were very small PWSs. For additional details on
these five, see exhibit 1.
19
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Table 2 shows details about the 14 (3 percent) community
systems which reported questionable (invalid or potentially
falsified) data. Community PWSs in all Region 5 states,
except Minnesota, submitted questionable data. Percentages
ranged from a low of about 1 percent in Michigan to a high of
7 percent in Indiana.
Table 2
PERCENTAGE OF COMMUNITY SURFACE WATER SYSTEMS THAT
SUBMITTED QUESTIONABLE DATA
STATE/
REGION
IL
IN
MI
MN
OH
WI
R5
TOTAL
TOTAL
SYSTEMS
129
57
70
26
149
20
1
452
REVIEWED
129
57
70
26
149
20
1
452
INVALID
3
1
1
0
2
1
0
8
POTENTIALLY
FALSIFIED
1
3
0
0
2
0
0
6
TOTAL
PERCENT
3.1
7.0
1.4
0.0
2.7
5.0
0.0
3. 116
SITE VISITS CONFIRMED
PROBLEMS WITH SOME PWSs
We visited 7 of the 51 PWSs that we identified as having
reported questionable data. Four of the seven PWSs seemed to
be conducting and reporting test results accurately. At the
remaining three, however, we found indicators that data might
have been falsified (43 percent). Of these three PWSs, one
each was in Indiana, Illinois, and Ohio. Operators at these
PWSs: (1) had recorded test readings before tests were
taken, (2) lacked documentation regarding tests taken, or (3)
disregarded results from continuous monitoring meters. Our
site visits also disclosed weaknesses in these three states'
oversight of data review and reporting. In all cases, a
state inspector accompanied us and verified our observations.
16Total percent refers to the percentage of PWSs reviewed which
reported invalid or potentially falsified data. This percentage was derived
by adding the number of PWSs that reported invalid data to the number that
reported potentially falsified data and dividing by the number of PWSs
reviewed.
20
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Some Site Visits
Showed Accurate Testing
Four of the seven PWSs we visited appeared to be operating
satisfactorily. Of these four PWSs, two were small PWSs, one
was medium-sized, and one was a very large PWS. Our
inspections of these PWSs' facilities, equipment, and records
showed that the reported data we initially questioned was
likely to be valid.
For example, one PWS had reported data to its state on an MOR
and had used correction fluid to cover high turbidity
readings originally recorded. Lower (better) readings were
then written on top of the correction fluid. As a result of
reviewing its on-site records and talking with officials at
the water plant, we determined that the correction fluid was
used to rectify errors made when transcribing the readings
from the bench sheets to the MORs.
All four PWSs conducted several tests we requested. The test
results they obtained were consistent with data they had
reported in the past.
An Indiana PWS Operator
Reported Potentially Falsified Data
It is likely that an operator at a large Indiana PWS reported
test data for several years for tests that were not taken.
The operator signed and sent MORs to Indiana which showed
consistent daily readings for several different tests since
1987. Our site visit showed that: (1) data for turbidity
and pH tests had probably been falsified and (2) Indiana's
oversight and review of reported data needed improvement.
Turbidity Test Reveals Inconsistencies
The turbidity reading the operator obtained during our visit
was inconsistent with readings he had reported for the last
seven years. The test reading he obtained was not close to
the readings of 0.25 and 0.30 that he had consistently
reported since 1987. The turbidity reading for finished
(treated) water from a faucet located within the plant was
2.4 nephelometric turbidity units (NTU). The current Federal
maximum limit for surface PWSs using conventional or direct
filtration systems is 0.5 NTUs.17 The operator had already
17Under the Surface Water Treatment Rule, a PWS using conventional or
direct filtration is in violation of the turbidity standard if. more than 5
percent of its test readings for the month are over 0.5 NTU.
21
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
recorded several turbidity readings on the MOR form for tests
taken earlier that day. These test readings were recorded at
0.25.
The continuous monitoring turbidimeter was not operating
during our visit, and documentation of required tests was not
maintained. A continuous monitoring device typically should
be easier to obtain readings from because tests do not have
to be performed manually. However, the operator instead used
a small table-top turbidimeter to conduct the test.
According to the operator, he did not use the PWS's
continuous monitoring device to record turbidity levels
because it is not practical to use. In addition, there were
no lab sheets, logs, or backup documentation for the
turbidity test results that are required to be taken.
This continuous
monitoring device
digitally measures
chlorine levels at a
water treatment plant.
22
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Tests For pH Also Reveal Problems
The operator also may have falsified pH test results. The pH
readings the operator obtained during our visit were
inconsistent with readings he reported for the last seven
years. He consistently reported a pH of 7.6 for the system's
finished (treated) water and a pH of 7.3 for its raw
(untreated) water since January 1987. The state inspector
reviewed the MORs during our visit and also questioned how a
surface water system could always have the same pH.
During the test, the electronic pH testing machine appeared
to be operating improperly because it was not providing one
definitive digital reading. According to the operator, the
machine had not been calibrated since he obtained it in 1984.
According to the state inspector, the machine should be
calibrated at least once a year. As a result, neither
Indiana officials nor the operator would have been able to be
sure that test data reported for the last six years were
accurate.
Using a different test method to verify results obtained from
the electronic testing machine, we found the results were not
consistent with (1) what the operator had reported since 1987
or (2) the results for tests documented as taken just hours
before we arrived. In addition, we also saw other test
readings the operator recorded for times that had not yet
elapsed on the day of our visit.
Problems With State Oversight
According to the Indiana field inspector, he had not reviewed
MORs for about a year. He stated that PWSs send MORs
directly to Indiana headquarters, and Indiana has
deemphasized the importance of field inspectors reviewing
MORs. According to Indiana's Public Water Supply Compliance
Section Chief, Indiana did not have enough field inspectors
to review MORs to determine if PWSs were falsifying data. He
stated that field inspectors examine MORs when operational or
maintenance problems are brought to their attention or when
they are asked to provide special technical assistance to
PWSs. As a good management practice, field inspectors,
responsible for troubleshooting problems at PWSs, should
review MORs on a regular basis so that potential problems can
be detected as soon as possible and corrected.
23
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
An Illinois PWS Operator Had No
Documentation To Show Tests Were Taken
During a site visit at a small Illinois PWS, we identified
several problems. The operator kept no turbidity bench
sheets or logs and, therefore, had no documentation of tests
taken. The operator reported the same turbidity reading of
0.2 nearly every day from April 1988 through December 1993.
Illinois officials stated that they were aware of the
repetitious reporting pattern and agreed that it was highly
improbable that the readings reported could have been true
for that long of a period. State officials, however, told us
that they had never taken any action to investigate or
correct the situation because they "only had indications of
falsification and no proof that would stand up at a [court]
hearing."
In addition, although this Illinois PWS's continuous in-line
turbidity meter was operating during our visit, according to
the operator, it was not functioning properly. He therefore
disregarded its results. The in-line meter reading was 0.12.
The operator used the table-top turbidimeter to take
turbidity readings. The turbidity standards test kit the
operator used to standardize the machine before taking tests,
however, was a year past its expiration date.
An Ohio PWS Had Data Reporting Problems
At a small PWS in Ohio, we noted the following: (1) at 10:30
a.m. that day's turbidity reading for 12:00 p.m. was already
recorded; (2) several turbidity readings on the PWS's bench
sheets for prior months did not match what was reported to
Ohio; and (3) the operator reported turbidity readings to
Ohio for February 29 and 30, 1993. The operator could not
adequately explain the above. In addition, on-site bench
sheets for the month of February 1993 did not show that any
tests were taken for those dates.
The Ohio inspector who accompanied us during the visit stated
that he does not review bench sheets when he performs
sanitary surveys of PWSs.
CONCLUSION
Our review of MORs showed that about 10 percent of the PWSs
reported test data that was questionable in accordance with
EPA guidance. Moreover, nearly 4 percent of the PWSs
actually reported invalid or potentially falsified test
24
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
results. We believe that percentage is significant because
the majority of PWSs we identified as reporting questionable
data were community systems. However, most of the cases
involved invalid data rather than potentially falsified data.
The Region should take necessary corrective actions, such as
recommending that its states provide training to those
operators in need or take enforcement actions against those
who intentionally falsified data.
RE COMMENDATION
We recommend that the Regional Administrator direct the
Region 5 Water Division to request that the states provide
training to, and/or consider taking enforcement actions, as
appropriate, against the 19 PWSs, or operators, which we
found reported invalid or potentially falsified test data.
REGION 5 COMMENTS AND ACTIONS
The Regional Administrator agreed with the recommendation.
He stated that the Region would, by October 31, 1994, send a
letter to the states requesting that they follow through on
the above recommendation.
PIG EVALUATION
The Region's actions, when completed, will address the
recommendation in this chapter.
25
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
[This page intentionally left blank.]
26
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
CHAPTER 3
STATES' EFFORTS TO DETECT
QUESTIONABLE TEST DATA NEED IMPROVEMENT
State officials placed low priority on reviewing records for
invalid or falsified data primarily because they did not
believe that falsification was widespread. State officials
generally trusted that data reported to them was valid, and
seemed to be unaware of the potential for data falsification.
In addition, according to officials in some states, limited
resources do not allow for any detailed reviews. As a
result, invalid or potentially falsified data has gone
undetected. In accordance with criteria in 40 CFR Part 142,
primacy states are responsible for administering and
enforcing drinking water regulations and related
requirements, including general review and oversight of PWS-
reported test data. In addition, according to OGWDW
guidance, states should try to identify and prosecute PWSs
suspected of falsifying data. However, according to state
officials, past experience has shown that developing
successful prosecutions of data falsification cases is very
time consuming and difficult.
STATES DID NOT HAVE FORMAL
PROCEDURES FOR DETECTING QUESTIONABLE DATA
None of the states in Region 5 had any formal, documented
procedures for reviewing MORs for invalid or falsified data.
According to officials in some states, engineers or field
inspectors reviewed the data on MORs when the forms were sent
in. However, in most cases, officials did not review data
with the idea that it could be invalid or falsified.
According to state officials they: (1) generally did not
believe data falsification occurs, and (2) had limited
resources to review reported data even if falsification were
occurring.
State officials were accustomed to scanning reported data for
maximum contaminant level violations; however, they did not
examine the data with the idea that violations might go
unreported because data could be invalid or falsified. State
officials generally agreed that unusual or repetitive
patterns in reported data should be questioned. However, the
reporting process is a self-monitoring process and they must
rely on PWS operators to report valid test results.
27
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Although some state drinking water officials said that state
personnel review MORs each month, these reviews were not
documented. None of the states in Region 5 required their
personnel to certify that the reported test data seemed
reasonable. We discussed with state and Region 5 officials
how to best address this issue. According to these
officials, a potential solution could be including a
certification block on MORs. By requiring state reviewers to
certify that they reviewed the data for reasonableness, they
would be more accountable and the states would have more
assurance that the data were valid. Moreover, by knowing
that state officials will review reported data, PWS operators
might be deterred from reporting falsified data.
The state officials we interviewed were not aware that
Federal guidance (OGWDW's Appendix G) existed which (1)
detailed ways to detect falsified data and (2) discussed the
importance of reviewing data for falsification. Moreover,
they had not received any training on how to detect falsified
data or how to recognize basic fraud indicators. As a
result, state officials were generally unaware of the
potential for data falsification.
Some States Have Found And
Acted On Falsification Cases
Although states lacked procedures to detect falsified data,
some states have detected cases of falsified data during
their normal work activities. For example, Ohio officials
cited a case in their southeast district in which a PWS (a
restaurant) submitted falsified bacteriological test results.
The restaurant owner did not have the water supply tested for
bacteria, as required, but instead physically altered another
test report form. Using correction fluid, the restaurant
owner changed the readings on the form to make it look like
tests were conducted on schedule. When she sent that form to
Ohio, a staff person noticed the substantial amount of
correction fluid on the document and researched this PWS's
prior test reports. The Ohio staff person discovered that
the form was simply an altered photocopy of a test report
from a different month.
Ohio officials also noted that they have revoked operators'
licenses five times since 1987 for falsification of drinking
water data. However, they stated that prosecuting data
falsification cases is very time consuming and resource
intensive. For example, Ohio prosecuted one PWS operator for
falsifying turbidity readings from 1984 to 1986. In 1989,
the Ohio Attorney General's Office filed misdemeanor charges
28
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
with the county prosecutor against the operator. The
operator plead guilty to misdemeanor charges of falsification
in 1990 and was fined $50 and $33 in court costs. The
operator's license was revoked in 1992.
STATES HAVE NOT ESTABLISHED CONTROLS
TO ENSURE VALID TEST DATA ARE REPORTED
State reporting requirements varied regarding the test data
PWSs were required to submit to comply with the Surface Water
Treatment Rule (the Rule). Some states, while allowing PWSs
to report the fewer data required by the Rule, have not
developed internal controls to verify that PWSs took tests
and reported valid results. Two states did not require some
classes of PWSs to report test data at all. As a result,
states are less able to detect falsified reporting because
they have minimal or no test data available to them for
evaluation.
Some states allow surface PWSs to report only a minimal
number of test readings to comply with the Rule. For
example, Illinois requires community PWSs to report actual
readings, by date, only if any exceed 5.0 NTUs. Otherwise
the community PWSs just report statistics on the number of
tests conducted and the number less than or equal to 0.5
NTUs. Although this data satisfies Federal requirements,
Illinois has no control procedures in place to assess the
validity of the data that is reported. Ohio, on the other
hand, requires PWSs to report daily readings for turbidity,
chlorine, pH, and other tests related to the treatment of
surface water. As a result, Ohio officials have more data
available to them to make judgements regarding data
integrity.
Neither Illinois nor Minnesota required noncommunity PWSs to
report test data on a regular basis, even though those PWSs
were required to test for contaminants regularly. As stated
in chapter 2, we were unable to adequately review data for
noncommunity PWSs in Illinois and Minnesota because the
reports were not sent to those states. Although the Illinois
Department of Public Health's regulations require
noncommunity PWSs to complete and send in MORs, the
requirement was not enforced. According to the Department's
management, they decided not to require monthly reporting
because they believed that PWS inspections were sufficient to
29
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
identify potential problems.18 Minnesota officials stated
that noncommunity PWSs have not been required to report
turbidity readings because the PWSs have been using an
alternative treatment technology and the Rule does not cite
turbidity requirements for PWSs using the alternative
treatment. As a result, Minnesota officials said they have
not established "meaningful" turbidity limits for these PWSs,
and will, therefore, not require turbidity monitoring or
reporting until December 1995.
The Rule, which became effective in July 1993, eased the
reporting requirements placed on PWSs and eliminated the
requirement to report daily turbidity readings. Instead, to
comply with the Rule, PWSs only have to report statistics
related to the number of tests conducted and any readings
exceeding specified limits. According to Region 5 officials,
the reporting burden was eased because the actual monitoring
and testing requirements under the Rule were increased.
Prior to July 1993, PWSs were required to report daily
turbidity readings each month; however, PWSs did not have to
conduct as many turbidity tests as they do now.
CURRENT STATE SANITARY SURVEY PROCEDURES
ARE NOT DESIGNED TO IDENTIFY QUESTIONABLE DATA
According to OGWDW's Appendix G, "A sanitary survey is likely
our [EPA and states] best vehicle for detecting instances of
data falsification." However, states did not have specific
procedures to identify falsified data while conducting
sanitary surveys. In addition, the frequency with which
states conducted sanitary surveys varied greatly.
Current state sanitary survey procedures did not include
reviews of test data to identify potential falsification.
State officials did not review monthly reports for data
falsification prior to conducting sanitary surveys.
Moreover, they did not review on-site PWS records during
surveys to verify the validity of reported data. For
example, although Wisconsin had detailed, documented sanitary
survey procedures, these procedures did not address data
falsification. Based on our audit, we believe reviewing MORs
and on-site records would enable state officials to focus on
suspect areas during sanitary surveys. In addition, state
field inspectors should analyze water samples during the
18Through a Memorandum of Agreement, the Illinois Department of Public
Health is responsible for oversight of noncommunity PWSs in the state.
30
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
surveys to determine if readings are similar to historical
data (or data recorded earlier in the day). Operators should
also be made to conduct tests to demonstrate their competency
to accurately test samples.
According to current Federal requirements in 40 CFR 141.21,
states must conduct a sanitary survey once every five years
at each surface PWS that does not collect five or more
bacteriological samples each month.19 However, EPA has
issued guidance recommending that states conduct
comprehensive sanitary surveys of PWSs at least once every
three years. We found that the frequency with which the six
Region 5 states conducted sanitary surveys varied. For
example, according to state officials, Ohio and Michigan
conducted surveys of surface PWSs annually. Indiana and
Wisconsin conducted sanitary surveys of surface PWSs once
every 5 years. Although states had to perform surveys only
once every 5 years to meet the minimum Federal requirement,
infrequent sanitary surveys may not provide effective
protection against data falsification.
CONCLUSION
State officials need to be made more aware that data
falsification does, in some cases, occur and that they need
to improve their capabilities for detecting questionable
data. State officials told us that they rely on PWS
operators to report valid test results and, therefore, do not
believe they need to review reported data for potential
falsification. Our audit has shown, however, that officials
have too often assumed that the data reported to them was
valid. Even though the data reporting process is based on
self-monitoring, states should not presume that oversight or
review of the data for reasonableness is unnecessary.
States should have specific procedures in place for reviewing
reported test data for reasonableness. State officials
should be held accountable for reviewing reported data. If
officials are not held accountable for reviewing data, the
likelihood that invalid and falsified test results would go
undetected would increase. Examining water system records
during sanitary surveys is one of the best ways for states to
19OGWDW and state officials met in June 1994 to discuss the need for
national guidance on conducting sanitary surveys. Their meeting included a.
discussion on how often states should conduct sanitary surveys. According
to an OGWDW official, future guidance on sanitary surveys will likely be
issued in 1995.
31
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
verify that previously reported data were based on actual
tests.
RE COMMENDATIONS
We recommend that the Regional Administrator direct the
Region 5 Water Division to advise states to:
1. develop a certification block to be included on
MORs to ensure that reported data is reviewed for
reasonableness; and,
2. add documented steps to their sanitary survey
procedures to verify the validity of reported data,
These steps should, at a minimum, include a sample
review and comparison of MORs and on-site PWS
records.
REGION 5 COMMENTS AND ACTIONS
The Regional Administrator agreed with the above
recommendations. He stated that the Region would, by October
31, 1994, send a letter to the states requesting that they
follow through on the above recommendations.
PIG EVALUATION
The Region's actions, when completed, will address the
recommendations in this chapter.
32
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
CHAPTER 4
REGION 5 OVERSIGHT NEEDS IMPROVEMENT
Region 5's oversight of drinking water data falsification
issues, including states' efforts to detect falsification,
needs improvement. According to Region 5 officials,
reviewing data for falsification was not a high priority
because they: (1) had limited program resources to manage
increasing drinking water program responsibilities, (2) did
not believe data falsification was widespread, and (3) relied
on their states to detect questionable data.
The Region did not closely monitor states' efforts to review
the validity of reported data. Region 5 officials did not
review states' efforts to detect questionable data as part of
their annual reviews of state drinking water programs.
Region 5 also did not conduct any comprehensive falsification
reviews of state-reported PWS data. In addition, Region 5
needs to reemphasize the importance of OGWDWs guidance on
detecting invalid and falsified data to its states.
Because the Region did not emphasize the importance of
reviewing data for falsification, the states also did not
consider it a priority. As a result, cases of potentially
falsified data have gone undetected. According to OGWDWs
Appendix G, EPA regions and states should try to identify and
prosecute PWSs suspected of falsifying data.
REGION 5 DID NOT CLOSELY MONITOR STATE EFFORTS
Most Federal environmental statutes, including the Safe
Drinking Water Act, embrace the concept that states should
have primary responsibility for operating regulatory and
enforcement programs. EPA and the states have recently
issued a policy statement describing a new framework for
their partnership.20 According to the statement, an
effective relationship between EPA and the states depends on
mutual dedication to shared responsibility and accountability
for implementing environmental programs. While recognizing
this concept, the statement continues to call for EPA to
perform its mandated statutory mission, including
constructive program review.
"Joint Policy Statement on State/EPA Relations. State/EPA Capacity
Steering Committee, July 14, 1994.
33
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
Region 5 officials did not closely monitor states' efforts to
detect invalid or falsified data. Moreover, Region 5
officials did not evaluate states' efforts in detecting such
data during their annual review of each state's drinking
water program. Regional officials were also not aware that
Illinois and Minnesota had not required noncommunity PWSs to
regularly report operational test data. Regional officials
also erroneously told us that Indiana checked for obvious
signs of data falsification, such as eraser marks and
correction fluid on MORs. However, our review of Indiana PWS
files, combined with Indiana officials' statements, showed
that Indiana did not review incoming drinking water test data
for falsification.
REGION 5 DID NOT REVIEW DATA
Region 5 did not conduct any detailed efforts to review
state-reported PWS data for falsification. The Region
performed a review once every two years to verify that the
states accurately reported, and input into FRDS, all of the
PWS-reported maximum contaminant level violations. According
to Region 5 officials, during these verification reviews,
they scanned the data for cases of potential falsification.
However, Region 5's documented verification review procedures
did not include specific methods for detecting potential data
falsification. Verification reviews are conducted to ensure
that reported violations have been entered into FRDS. Such
reviews did not, however, allow officials to determine
whether a PWS reported all potential violations to its state.
As a result, violations might have gone unreported and
undetected. According to Region 5 officials, they had not
identified cases of potential data falsification during
verification reviews.
The Region is specifically responsible for reviewing test
data that Indian land PWSs submit directly to it. Although
the Region was responsible for direct oversight of one Indian
land surface PWS in the Region, many of the MORs for that
system were not in the file. According to Regional
officials, their monitoring of this PWS's routine reporting
activity needs improvement.
REGION 5 NEEDS TO REEMPHASIZE OGWDW GUIDANCE
According to Region 5 officials, in April 1991 they sent a
copy of OGWDW's Appendix G to each of their states, and
discussed data falsification at their annual meeting with
34
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
state drinking water officials. However, Region officials
had no documentation to show that they provided the guidance
to the states. Moreover, as stated in chapter 3, the state
officials we interviewed were not aware that Federal guidance
existed addressing data falsification. OGWDW advised the
regions to work with their states to identify and prosecute
PWSs suspected of falsifying data. According to Region 5
officials, they also have not conducted any training for
Regional staff on how to detect potentially falsified data
using the guidance in Appendix G.
CONCLUSION
Regional officials agreed that they placed low priority on
detecting invalid or falsified data. However, they believed
their efforts in this area were adequate given their limited
resources and increasing drinking water program
responsibilities. Region 5 officials did not believe that
data falsification occurred often enough to warrant any
formal reviews.
Region 5 oversight of states' efforts to detect falsified
data needs improvement. We believe the results of our review
were significant and showed that improvements in oversight
can be made. As a result, the Region should encourage states
to examine data for reasonableness during sanitary surveys.
In addition, Region 5 could not rely on its states to detect
invalid or falsified data because states' placed low priority
on this activity. Without increased Regional awareness and
oversight of states' activities in this area, the states
cannot be expected to emphasize this issue. The Region
should be aware of what each state is doing to detect or
prevent falsified data. The Region should also, as a good
management practice, continue to distribute relevant guidance
to its states to heighten awareness of the potential for data
falsification and to aid them in detecting invalid and
falsified data.
RE COMMENDATIONS
We recommend that the Regional Administrator direct the
Region 5 Water Division to:
1. evaluate states' efforts in detecting invalid or
falsified data during Region 5's annual review of
each state's drinking water program;
35
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
2. redistribute OGWDW s Appendix G to each of the
states in the Region and reemphasize its
importance; and,
3. incorporate techniques for identifying questionable
data into sanitary survey training. This training
should be given to both state and Regional staff.
REGION 5 COMMENTS AND ACTIONS
The Regional Administrator agreed with the above
recommendations. Regarding the first recommendation, Region
5 Water Division officials plan to evaluate the states'
efforts to detect invalid or falsified data by September 30,
1995. The Water Division issued its fiscal year 1995 program
grant guidance to the states in April 1994. This guidance
included two activities. First, Water Division officials
requested that the states screen data submitted by PWSs for
evidence of data falsification, and take follow-up
enforcement action as appropriate. Second, Water Division
officials asked the states to report on the number of
falsification cases identified and the type of follow-up
action taken. According to the Regional Administrator,
although these activities were not in the fiscal years 1993
and 1994 guidance to the states, Water Division officials put
them back in for fiscal year 1995, in response to the OIG's
audit findings. The Regional Administrator stated that the
states have included these two activities in their program
plans for 1995.
Regarding the second recommendation, the Region agreed to
redistribute Appendix G to the states by October 31, 1994.
To address the third recommendation, Region 5 Water Division
officials have worked with OGWDW to add data falsification
review techniques to the next round of sanitary survey
training classes. Region 5 plans to hold two sanitary survey
training classes for state officials, one in October and one
in November 1994, within the region. Regional officials plan
to ask the states to report on the usefulness of the data
falsification training in their annual self-evaluation of
progress on grant commitments.
PIG EVALUATION
The Region's actions, when completed, will address the
recommendations in this chapter.
36
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
EXHIBIT 1
Page 1 of 1
PERCENTAGE OF NONCOMMUNITY SURFACE WATER SYSTEMS
THAT SUBMITTED QUESTIONABLE DATA
STATE/
REGION
IL
IN
MI
MN
OH
WI
R5
TOTAL
TOTAL
SYSTEMS
38
14
7
89
22
0
0
170
REVIEWED^
Q21
14
7
Q21
22
N/A
N/A
43
INVALID
N/A
1
2
0
2
N/A
N/A
5
POTENTIALLY
FALSIFIED
N/A
0
0
N/A
0
N/A
N/A
0
TOTAL
PERCENT
N/A
7.1
28.6
N/A
9.1
N/A
N/A
11. 622
This exhibit shows that over 11 percent of the noncommunity
PWSs we reviewed reported questionable (invalid or
potentially falsified) data. The total percentages among the
three states with questionable data ranged from a low of
about 7 percent in Indiana to a high of about 29 percent in
Michigan.
21A total of 38 noncommunity surface system files in Illinois and 89
in Minnesota were not reviewed because sufficient documentation was not
available. Minnesota did not require noncommunity surface systems to report
test data on a monthly basis, as it did for community systems. Illinois did
not enforce its requirement that noncommunity surface systems report on a
monthly basis.
22Total percent refers to the percentage of PWSs reviewed which
reported invalid or potentially falsified data. This percentage was derived
by adding the number of PWSs that reported invalid data to the number that
reported potentially falsified data and dividing by the number of PWSs
reviewed.
37
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
[This page intentionally left blank.]
38
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
EXHIBIT 2
Page 1 of 3
)
UWTtD STATES CNVWIOHmKNTAl s«OT|CTiON AGCNCY
WASHMGTON DC 10400
JU.26
SUBJECT: Procedures to Enure Drinking Water Data Integrity
fROJt: jL» Michael Simmons TjCtcXcA-'H- Aya-£^~
/T Associate Assistant Inspector General
I for Internal and Performance Audits
TO: James R. Elder
Director, Office of Ground Water
and Drinking Water
we are writing to bring to your attention aa issue that wo
noted during the course of an audit conducted by our Northern
Audit Division. The Deputy Director of the Office of Ground
Water and Drinking Water (OCWTJW) requested that we conduct work
to identify how frequently public water system* report invalid or
fraudulent drinking water test data. This information would aid
OGHDW in deciding whether to shift limited resources toward more
oversight and review of reported data. We are drafting a report
to the Regional Administrator, Region S, to present the results
of this work which focused primarily on regional water systems'
data. Although our audit focused en EPA Region S and itm six
states, one of the issues wo identified applios to all regions
and, wo believe, requires management attention before our
September 1M4 final report, me determined that neither the
regions nor states take much action to ensure drinking watar data
integrity, however, wo have identified aa opportunity to add a
measure of protection to the data walla using few program
resources.
It has com* to oar attention that therm is aa ongoing study
group—consisting of OCWDW and stats officials—addressing
recommendations made in a O.t. General accounting Office report
regarding sanitary surveys. The troop Is mooting to develop EPA
guidance on. among other things, what steps should bo performed
when conducting sanitary surveys and how often. We believe the
study group should bo mad* aware of oar findings in Region 5 and
consider this Information whoa developing future guidance for all
regie
Our audit objectives were to: (1) determine whether surface
water systems submitted questionable drinking water test data and
39
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
EXHIBIT 2
Page 2 of 3
whether statss had detected such data; (2) detanlne what
procedures legion 5 and its states used to detect questionable
data; and (!) identify the potential health iapactf of not
detecting invalid or fraudulent data. He used OCVDU's Xpo«rv
-------
DRINKING WATER DATA INTEGRITY
EXHIBIT 2
Page 3 of 3
water system that does net collect five or more bacteriological
samples each month. However, KPA issued guidance In 1*17
recommending that state* conduct comprehensive eanitary surveys
of public water systems at least once every three years.
We also found that many state official* were unaware that
c existed. some regions had not distributed copies of
this OCWDW evidence to their states. As a result, this guidance
has not been as effective as It could be.
He recommend that you direct the study group to develop
national guidance on sanitary surveys that incorporates and
improves upon the information related to data falsification
contained in Appandi» e. Examining water system records during
sanitary surveys is one of the best ways for states to verify
that previously reported data were valid and based on actual
tests. In addition, regions should be encouraged to distribute
future sanitary survey guidance to their states so that
appropriate oversight can be maintained.
Please advise us of the study group's progress la this
matter, *e would welcome the opportunity to comment en any draft
product they issue and request that we receive a copy of the
final guidance when published. Ve are available to discuss our
audit In more detail with you, your staff, or the study group am
needed. If you have any concerns or questions, pleas* contact
Klmberly O'Lone, Audit Manager of our northern Audit Division at
(312) MC-im.
cc: Peter Cook (4101)
Shelly Metsenbaum (1901)
Valdas Adamkus (»-l»J)
Dale sryson (W-ISJ)
Judy lecbt (4103)
41
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
[This page intentionally left blank.]
42
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
EXHIBIT 3
Page 1 of 2
UNTIED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20480
SEP 2 9 1994
MmOftAMDtm
WATER
SUBJECT: Procedures to Ensure Drinking,water pata ^ntegrity
FROM: James R. Elder, Director
office of Ground Water a:
TO: Michael Simoon*
Associate Assistant Inspector General
for internal and Performance Audits
This is in response to your July 26, 1994 memorandum
concerning an audit undertaken by your Northern Audit Division on
drinking water data integrity. The audit focused on our Region v
office and its six States. Our Region V office is preparing a
response to the draft report. The purpose of this memorandum is to
respond to your recommendation to have our sanitary survey study
group consider the findings of the audit.
In short, you reviewed 495 drinking water systems in Region V
and found 19 systems with questionable data that bad gone
undetected by States. You also found that states' sanitary survey
procedures were not designed to identify questionable data and you
recommend that the study group develop national sanitary survey
guidance that incorporates and improves on the information related
to data falsification contained in Appendix G: Guidance on
Detecting Invalid or Fraudulent Data.
We did establish a group of Regional and State staff to
consider the recommendations made by the General Accounting Office
(GAO) in their report on the sanitary survey program.
Unfortunately, we held a one time meeting of this group on June 7,
1994, before we received your memorandum suggesting that we Include
your audit's findings on the agenda. The group, therefore, did not
discuss your findings and recommendations.
We have, however, taken some action to implement your
recommendation and we may, depending on resources, take additional
steps in FY 95. We agree that, although there were only a small
percentage of cases with questionable data, the most appropriate
tool we have for identifying these situations is the sanitary
survey, we have included data falsification in the sanitary survey
training courses, including those scheduled in Region V this fall.
We are also hoping to either update our training manual for the
sanitary survey course or develop separate sanitary survey guidance
in FY 95. in either case, we will include data falsification as a
specific topic.
43
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
EXHIBIT 3
Page 2 of 2
Our efforts will be focused on encouraging States to look for
casts with questionable data as part of their ongoing field
surveillance efforts. With the critical resource demands that are
placed on States for maintaining primacy, I do not envision asking
States to implement a separate initiative in this area.
Thank you for your thoughtful insights and recommendations for
improving the data integrity of the drinking water program. Zf you
wish to discuss this further, please call ae or have your staff
contact Peter Cook at (202) 260-5508.
44
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
APPENDIX 1
Page 1 of 5
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
REGION 5
"' "*"'" 77 WEST JACKSON BOULEVARD
CHICAGO. IL 60604-3590
SEP 22 ISM
REPLY TO THE ATTENTION OF
R-19J
MBCRHPCM
SOBJBCT: Region 5 Response to the Office of Inspector General (OIG)
Draft Report on Region 5 Procedures to Ehsure Drinking
Water Data Integrity
PRCM: Valdas V. Adamkus
Regional Administrator
TO: Anthony C. Carrollo, Divisional Inspector General
for Audits, Northern Division
thank you for the opportunity to respond to the draft report on Region 5
Procedures to Ehsure Drinking Water Data Integrity. Most of the Region's
issues with the draft report were resolved through intense negotiations
between our staffs, and we appreciate the responsiveness of the OIG to the
Region's concerns, the Region's remaining ccnrents are attached, along with
our formal response to each of the OIG recamendations.
If you have any questions concerning these comments, please call Charlene
J. Denys, Chief, Drinking Water Section, at (312) 886-6206, or Christiane
Saada-Blums of her staff, at 886-6205.
Valdas V.
Attachment
45
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
APPENDIX 1
Page 2 of 5
KEQXCN 5 EKLUflJUKKS TO BEURE UilhRIND HKXSt IMA
RBGZCN 5 KBSPCKSB ID THE KKPCRT PHCM TW
OFFICE OF THE DEFECTOR GBORAXi (OIO)
Listed below are the Region's remaining concerns with the DIG report, as well
as the Region's response to each reconmendation.
CCFMENTS CK ly KKKKT
1. The OIG's findings of 1.2 percent potentially falsified data shows that
there is not a large data falsification problem in Region 5. However,
the report characterizes data falsification as a significant problem,
requiring increased Regional monitoring of the States' efforts to detect
falsification, the Region disagrees with this characterization of the
problem. Much of data that is reported to the State cones directly from
the laboratories conducting the analyses, which can be up to 83
contaminants, depending on the type of system. This further reduces the
opportunities for systems to alter the data. The OIG focused only on
data collected and reported by the systems (turbidity and chlorine
residual), and this is a small subset of the data the systems must
report. While we believe that the percentage should be zero, when the
percentage of potentially falsified data is taken in context of the
total amount of data reported, the percentage of potentially falsified
data is very small, and is not significant.
The findings of 1.2 percent potential data falsification do not warrant
a substantial increase in Regional or State resources to identify data
falsification cases. In addition, the results do not support a shift in
Regional or State Public Water System Supervision (PWSS) pzujidin
priorities.
The OIG's findings of 2.6 percent data integrity problems for turbidity
and chlorine residual measurements underscore the need to continue to
stress the value of thorough sanitary surveys. The Region and the
States already recognize that value. State resources have never been
sufficient to support all of the surveys and technical assistance that
the States would like to provide.
2. The report uses the cases of Indiana, Minnesota, and Illinois to
illustrate that the Region was not aware of State activities concerning
data falsification, and uses these examples to recommend increased
Regional oversight of State programs. The Region does not agree that
these are valid examples of lack of Regional oversight.
The report states that the Region erroneously told OIG that Indiana
checked for obvious cases of data falsification, when the State did not.
During the last two years, the Region conducted audits about every six
months in Indiana, to ensure that the State was correctly implementing
the EWSS piujidm. (In other States, audits occur annually.) Reviewing
the files for data falsification was not possible until Indiana
established a good overall records management system. The Region
decided to focus on the records management problem first, since checking
records is impossible until the records are organized. Only since
Fiscal Year (FY) 1993 have State records been sufficiently organized to
46
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
APPENDIX 1
Page 3 of 5
support basic ocnpliance and enforcement activities. When the Region
responded to the OIG's original request for information, the Region
referred to the written procedures in Indiana's compliance strategy, and
confirmed them with Indiana's FWSS program director at the time. The
conpliance strategy details the State's procedures to ensure system
conpliance with the regulations, and the State's enforcement protocol.
The report also states that the Region was unaware that Illinois and
Minnesota were not requiring nonujiiiunity water systems to submit
monthly operation reports (MCR). These are not valid exanples, because
the activities in both States fall under the States' rule implementation
discretion. These are cases where it is not appropriate for the Region
to know every detail. Specifically, both States' regulations require
monthly reporting of turbidity data. Section 611.262 of the Illinois
rules contains this requirement, and since Minnesota adopted the Federal
rules by reference, the citation is 40 Cbde of Federal Regulations
141.75. In Illinois, the State has contracted out noncommunity water
system tracking responsibilities to the county health departments, which
are responsible for tracking the submittal of MCRs. In Minnesota, the
State has been testing alternative treatment technology for small
surface water systems. Since the regulations do not specify turbidity
requirements for alternative technologies, the State was within its
discretion not to require MCRs from these systems.
RESPONSE TO THE 016 RECOMEHDKnCKS
The Region agrees with the OIG's three reccmraendations for State follow-up.
The recommendations and the Region's response are listed below.
Ran iiiimudaticp 1; Request that the States provide training to, and/or
consider taking enforcement actions, as appropriate, against the 19 public
water systems (PWS), or operators, which the DIG found reported invalid or
potentially falsified test data.
By October 31, 1994, Region 5 will send a letter to the
States that requests that they follow through on the above
recommendation. In seme cases, the States have already corrected the
problem, and no further action is needed. Please note that enforcement
action will only be taken if further investigation by the States
confirms that the potentially falsified data was indeed falsified. Even
if the State confirms that data falsification occurred, the State may
decide not to take enforcement action, because other enforcement cases
that have direct public health implications may take priority. For
example. States may decide to direct their limited resources to
enforcement actions against systems that are violating the maximum
contaminant level for a particular contaminant, or that failed to
collect and analyze samples. In addition, please note that the States
can refer cases to the State Attorney General (SAG) for enforcement
action, but this does not guarantee that the SAG will act upon the case.
Also note that successfully prosecuting violators is difficult, and has
been a low yield investment of limited resources.
47
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
APPENDIX 1
Page 4 of 5
2; Request that the States develop a certification block to be
included en MSRs to ensure that reported data is reviewed for reasonableness.
By October 31, 1994, Region 5 will send a letter to the
States that requests that they follow through on the above
recomnendation.
Request that the States add documented steps to their
sanitary survey procedures to verify the validity of reported data. These
steps should, at a minimum, include a sample review and comparison of fDRs and
on-site FWS records.
Ragponae: By October 31, 1994, Region 5 will send a letter to the
States that requests that they follow through on the above
recommendation. The Region will be providing sanitary survey training
this fall that incorporates procedures for checking for data
falsification. Please see the Region's response to recaimendation 6
below.
The Region also agrees with the OIG's three recommendations for Regional
follow-up. The recaimsndations and the Region's response are listed below.
»tiop 4; Evaluate States' efforts in detecting invalid or falsified
data during Region 5's annual review of each State's drinking water program.
j: The FY 1995 EVES program grant guidance included two
applicable activities. First, the Region asked the States to screen
data submitted by FWSs for evidence of data falsification, and take
follow-up enforcement action as appropriate. The States were also asked
to report on the number of falsification cases identified, and the type
of follow-up taken. The States included these activities in their
pru-jiaiu plans. These elements will be evaluated by the Region, along
with all other elements, as part of our annual evaluation cycle.
Redistribute the Office of Ground Water and Drinking
Water's (OGWCW) .appendix G to each of the States in the Region and reemphasize
its importance.
By October 31, 1994, Region 5 will redistribute ^pendix G to
the States. It will accompany the letter referred to in the response to
recommendations 1 through 3 above.
6; Incorporate techniques for identifying questionable data
into sanitary survey training. This training should be given to both State
and Regional staff.
Region 5 has already worked with the OGWDW to add data
falsification review techniques to the next round of sanitary survey
training. Two training classes will be held in Region 5 in October and
Ifavember 1994; one class on the Illinois/Indiana border, and one on the
Michigan/Ohio border. In addition, the Region will request that the
48
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
APPENDIX 1
Page 5 of 5
States report en the usefulness of the data falsification training
coipcnent in their annual self-evaluation of progress on grant
ocnnu.tnients.
49
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
APPENDIX 2
Page 1 of 1
ABBREVIATIONS
CDC Centers for Disease Control and Prevention
CFR Code of Federal Regulations
EPA Environmental Protection Agency
FRDS Federal Reporting Data System
GAO General Accounting Office
MOR Monthly Operational Report
NTU Nephelometric Turbidity Units
OGWDW Office of Ground Water and Drinking Water
OIG Office of Inspector General
PWS Public Water System
50
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
APPENDIX 3
Page 1 of 1
DISTRIBUTION
Inspector General (2410)
Assistant Administrator for Water (4101)
Director, Office of Ground Water & Drinking Water (4601)
Regional Administrator, Region 5 (R-19J)
Director, Region 5 Water Division (W-15J)
OIG Office of Investigations (II-13J)
Agency Follow-up Official (3304)
Attention: Assistant Administrator for the Office of
Administration and Resources Management
Agency Follow-up Coordinator (3304)
Attention: Director, Resources Management Division
Audit Follow-up Coordinator (3304)
Attention: Management Controls Branch
Audit Follow-up Coordinator (3802F)
Attention: Office of Policy, Training, and Oversight
Division
Audit Follow-up Coordinator (1104)
Attention: Executive Support Office
Region 5 Public Affairs (P-19J)
Region 5 Intergovernmental Relations Office (R-19J)
Headquarters Library (3404)
Region 5 Library (PL-12J)
51
Report No. E1HWC4-23-0009-4100540
-------
DRINKING WATER DATA INTEGRITY
[This page intentionally left blank.]
52
Report No. E1HWC4-23-0009-4100540
------- |