US EPA Headquarters Library
U'b Mail code 3404T
1200 Pennsylvania Avenue
^^fsfef
OFFICE OF INSPECTOR GENERAL
Evaluation Report
Effectiveness of Effluent Guidelines
Program for Reducing Pollutant
Discharges Uncertain
Report No. 2004-P-00025
August 24, 2004
-------
Report Contributors:
Renee McGhee-Lenart
Julie Hamann
Kevin Morris
Gerry Snyder
Elizabeth Grossman
James Haller
Abbreviations
AOX Adsorbable Organic Halides
COD Chemical Oxygen Demand
EPA Environmental Protection Agency
NPDES National Pollutant Discharge Elimination System
NRDC Natural Resources Defense Council
OIG Office of Inspector General
PCS Permit Compliance System
SIC Standard Industrial Classification
Photo Caption:
Effluent from pulp mills being discharged into the Columbia River in
Longview, Washington (EPA Photo)
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
August 24, 2004
MEMORANDUM
SUBJECT: Effectiveness of Effluent Guidelines Program for
Reducing Pollutant Discharges Uncertain
Report No. 2004-P-00025
FROM: Dan Engelberg /si
Director for Program Evaluation, Water Issues
Office of Program Evaluation
TO: Benjamin Grumbles
Acting Assistant Administrator
Office of Water
This is our final report on the subject evaluation conducted by the Office of Inspector General
(OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that
describe the problems the OIG has identified and corrective actions the OIG recommends. This
report represents the opinion of the OIG, and the findings contained in this report do not
necessarily represent the final EPA position. Final determinations on matters in this report will
be made by EPA managers in accordance with established audit resolution procedures.
On June 4,2004, the OIG issued a draft report to EPA for review and comments. A response
was submitted on July 7,2004, and an exit conference was held on August 19,2004.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 90 calendar days of the date of this report. You should include a corrective action
plan for agreed upon actions, including milestone dates. We have no objections to the further
release of this report to the public. For your convenience, this report will be available at
http://www.epa. eov/oie. In addition to providing a written response, please e-mail an electronic
response to en gel berg, dantjfcep a. goy.
If you or your staff have any questions regarding this report, please contact me at (202) 566-0830
or Renee McGhee-Lenart, Assignment Manager, at (913) 551-7534.
-------
-------
Executive Summary
Purpose
Effluent guidelines are national technology regulations that limit the discharge of
pollutants to surface waters and publicly owned treatment works. By creating
minimum levels of treatment for different industrial sectors based on the
environmental performance of specific technologies, effluent guidelines are
intended to establish consistent limits across the country. Guidelines produce an
environmental outcome by having their requirements factored into individual
facilities' discharge permits as they are renewed. The Environmental Protection
Agency (EPA) has developed effluent guidelines for 55 industrial point source
categories affecting between 35,000 to 45,000 facilities that directly discharge to
the nation's waters. According to EPA, effluent guidelines are responsible for
preventing the discharge of almost 700 billion pounds of pollutants each year.
For this evaluation, we sought to answer the following questions:
• How has EPA's effluent guidelines development process changed over time?
• How effectively are effluent guidelines used to reduce pollutant loadings?
• To what extent does EPA measure the effectiveness of the effluent guidelines
program?
Results in Brief
The effluent guidelines program underwent a number of changes in the 1990s.
EPA began covering a broader range of pollutants, over a broader array of
industries, and generally promulgated more effluent guidelines. Some changes
were under the impetus of a Federal Advisory Committee Act committee, formed
as the result of a 1992 Consent Decree. EPA issued a draft strategy in November
2002 on how it will select and revise effluent guidelines in the future, and the
program is currently undergoing review for resource allocation.
Regarding effectiveness, the impact of effluent guidelines remains uncertain.
Although effluent guidelines were used in the National Pollutant Discharge
Elimination System (NPDES) permits we analyzed, pollutant discharge data were
not readily available to determine whether effluent guidelines reduced pollutant
discharges. We found a lag in issuing NPDES permits that utilized the revised
effluent guidelines. Once reissued, permit limits were derived from 1he revised
guidelines to a very large extent. We also found that adequate information was
widely absent, although revised guideline-derived permit limits had an impact on
the limited number of facilities with adequate information. Due to a lack of
-------
pollutant discharge data, we could not determine the extent of environmental
benefits brought about by EPA's investment in the effluent guidelines program.
Further, EPA does not measure the effectiveness of either the effluent guidelines
program or individual effluent guidelines. Consequently, EPA does not have
sufficient evidence to show that this program has actually produced reductions.
Although our work showed significant reductions in a few facilities, EPA has not
systematically collected data to evaluate this program as a whole. Therefore, EPA
cannot support a statement made in its recent Annual Report that industrial
discharges of pollutants have been reduced by billions of pounds as a result of
effluent guidelines. The effluent guidelines program has a marked insufficiency
of information to make managerial decisions because EPA has not developed a
systematic way of collecting such information.
Recommendations
We recommend that the Acting Assistant Administrator, Office of Water,
systematically collect and monitor data for a select number of facilities to measure
the effectiveness of specific effluent guidelines. We also recommend that the
Assistant Administrator develop performance measures based on the systematic
collection of data and take the necessary steps to ensure mat appropriate data is
collected.
Agency Comment and OIG Evaluation
In a July 7, 2004, response to our draft report (see Appendix D), the Office of
Water generally agreed with our findings and recommendations. The Office of
Water agreed to perform retrospective analyses to determine the effectiveness of
several effluent guidelines. The Office of Water also agreed that actual pollutant
discharge data should be used to develop performance measures and stated that
the work evaluating the effectiveness of several effluent guidelines will help
address this recommendation. Finally, the Office of Water agreed that adding a
data field to the Permit Compliance System would be a useful way to link
reporting facilities with the appropriate effluent guidelines.
We agree with the Office of Water's proposed actions. However, we would like
to obtain additional information about the retrospective studies EPA plans on
undertaking and how these individual studies will be used to develop an
assessment of the overall program. For example, we would like to know which
effluent guidelines will be selected for analysis, the methodology to be used, and
timeframes for completing'the studies.
We also made technical changes to the draft in response to the Agency's
comments.
-------
Table of Contents
Executive Summary i
Chapters
1 Introduction 1
2 Effluent Guidelines Program Has Undergone Changes 5
3 Effectiveness of Effluent Guidelines Remains Uncertain 9
4 EPA Does Not Adequately Measure Program Performance 15
Appendices
A Details on Scope and Methodology 19
B NPDES Permit Analysis Results 21
C Case Study of a Pulp and Paper Facility 23
D Agency Response 29
E Distribution 33
-------
-------
Chapter 1
Introduction
Purpose
Effluent guidelines are national technology regulations that limit the discharge of
pollutants to surface waters and publicly owned treatment works. By creating
minimum levels of treatment for different industrial sectors based on the
environmental performance of specific technologies, effluent guidelines are
intended to establish a minimum floor of control across the country. Guidelines
produce an environmental outcome by having their requirements factored into
individual facilities' discharge permits as they are renewed. The Environmental
Protection Agency (EPA) has developed effluent guidelines for 55 industrial point
source categories affecting between 35,000 to 45,000 facilities that directly
discharge to the nation's waters. Guidelines cover industries as diverse as iron
and steel to centralized waste. According to EPA, effluent guidelines are
responsible for preventing the discharge of almost 700 billion pounds of
pollutants each year through their utilization in National Pollutant Discharge
Elimination System (NPDES) permits. EPA has budgeted about $22 million a
year for the last 3 fiscal years (2001 to 2003) to develop effluent guidelines. For
this evaluation, we sought to answer the following questions:
• How has EPA's effluent guidelines development process changed over time?
• How effectively are effluent guidelines used to reduce pollutant loadings?
To what extent does EPA measure the effectiveness of the effluent guidelines
program?
Background
In 1972, Congress established the effluent guidelines program by adopting the
Federal Water Pollution Control Act of 1972, which was amended by the 1977
Clean Water Act Amendments and the Water Quality Act of 1987. Congress
adopted these Acts to "restore and maintain the chemical, physical, and biological
integrity of the Nation's waters." EPA's Office of Water is responsible for
implementing these Acts, which provide EPA and the States with a variety of
programs to protect and restore the nation's waters.
The effluent guidelines program, along with the water quality standards and
criteria program, form the basis of all water quality programs used by EPA to
reduce point source loadings. National effluent guidelines regulations typically
specify the maximum allowable levels of pollutants that may be discharged by
-------
facilities within an industrial category. While pollutant limits are based on the
performance of specific technologies, they do not generally require each facility to
use these technologies; rather, they allow it to use any effective alternatives to
meet the numerical pollutant limits.
Each facility within an industrial category must generally comply with the
applicable discharge limits, regardless of its location within the country or on a
particular water body. In this way, the limits are consistent for all facilities wilhin
an industrial category or subcategory. National regulations apply to three types of
facilities within an industrial category:
• Existing facilities that discharge directly to surface waters.
• Existing facilities that discharge to publicly owned treatment works.
• Newly constructed facilities that discharge directly to surface water.
« Newly constructed facilities that discharge to publicly owned treatment works.
According to EPA, effluent guidelines, through their use in NPDES permits, are
responsible for preventing the discharge each year into public waters of over
1 billion pounds of toxic pollutants, such as heavy metals; over 470 billion pounds
of non-conventional pollutants, such as nutrients and salts; and almost 220 billion
pounds of conventional pollutants, such as suspended solids. All facilities that
discharge pollutants from any point source into waters of the United States are
required to obtain a NPDES permit. Table 1.1 provides definitions for each
pollutant type and additional examples.
Table 1.1: Definitions and Examples of Pollutant Types
PoltwiantType
Conventional
Toxic
Non-conventional
BeflniSoj*
Pollutants typical of municipal
sewage and for which municipal
secondary treatment plans are
typically designed. These
pollutants are defined by regulation.
Pollutants or combination of
pollutants that cause death,
disease, or other injuries to humans
or animals upon exposure,
inhalation, or ingestion. The
pollutants are defined by regulation.
All pollutants not listed by
regulation.
examples ^"V^"
Biological oxygen
demand, total
suspended solids
Dioxin, chloroform
Acetone, ammonia
Initially, the 1972 Clean Water Act directed EPA to develop effluent guidelines
for existing industrial dischargers by certain statutory deadlines. EPA was unable
to do this by the statutory deadlines and was sued by the Natural Resources
-------
Defense Council (NRDC). In 1976, EPA entered into a consent decree with
NRDC and agreed to speed the completion of effluent guidelines and address
more toxic pollutants when developing and revising effluent guidelines.
The Clean Water Act was amended by the Water Quality Act of 1987, which
required EPA to establish schedules for reviewing and revising existing effluent
guidelines and promulgating new ones. In 1990, EPA published its first Effluent
Guidelines Plan, with schedules developing new and revised effluent guidelines
for several, industrial categories. Following another suit from the NRDC and
Public Citizen, Inc., EPA, in 1992, agreed to abide by a consent decree that
established a schedule for EPA to promulgate effluent guidelines for 19 industrial
categories. The consent decree required EPA to develop effluent guidelines for
certain industries, and allowed EPA the discretion of selecting other industries for
effluent guidelines development.
The consent decree also required that EPA establish an Effluent Guidelines Task
Force (Task Force) to develop recommendations on how to improve the effluent
guidelines program. The Task Force sought to determine ways in which the
effluent guidelines process could be streamlined.
State and EPA permits writers are responsible for writing NPDES permits. When
developing a permit, the permit writers must calculate technology-based effluent
limits from effluent guidelines and compare them to water quality-based effluent
limits for each pollutant in a permit. The Clean Water Act and EPA regulations
require the permit writer to apply the most stringent limit. A permit writer can use
an effluent guideline in developing a facility's permit after the effluent guideline
is effective (typically about 60 days after the effluent guideline is promulgated).
Scope and Methodology
We conducted our evaluation in accordance with Government Auditing Standards,
issued by the Comptroller General of the United States. We conducted our field
work from August 2002 to November 2003. We evaluated the effluent guidelines
program by developing and applying a four-phase model that describes the four
key processes involved in the program (Table 1.2).
-------
Table 1.2: The Four Phases of the Effluent Guidelines Program
- Pfease>
iiS
D
s
U
f-
'IE
:;'::! :
e^ectior».-,:.,.:..:.K.,
eveioprhent
||gl||::;-^ii
fiii^ibri'H:l;ffi
^SvS1::.';1":: ;;.:::::.:::«.:.v:;:i
;;.;:.;:V;:rV<:;i;i:.-V;O?:i::::;:
'::,yv o j:;::': ;: 1V;:-':V':.!-* o:;;;
yaliiiation^s^-
'II^aiPi^Kl?
Definition
EPA examines many different industries and then selects certain
industries for effluent guidelines development.
EPA develops effluent guidelines for each selected industrial
category. Guidelines are prepared according to formal rulemaking
procedures. To develop an effluent guideline, EPA must conduct
technology, economic, and environmental assessments.
To be utilized, an effluent guideline must be implemented in an
NPDES permit and used as the basis of pollutant limits.
To determine whether effluent guidelines result in environmental
improvements, EPA must collect and analyze data to measure
results and improve the prior three phases.
To evaluate these four phases, we interviewed EPA officials, State permit writers,
publicly owned treatment works' representatives, environmental groups, and
industry officials. We also reviewed the development documents for several
effluent guidelines, recommendations made by the Task Force, EPA's disposition
on the Task Force recommendations, and EPA's Annual Performance Plans.
We evaluated the effectiveness of the effluent guidelines program by analyzing
the NPDES permits of three industries: pesticide manufacturing; pharmaceutical
manufacturing; and pulp, paper, and paperboard.
We evaluated the effluent guidelines program because it is a key water program
and the Office of Inspector General (OIG) and General Accounting Office had not
evaluated the program in recent years.
Appendix A provides further details on our scope and methodology.
-------
Chapter 2
Effluent Guidelines Program Has Undergone Changes
The effluent guidelines program underwent a number of changes in the 1990s and
is still in the midst of change. EPA began covering a broader range of pollutants,
over a broader array of industries, and promulgated effluent guidelines at a faster
pace. Some changes were under the impetus of a Federal Advisory Committee
Act committee, also known as the Effluent Guidelines Task Force (Task Force),
formed as a result of a 1992 Consent Decree. The Task Force issued five reports
from 1995 to 2000 and made recommendations to EPA on how to improve the
effluent guidelines program. However, EPA has only incorporated about half of
the Task Force's recommendations. Additionally, EPA is Devaluating the future
of the effluent guidelines program.
Coverage Increased
Broader Range of Pollutants Covered
Over time, EPA has developed effluent guidelines covering a broader range of
. pollutants. Expanding pollutant coverage is important because of guidelines'
important role in providing coverage on a national basis since water quality
standards' coverage of toxic pollutants is relatively limited.
The first round of effluent guidelines that EPA developed during the 1970s
typically covered conventional pollutants. As a result of the 1977 Clean Water
Act amendments and pressure from law suits, EPA focused on covering a broader
range of toxic and non-
conventional pollutants, such as
dioxin and furan, in its effluent
guidelines. The growth increased
during the 1990s. For the three
industries whose revised effluent
guidelines we reviewed, the
guidelines in each industry had
dramatically greater pollutant
coverage in the 1990s than they
had under their original
guidelines, adopted in the late
1970s and early 1980s (see
Figure 1). The revised guidelines for these industries covered an additional 165
pollutants unregulated by the previous technology standards. All of the additional
pollutants covered by the three new effluent guidelines were toxic (25 percent) or
non-conventional (75 percent) pollutants. By controlling a wider range of
Figure 1
1 150-,
a 125 -
0. jg .
* i=
E
: Additional Pollutants Covered In Three
Revised Effluent Guidelines
12
ii
I
I
: ^f. ;
i ^lm;mm, 20 !
z w
Pesticide Pharmaceutical Pulp & Paper
i
1 Previous EG S Revised EG
-------
pollutants, especially toxic and non-conventional pollutants, EPA expects to
improve human health, water quality, and aquatic life. Many of these pollutants
have been associated with serious environmental and human health impacts.
Broader Range of Industries Covered
EPA has promulgated effluent guidelines for 55 point source categories, which
have, over the last 10 years, covered a broader range of industries. In 1974, EPA
had promulgated effluent guidelines for 15 industries, such as sugar processing,
coal mining, and cement manufacturing. In recent years, EPA has increasingly
focused on non-industrial point source categories, such as centralized waste
treatment and landfills. By broadening the types of industries included in the
effluent guidelines program, EPA expects to be able to control waste streams that
were not envisioned in the past.
More Effluent Guidelines Promulgated
EPA has dramatically increased its rate of issuance of effluent guidelines in recent
years. Whereas it issued 4 effluent guidelines in the 9 years from 1987 to 1995, it
issued 14 in the subsequent 8 years from 1996 to 2003 (see Figure 2). Some are
new guidelines for industries not
previously covered, and others are
revisions of guidelines for
previously covered industries.
EPA was required to promulgate
effluent guidelines at a faster pace
by the 1992 Consent Decree. The
consent decree had established a
schedule by which EPA was to
propose and take final action on
both the revision of existing
effluent guidelines and the
promulgation of new effluent
guidelines. It is important that
effluent guidelines are
promulgated timely since, without effluent guidelines, the potential reductions of
many pollutant discharges would not occur.
Some Changes Resulted from Task Force Recommendations
The effluent guidelines program changed as a result of the Task Force
recommendations. The 1992 Consent Decree established the Task Force to
provide recommendations on how to improve the effluent guidelines program.
The Task Force consisted of representatives from EPA, States, local governments,
figure 2: Newand Revised Effluent Guidelines
Promulgated Between 1987 and 2003
8 S
I 4
•o
33
02
.Q
V V V
Ysr
-------
environmental groups, industry,
and academia. It issued 5 reports
(see box) that made
80 recommendations to EPA on •
how to improve the effluent
guidelines program. The Task
Force recommendations fell into
three main categories. Table 2.1
notes those three categories and the
number of recommendations for
each, and provides examples.
Five Tusk Force Reports
Selection Criteria for Preliminary Industry
Studies (1995)
Fostering Pollution Prevention and
Incorporating Multi-Media Considerations into
Effluent Guidelines Development (1096)
Issue Paper Design of Preliminary Studies
(1996)
Removing the Bottlenecks from the Effluent
Guideline Development Process (1996)
Recommendations on Streamlining the Effluent
Guideline Development Process (2000)
Table 2.1: Examples of Effluent Guidelines Task Force Recommendations
. Recommendations
Category
Improving the Pace of
Guideline Development
improving the Selection of
Industries for Guideline .
Development
Incorporating Pollution
Prevention and a Mufti-Media
Consideration into Guidelines
Quantity
41
24
15
Exaroptesof
RecommefldatJons , y
Developing surveys quicker by working
cooperatively with industry and keeping
the number of individual questions to
ones absolutely needed.
Seek out and contact trade associations
in order to provide such groups the
opportunity to contribute available data.
Promote pollution prevention by
allowing an extension of time for a
facility to come into compliance.
EPA stated that it has fully implemented 36 of the 80 recommendations, partially
implemented 39, and has not implemented 5 as of 2003.
EPA Reevaluating Future of Effluent Guidelines Program
EPA anticipates that it will fulfill its commitments under the provisions of the
1992 Consent Decree by the end of 2004 and is preparing to manage the effluent
guidelines program outside of the requirements of the decree. Additionally, in
late 2003, the program was reviewed by the Office of Water to determine resource
allocation. However, it remains to be seen whether the changes resulting from the
1992 Consent Decree, such as the increase in the number of effluent guidelines
promulgated, will continue in the future.
EPA is charting a new course for the future of the effluent guidelines program.
First, in November 2002, EPA developed a draft strategy,^ STRATEGY FOR
NATIONAL CLEAN WATER INDUSTRIAL REGULATIONS: Effluent Limitations
-------
Guidelines, Pretreatment Standards, and New Source Performance Standards.1
The strategy was developed to provide interested stakeholders the chance to
comment on how national industrial technology-based regulations could best meet
the needs of the broader national clean water program. It also outlined the criteria
EPA expects to use to assess the need to develop and/or revise effluent guidelines
for specific industrial categories. Since EPA will be responsible for selecting
industries for effluent guidelines development after the consent decree is fulfilled,
this draft strategy was an important step in establishing how the program will
proceed in the future.
In the future, EPA will be developing effluent guidelines with fewer staff. In the
Fall of 2003, the Office of Water reviewed the staff allocation for the Engineering
and Analysis Division that develops effluent guidelines. Office of Water
management chose to reduce the number of staff in the division from 55 to 35 and
move these staff to other divisions. For example, 11 staff will be moved to the
Standards and Health Protection Division. This reflects the enhanced emphasis
on developing water quality standards and enhancing EPA and States' capabilities
to monitor water quality. It remains to be seen how this reallocation of resources
will affect the effectiveness of the effluent guidelines program in the future.
EPA may have to reevaluate the program again in the near future. In a memo
submitted to EPA on March 18, 2004, concerning EPA's Preliminary Effluent
Guidelines Program Plan for 2004/2005, NRDC urged EPA "to reevaluate both its
legal obligations under the CWA (Clean Water Act) and the overarching purpose
of the requirements under section 304(m) of the Act and to make a commitment to
updating technology standards and reducing water pollution in our nation's
waters." NRDC is alleging that EPA's plan for new guidelines violates section
304(m) of the Clean Water Act because it does not establish a schedule for annual
review and revision for all effluent sections, as well as fails to identify categories
of sources discharging toxic or non-conventional pollutants for which guidelines
have not previously been published. Specifically, NRDC's comments say the
guidelines "suffer from many of the same deficiencies" as the previous plans over
which NRDC filed suit.
'See http://vvww.epa.gov/euide/strateRV/
-------
Chapters
Effectiveness of Effluent Guidelines Remains Uncertain
The impact of EPA's effluent guidelines in helping EPA reduce pollutant
discharges and meet Clean Water Act goals remains uncertain. Although effluent
guidelines were used in the NPDES permits we analyzed, pollutant discharge data
were not readily available to determine whether effluent guidelines reduced
pollutant discharges. To determine the effectiveness of an effluent guideline, we
had to determine how quickly permits for facilities affected by guidelines were
reissued; how widely guidelines were used in these permits; and the magnitude, if
any, of resultant pollutant discharge reductions. We found:
• A lag in issuing permits that utilized the revised effluent guidelines.
• Once reissued, permit limits were derived from the revised guidelines to a
very large extent.
• Adequate information was generally absent, although revised guideline-
derived permit limits had an impact on the limited number of facilities we
examined that had the necessary information for our assessment.
Due to a lack of pollutant discharge data, we could not determine the overall
extent of environmental benefits brought about by EPA's investment in the
effluent guidelines program.
Lag In Reissuing Permits Delayed Realizing Benefits
A delay in issuing permits slowed the implementation and, thus, the benefits (and
costs) of effluent guidelines. For our review, we focused on effluent guidelines
for three industries: pesticide manufacturing (issued in 1993); pharmaceutical
manufacturing (issued in 1998);
and pulp, paper, and paperboard
(issued in 1998), As shown in
Figure 3, of the 109 major
facilities that we identified as
being covered by these three
guidelines, 69 had been issued a
new permit utilizing the revised
guidelines. Of the remaining
40 facilities, 36 were operating
with expired permits as of
December 31, 2002, our
J"l
J= en .
Number of Fac
ro *• <
s o o <
Figure 3: Many Facilities Are Operating With
Expired Permits
69
&M
P
~
P
36
4
Facilities with Expired Permits Permits Utilizing
New Permits The Previous
Guideline
-------
established cut-off date, and 4 had been reissued permits erroneously using the
previous guideline for the industry.
A high proportion of the expired permits had been expired for a long time. Over
60 percent (22 of 36) of the permits had been delayed 4 or more years. Even
among those facilities whose permits had been reissued by the end of 2002,
reissuance was often delayed. More than three-quarters (77 percent) of the 69
facilities whose permits had been reissued by the end of 2002 had not been issued
on time (see Table 3.1). As a result, much of the potential benefit of EPA's
investment in new effluent guidelines was delayed.
Table 3.1 The Time Elapsed Before the Permits Implementing
the Three Effluent Guidelines Were Reissued
.. Tirrte Elapsed
vReirojfe;Rei^ued.;^
fj^fliiiifs:R"eji$^^
;^erniii;(CeiSlue:d:;iit!3:l;^e^:ii&tev:
^eiroite;Reissue^!2
-------
Reissued Permits Employ Guidelines to Large Extent
Although variation occurred, permit writers used effluent guidelines to derive
85 percent of the pollutant discharge limits for the 69 facilities whose permits we
reviewed. Our analysis only focused on the pollutants that had effluent
guidelines pollutant limits. This extremely high reliance on effluent guidelines to
derive permit limits indicates that these guidelines may have filled a void in the
coverage of pollutants discharged into the nation's waters. Since permit writers
are required to use the most stringent limit for a pollutant (most stringent of water
quality standards or effluent guidelines)., our results suggest that (1) deriving
discharge limitations from effluent guidelines provided more stringent controls
than water quality standards, or (2) water quality standards did not exist for some
of the applicable pollutants.
Although the guidelines were used at a high level overall, the three industries
differed in their use of the guidelines. Of the 1,828 pollutant limits reviewed for
these 69 facilities, effluent guidelines were used to derive 91 percent of the pulp,
paper, and paperboard pollutant
Figure 4: Effluent Guidelines Usage by Industry
and Pollutant Type
1001
limits; 84 percent of the
pharmaceutical manufacturing
pollutant limits; and 66 percent of
the pesticide manufacturing
pollutant limits (see Figure 4 and
Appendix B). The use of the
pesticide manufacturing effluent
guideline was tower because
about 20 percent of the pollutant
limits were limited by another
effluent guideline - the organic
chemicals, plastics, and synthetic
fibers effluent guideline.
Data Largely Unavailable, Although Discharge Reductions Noted
in Most Cases Reviewed
A lack of information about discharges prior to the imposition of new limits
prevented us from answering whether effluent guidelines effectively reduced
pollutant discharges. We found adequate data to analyze changes in discharges
for only seven facilities (four covered by the pulp, paper, and paperboard rule and
three by the pharmaceutical rule). Our approach and findings may be useful to
EPA should it decide to conduct its own evaluation of this program. Details
follow.
11
-------
Data Largely Lacking
EPA lacked the data needed to determine the degree to which these effluent
guidelines reduced pollutant loadings. Although EPA collects some facility data
to develop effluent guidelines limits, it does not systematically and continuously
collect data to examine guidelines' effectiveness.
Additionally, data were not generally available in the Permit Compliance System
(PCS), a national information system that tracks NPDES implementation. Data
characterizing the industry prior to effluent guidelines promulgation did not exist
in the PCS due to a lack of monitoring requirements in earlier NPDES permits.
When data were available, they were often reported differently (i.e., daily
maximum versus monthly average) before and after the promulgation of the
effluent guideline, making comparison difficult Additionally, the analytical
methods used to measure certain pollutants, such as dioxin and furan, are only
reliable to a certain level, and pollutant loadings measured below this level are
considered "non-detect," making analysis difficult.
Additionally, pollutant discharge data did not exist prior to the effluent guidelines
being implemented for us to examine pollutant discharge changes, or the
pollutants were limited by water quality standards and not effluent guidelines.
For these reasons, we were unable to conduct the broad analysis we had intended
because of the lack of available, reliable, and valid data. As a result, we were only
able to analyze the discharges of two pollutants:
• Adsorbable organic halides (AOX), for pulp, paper, and paperboard facilities.
• Chemical oxygen demand (COD), for pharmaceutical manufacturing facilities.
Facilities Implementing Effluent Guidelines Demonstrated Pollutant
Discharge Reductions
Figure 5
Decline in Discharges
Seven Facilities
100%
| 80%
£ 60%
1 40%
| 20%
0%
No sign if.
difference
No jjgnif.
difference
|E3 Pharmaceutical BPulp & Paper |
Pollutant discharge reductions
occurred at five of the seven
pharmaceutical manufacturing and
pulp, paper, and paperboard
facilities we analyzed (see
Figure 5). Further, at the
remaining two facilities (both
pharmaceutical firms), they had
already been meeting the new,
more stringent limits under their
previous permit and thus were
already discharging below effluent
guidelines limits. Facility
12
-------
F Ig a r«
P a
5
S
a
S
x
s
1,000
2,500
2,000
1.SOO
1,000
600
0
managers at five of the facilities showing reductions told us that the revised
guidelines were the principal reason they implemented new technology to reduce
their pollutant discharges. Our results for this small number of facilities
document the potential gains resulting from effluent guidelines regulations, and
we believe additional evaluation of this program by EPA is warranted. Details
regarding each facility type follow.
Pulp, Paper, and Paperboard Facilities: The 1998 Pulp, Paper, and Paperboard
Effluent Guideline was developed primarily to reduce the discharge of dioxin.
Studies have shown that exposure to dioxin at high enough doses may cause a
number of adverse health effects, such as cancer and reproductive or
developmental effects, and dioxin has also caused some fish consumption
advisories. However, dioxin is often discharged at extremely low concentrations,
even below detection limits, making it difficult to analyze trends in dioxin
discharges. AOX, on the other hand, is discharged at much higher levels, making
detection and trend analysis possible. EPA determined that when AOX is
decreased, a decrease in dioxin occurred,
Our statistical analysis on changes in
AOX levels found that all four pulp,
paper, and paperboard facilities
significantly reduced AOX discharges
after implementing new technology to
comply with effluent guideline-based
pollutant limits. Decreases ranged from
44 percent to 71 percent Figure 6
illustrates change in loadings at one of
the facilities; Appendix C provides a
detailed case study.
6: AOX Loading* From On* Pulp,
per, and Paparboard Facility
Pharmaceutical Facilities: Three facilities in our review had pollutant discharge
data for COD. COD is the measure of oxygen-consuming capacity present in the
wastewater; adequate oxygen levels in water are a necessity for fish and other
aquatic life. One facility decreased its pollutant discharges due to the effluent
guideline, while the other two facilities had been discharging below mandated
effluent guideline limits prior to the imposition of the guideline. The facility
reducing COD discharges did not yet have an approved NPDES permit with the
revised effluent guideline based COD limits, but the facility manager expected the
permit to be approved soon.
Conclusion
Our evaluation has shown that the effluent guidelines program is having some
impact, because the guidelines were being used in permits and were reducing
loadings in the few cases we could examine. Some of the impact may be lessened
13
-------
because effluent guidelines are not implemented timely into permits. However, in
general, EPA does not have the data needed to measure the results of the effluent
guidelines program. Therefore, due to the lack of pollutant discharge data, we
cannot determine whether this program is achieving environmental improvements.
Recommendation
We recommend that the Acting Assistant Administrator, Office of Water
3.1 Evaluate the effectiveness of effluent guidelines by systematically
collecting pollutant discharge data before and after an effluent guideline is
promulgated for a select number of facilities for each guideline.
Agency Comment and OIG Evaluation
The Office of Water agreed to perform retrospective analyses to determine the
effectiveness of several effluent guidelines. We agreed with the Office of Water's
proposed actions. However, we would like to obtain additional information about
the retrospective studies EPA plans on undertaking and how these individual
studies will be used to develop an assessment of the overall program. For
example, we would like to know which effluent guidelines will be selected for
analysis, the methodology to be used, and timeframes for completing the studies.
14
-------
Chapter 4
EPA Does Not Adequately Measure
Program Performance
EPA's assessments of the performance of the effluent guidelines program are
flawed. Although our work showed significant reductions in a few facilities, EPA
has not systematically collected data to evaluate this program as a whole. EPA
has developed some performance measures, but the data used to determine
whether the measures have been met are imprecise. Therefore, EPA cannot
support the following statement it made in its recent Annual Report:
-, ,J.^\ 1 ,.
JFV 2&02 Result: industrial Discharges of pollutants to the Nalfort's waters were
significantly reduced through implementation of effluent guidelines, A total of
approximately 2 billion pounds of industrial discharges was eliminated.
The effluent guidelines program, like other water programs that we have recently
examined, suffers from a marked insufficiency of information to make managerial
decisions, because EPA has not developed a systematic way of collecting such
information. In late 2003, Office of Water management decided to reprogram
Agency resources, moving staff from the effluent guidelines program to other
programs. These decisions are being made without sufficient understanding of
how they will effect the environment. EPA has acknowledged these limitations,
but has not put into place recommendations that we made in the past that would
help to correct this problem.
Congress and the President Require EPA to Report on
Program Effectiveness
The Government Performance and Results Act of 1993 and the President's
Management Agenda require EPA to measure the effectiveness of its programs:
• The Government Performance and Results Act requires agencies to
establish performance indicators to measure the outputs, service levels, and
outcomes of each program activity and to compare actual program results with
the established performance goals. An output measure is the tabulation,
calculation, or recording of activity, while an outcome measure indicates the
results of a program activity compared to its intended purpose.
• The President's Management Agenda notes that "many agencies and
programs lack rigorous data or evaluations to show that they work. Such
evidence should be a prerequisite to continued funding." It further notes that
all Federal agencies will be expected to use the Office of Management and
15
-------
Budget Program Assessment Rating Tool results and performance measures to
support and explain budget requests.
EPA Has Identified Useful Goals and Performance Measures
The goals and performance measures shown in Table 4.1, referred to as "reducing
industrial discharges," are designed to provide useful outcome-oriented
information to assess program results.
Table 4.1: EPA's 2003 Annual Performance Goals and Measures
for Reducing Industrial Pollutant Discharges (in millions of pounds)
Reduction in
compared toVJl 992 evels'as; pred icied; ty^ffi^'$ojp&<^£;^s§<
' (cumulative JXii%&f;;®
Redu(^on:ih loadings for conventlonaipoilirtar^fcr fed
fealities sul^ect to effluent guidelines promutgated be^
;-ai^:2bOQ/;a^^
;;pro$<$qins'(&^
10.5
572.0
1,007
13.5
715.7
1,199.8
In Fiscal Year 2003, EPA's annual performance goal was to develop effluent
guidelines that, when implemented, reduce pollutant loadings into surface waters.
The quantity of pollutant loadings being prevented from being discharged is a
valuable measure of outcomes and an indicator of program success. However, it
is not a true measure of environmental outcomes; measures such as the number of
impaired waters that were restored to designated uses or the number of fish
advisories reduced would be closer to a true environmental outcome measure.
As EPA gains experience designing measures, and improves the quality of its
environmental data bases, it should be able to construct such measures.
EPA's Performance Reports Are Imprecise
Although performance measures for the effluent guidelines program provide a
useful target against which to measure progress, the process EPA uses to measure
its progress is highly uncertain. EPA performance measures are based on two
estimates: an average reduction per facility and the number of facilities
implementing the guideline. As a result, EPA cannot truly assess benefits, since
each of these factors is highly uncertain. Details follow.
16
-------
Accuracy of Reduction Projections Untested
Because EPA has chosen to assess performance by estimating rather than
measuring results, it cannot verify the degree to which these projections are
accurate. Pollutant discharge reduction estimates are typically based on data that
EPA gathers at facilities when it develops the effluent guidelines. Because the
Clean Water Act does not authorize EPA to mandate which technologies facilities
will employ to meet effluent guidelines-based permit limits, variations occur in
the pollutant discharges at facilities. Although EPA conducts detailed analysis as
part of its rulemaking process, it does not verify the accuracy of the predictions
once the rule has been issued and, therefore, is not in a position to tell whether its
projections of facility-level discharge reductions are, in fact, accurate.
EPA officials stated that, due to limited resources, they had not assessed the
results of effluent guidelines after promulgation because they were focused on
developing and promulgating effluent guidelines required by the consent decree.
However, one EPA official said work has begun to assess results for one industry.
Estimates of Number of Facilities Using Guidelines Untested
Determining the number of facilities whose permits were issued and who utilized
the effluent guidelines is the second component of developing an estimate of
reduced pollutant discharges. However, EPA used a process to determine the
number of permits that could likely be in error and could result in an overestimate
of the loading reductions.
EPA's Water Permits Division obtains the Standard Industrial Classification (SIC)
codes for the applicable industrial categories with effluent guidelines. Staff from
five of the nine regions we interviewed explained that the numbers that they
supply to EPA headquarters are based on simply identifying those permits that
match the SIC codes supplied to them, typically through the PCS. Two regions'
officials said they send the list to the States to complete.
We found that identifying facilities that implemented effluent guidelines by using
SIC codes can result in overestimating the number of facilities. For example, the
revised pulp, paper, and paperboard effluent guidelines applied only to facilities
that produced bleached papergrade kraft or papergrade sulfite. The only way to
determine which facilities applied those subparts was to examine the permit file.
If only a SIC code search was done, EPA would identify all pulp, paper, and
paperboard facilities and not just those that produce papergrade kraft or
papergrade sulfite. EPA regional officials said they rarely examine the NPDES
permits to determine whether the facility was subject to the effluent guideline
before providing the number of facilities to EPA's Water Permits Division. It is
also possible that some industrial facilities implementing the effluent guideline
might not fall into the set of SIC codes provided to EPA regions, which could
result in undercounting the number of facilities.
17
-------
The reason EPA requests this type of information from the regional offices was
because permit writers were not required to input data on which effluent
guidelines were applicable to a facility. In our report, EPA Should Require
Program Results Data Fields For the Effluent Guidelines Program in the
Modernized Permit Compliance System (2002-M-000052), dated September 30,
2002, we recommended that EPA develop a required data field in the modernized
PCS to capture the effluent guideline or guidelines that apply to each permitted
- facility. EPA's Office of Enforcement and Compliance Assurance stated in its
response that such a field will be included in the modernized PCS. It is important
that this be done as soon as possible to ensure that an accurate number of permits
that utilized effluent guidelines is used in EPA's effluent guidelines performance
measures.
Conclusion
EPA does not have the data to measure the results of the effluent guidelines
program. EPA's current method of measuring performance is too imprecise since
it relies on estimates. Because EPA does not measure the effectiveness of the
program, the Office of Water cannot ensure that (1) resources are allocated
appropriately and efficiently, resulting in the largest degree of environmental
improvement for the resources it expends; and (2) the program is accomplishing
its pollution reduction goals.
Recommendations
We recommend that the Acting Assistant Administrator, Office of Water:
4.1 Develop performance measures that are based on actual pollutant discharge
data rather than discharge estimates gathered before the effluent guideline
was effective.
4.2 Work with the Office of Enforcement and Compliance Assurance to ensure
that it develops a required data field in the modernized PCS to capture the
effluent guideline or guidelines that apply to each permitted facility.
Agency Comment and OIG Evaluation
The Office of Water agreed that actual pollutant discharge data should be used to
develop performance measures and stated that the work evaluating the
effectiveness of several effluent guidelines will help address this recommendation.
Also, the Office of Water agreed that adding a data field to the PCS would be a
useful way to link reporting facilities with the appropriate effluent guidelines. We
agree with the Office of Water's responses to these recommendations.
18
-------
Appendix A
Details on Scope and Methodology
We evaluated Ihe effluent guidelines program by developing and applying a four-phase model that
describes the program: selection, development, utilization, and evaluation.
Selection
The first phase of mis process, 1he selection of industries for effluent guidelines development, is
not addressed in this report. Instead, we reported on the effluent guidelines selection process in a
briefing to EPA management on their draft strategy; that draft strategy is titled ,4 STRATEGY FOR
NATIONAL CLEAN WATER INDUSTRIAL REGULATIONS: Effluent Limitations Guidelines,
Pretreatment Standards, and New Source Performance Standards, dated November 2002.
Development
To evaluate the development process, we interviewed EPA officials, State permit writers, publicly
owned treatment works representatives, environmental groups, and industry officials. We also
reviewed documents EPA prepared describing the process used to develop several effluent
guidelines, recommendations made by the Effluent Guidelines Task Force, EPA's disposition on
the Task Force recommendations, and a flowchart of the effluent guidelines development process.
Utilization
To determine the utilization of effluent guidelines, we narrowed our focus to a limited number of
industries' effluent guidelines. We developed criteria to assist us in selecting among the different
industries with effluent guidelines. We chose industries with effluent guidelines that (1) had been
effective for several years so that the likelihood of effluent guidelines being utilized in the permits
reviewed was great; (2) limited toxic pollutants; and (3) had a defined number of SICs to define
the universe of major dischargers. Pesticide manufacturing; pharmaceutical manufacturing; and
tiie pulp, paper, and paperboard industries were the only industries that fit all of these criteria. As
a result, our results cannot be extrapolated to other industries' utilization of effluent guidelines.
We restricted our review to major, direct dischargers with NPDES permits in the three industries
listed above. Since the OIG is conducting a separate review on indirect dischargers, we focused
only on direct industrial dischargers.
To identify facilities that should have complied with one of the three effluent guidelines, we used
the PCS to search for major facilities by SIC code. We used the SIC codes for the pesticide
manufacturing; pharmaceutical manufacturing; and pulp, paper, and paperboard effluent
guidelines found in their development documents. For the pulp, paper, and paperboard effluent
guideline, we sampled all facilities listed by EPA as facilities producing bleached papergrade kraft
and soda or papergrade sulfite.
19
-------
From our research, we found 215 facilities with permits in these three industries. Of the 215
permits we requested, 104 were either (1) not required to implement any of the three effluent
guidelines we reviewed because the effluent guidelines were not applicable to those facilities, or
(2) information was unavailable. Two permits were issued prior to the revised guidelines
promulgation date and thus were not reviewed. We analyzed 109 NPDES permits required to
implement 1 of the 3 effluent guidelines to determine how timely the new effluent guidelines were
implemented into the permits, if at all. We did this by examining the most current permit to
determine whether the new effluent guideline had been implemented.
For those permits that implemented the new effluent guidelines (69 permits), we determined to
what extent effluent guidelines were used as the basis of pollutant limits. Our analysis only
focused on the pollutants that had effluent guidelines pollutant limits listed in the applicable Code
of Federal Regulations.
Finally, we set out to determine the impact that effluent guideline-based limits had on the
facilities' pollutant discharge levels. We found that comprehensive discharge data was not
available for most facilities. We identified only four pulp and paper facilities and three
pharmaceutical manufacturing facilities for which pollutant discharge data was readily available.
We interviewed the facility managers for the seven identified facilities and analyzed discharge
data from the PCS to determine whether there was a statistically significant difference in pollutant
loadings before and after facilities implemented technology changes to comply with the
guidelines.
Evaluation
To assess the evaluation process of the effluent guidelines program, we interviewed and obtained
data from EPA officials. We also interviewed State permit writers and industry representatives.
Further, we reviewed EPA's Annual Performance Plan for the last 5 years, analyzing the
performance measures related to the effluent guidelines program.
20
-------
Appendix B
NPDES Permit Analysis Results
This appendix displays further results of our evaluation of 69 facility permits: 47 pulp, paper, and
paperboard; 12 pharmaceutical manufacturing; and 10 pesticide manufacturing permits. We
determined whether the pollutant limits in these permits were based on effluent guidelines, water
quality standards, or permit writer's best professional judgment. Permit limits were occasionally
based on a combination of the above, such as effluent guidelines and best professional judgment.
We only evaluated those pollutants subject to the three effluent guidelines. For example, the
pollutant fecal coliform was in several pulp, paper, and paperboard permits, but since the pollutant
is not covered by the pulp, paper, and paperboard effluent guideline it was not considered in our
analyses. The number of pollutant limits in each category is presented in Table B.I.
Table B.1: The Classification of Pollutant-Based Limits in 69 NPDES Permits Reviewed
f f ••••••
Basis of p
-------
22
-------
Appendix C
Case Study of a Pulp and Paper Facility
Tembec USA operates a pulp and paper mill in St. Francisville, Louisiana, a town approximately
30 miles north of Baton Rouge with a population of 1,712. The facility is a bleached kraft pulp
and paper mill that produces board, container, and tissue papers, discharging its wastewater to the
Mississippi River. In the spring of 1999, Tembec changed its pulp bleaching process to reduce
their toxic discharges and comply with the Pulp, Paper, and Paperboard Effluent Guidelines. The
average AOX discharge between July 1999 and February 2003 was 68 percent lower than the
average AOX discharge between January 1995 and April 1999.
The Pulp, Paper, and Paperboard Effluent Guideline
The Pulp, Paper, and Paperboard Effluent Guidelines were revised in 1998 primarily to reduce the
discharges of toxic and non-conventional pollutants from two categories of mills: bleached
papergrade kraft and soda mills and papergrade sulfite mills. The National Dioxin Study, initiated
in 1983, found that dioxin was present in fish downstream of 57 percent of the pulp and paper
mills studied. To further investigate, EPA conducted a Five-Mill Study and the 104-Mill Study.
During the late 1980s, EPA found that dioxin was generated when chlorine or chlorine derivatives
were used in the bleaching process. Accordingly, EPA developed the 1998 Pulp, Paper, and
Paperboard Effluent Guidelines based on the complete substitution of chlorine dioxide for
chlorine as the key process technology. The guidelines required facilities to virtually eliminate all
dioxin and dioxin-related discharges.
Dioxins and Adsorbable Organic Halides
Dioxins refers to a group of chemical compounds that share certain chemical structures and
biological characteristics. Several hundred of these compounds exist. Studies have shown that
exposure to dioxins at high enough doses may cause an increased risk of cancer and reproductive
or developmental effects. Dioxins are also persistent compounds that break down slowly in the
environment, often becoming concentrated in food chains. The Pulp, Paper, and Paperboard
Effluent Guideline of 1998 addressed discharges of 2,3,7,8-tetrachlorodibenzo-Jp-dioxin (TCDD),
one of the most toxic dioxins. This dioxin has been identified as a human carcinogen based on
the weight of animal and human evidence.
Dioxins are often discharged at extremely low quantities, even below the detection limits made
possible by modem analytical techniques. For example, the current minimum detection level for
TCDD is 10 picograms per liter. Dioxin discharges below the minimum detection level cannot be
quantified and are reported in the PCS as "0," complicating analyses of dioxin discharge trends.
Still, unquantifiable dioxin discharges can bioaccumulate in fish tissue, resulting in possible harm
to those eating the fish. In fact, most dioxin exposure occurs through diet, with over 95 percent
coming through dietary intake of animal fats.
23
-------
Adsorbable Organic Halides (AOX) is a gross measure of chlorinated organic matter. EPA
concluded that reductions in AOX will reduce the mass of dioxin, and included AOX h'mitations
in the 1998 Pulp, Paper, and Paperboard Effluent Guidelines. AOX is also discharged at
significantly higher levels than dioxin, thereby eliminating the issue of discharges below the limit
of detection and making analyses of AOX discharge trends less complicated. OIG recognizes that
both the American Forest and Paper Association and the Alliance for Environmental Technology
did not agree with EPA's scientific conclusions concerning AOX as it relates to dioxin. Still,
AOX was included in the final rule-making, so we included it in our evaluation as a surrogate
measure for dioxin, based on EPA's statement, "that reducing AOX loadings will have the effect
of reducing the mass of dioxin... and other chlorinated organic pollutants discharged by this
industry." Further, in the permits we reviewed, AOX was the only pollutant (1) whose basis was
effluent guidelines; (2) that was revised in the 1998 Pulp, Paper, and Paperboard Effluent
Guidelines; and (3) that had discharge data available in the PCS.
AOX Discharge Trends at Tembec USA
In November 1994, EPA issued an NPDES permit for Tembec USA's St. Francisville Mill that
included quarterly monitoring requirements for AOX. The mill began reporting daily maximum
discharges of AOX in January 1995, as well as wastewater flows. Later, in 1999, the facility
implemented chlorine free bleaching, which was the pollution control technology recommended
by the effluent guideline. The facility expected that its NPDES permit was to be renewed in 1999
and would include new pollutant limits based on the 1998 Pulp, Paper, and Paperboard Effluent
Guidelines. Although the permit was not renewed until October 2002, the facility implemented
the new technology to meet the expected effluent guideline-based limits. The facility significantly
reduced its average daily maximum AOX discharges between April 1999 and July 1999. See
Figure C. 1 and Table C.I.
Figure C.1: Daily Maximum AOX Discharges for Tembec USA
1500
1000 -
500
M o nIto rlng D a te
24
-------
Table C.1: Daily Maximum AOX Discharge t-Test,
Assuming Unequal Variances (Tembec USA)
Observations
Discharges From
1/95 to 4/99
1.529
157.697
397.1
18
21
9.31
"< 0.001
1.721
< 0.001
2.08
Discharges From
7/99 to 2/03
604
19,972
141.3
18
Noting that the AOX discharges on July 1995 and April 1999 appeared to be outliers and might
lead to an overestimate of average AOX discharge before the new process was put in place, these
values were excluded from the data set and a t-test was re-run. Excluding these values, there was
still a significant reduction in AOX discharges between January 1999 and July 1999. See
Table C.2.
Table C.2: Daily Maximum AOX Discharge t-Test,
Excluding Outliers, Assuming Unequal Variance (Tembec USA)
; Discharges from
1.423
66,941
397.1
16
23
11.259
< 0.001
1.714
< 0.001
2.069
Discharges from
604
19,972
141.3
18
The AOX discharge data were also normalized by their respective wastewater flows to ensure that
the reductions in AOX discharges were not due to decreases in the wastewater flows of the
facility. Similar to the non-flow normalized data, there was a significant decrease in AOX
discharges between April 1999 and July 1999. See Figure C.2 and Table C.3.
25
-------
Figure C.2: Flow Normalized Daily Maximum
AOX Discharges at Tembec USA
60 -
' ' \
f
* - - .
-.*.'*' * - .
-. .-»•* -,
'
M on llorlng Da (e
Table C.3: Flow Normalized Daily Maximum AOX Discharge
t-Test, Assuming Unequal Variance (Tembec USA)
Discharges From
55
323
18
18
19
8.206
< 0.0001
1.729
< 0.0001
2.093
Discharges prom
7/99*02/03
19
20
4.5
18
The AOX daily maximum discharge data were also log-transformed to account for the possibility
that AOX discharges are not normal, but lognormal. Data is lognormal when the log of the data
series is normally distributed. Figures C.3 and C.4 show that the AOX loadings data may be
lognormal, as the log of the loadings data appears slightly more normal than the untransformed
data.
26
-------
Figure C.3: Distribution of Daily Maximum AOX Discharges,
Post Technology Implementation (Tembec USA)
300- 400- 500- 600- 700- 800- 900-
400 500 600 700 800 900 1000
Discharge: (ics/doy)
Figure C.4: Distribution of Log Transformed Daily Maximum
AOX Discharges, Post Technology Implementation (Tembec USA)
Using the log-transformed data, there was still a significant reduction in AOX discharges between
April 1999 and July 1999. See Table C.4.
27
-------
Table C.4: Log Transformed Daily Maximum AOX Discharge t-Test,
Assuming Unequal Variance (Tembec USA)
^
Discharges Ftom
I 1/95 to 4/99
3.17
0.012
0.1
18
34
11.655
< 0.0001
1.691
< 0.0001
2.032
Discharges Froth
7/99 to 2/03
2.77
0.010
0.1
18
The average daily maximum AOX loading prior to the implementation of chlorine free bleaching
was 1,529 Ibs/day with an average flow of 30 million gallons/day. Normalized for flow, the
facility was discharging 50.3 Ibs of AOX per million gallons of flow. The average daily
maximum AOX loading after the implementation of chlorine free bleaching technology was
604 Ibs/day with an average flow of 37 million gallons/day. Normalized for flow, the facility was
discharging 16.25 Ibs of AOX per million gallons of flow. After the implementation of chlorine
free bleaching technology to comply with the pulp and paper effluent guideline, the average daily
maximum AOX discharge was reduced by 34.5 Ibs per million gallons of flow, a 68 percent
reduction.
28
-------
Appendix D
Agency Response
July 7, 2004
WATER
MEMORANDUM
SUBJECT: Draft Evaluation Report: "Effectiveness of Effluent Guidelines Program for
Reducing Pollutant Discharges Uncertain"
FROM: Benjamin H. Grumbles /s/
Acting Assistant Administrator
TO: Dan Engelberg
Director for Program Evaluation Water Issues
Office of Program Evaluation
Office of Inspector General
Thank you for your memo dated June 4, 2002, transmitting the draft report on the subject
evaluation, No. 2003-000228. We appreciate your careful and thoughtful study of the effluent
guidelines program, and we appreciate the opportunity to provide you with the following
comments. The questions you sought to answer are important ones. Your findings highlighted
the critical role of effluent guidelines in improving water quality. For example, the draft report
indicates that permit writers used effluent guidelines to derive 85 percent of the pollutant
discharge limits for the 69 facilities whose permits you reviewed. This evidence of relying on
effluent guidelines is an excellent indicator of the program's success.
In the following paragraphs, I am pleased to respond to the draft report's specific
recommendations, hi addition, an attachment to this memo provides some additional
clarifications.
Recommendation 1 (Report Recommendation 3.1):
Evaluate the effectiveness of effluent guidelines by systematically collecting pollutant discharge
data before and after an effluent guideline is promulgated for a select number of facilities for
each guideline.
I appreciate this suggestion, and acknowledge the value of collecting pollutant discharge data
before and after promulgation of an effluent guideline. There are advantages to being able to show
the benefits generated by effluent guidelines with such a comparison.
29
-------
We need to find a way to collect this data within reasonable resource constraints. Potential costs
are high. For example, the cost to collect and analyze statistically valid data to support the
development of an effluent guideline - the "before" part of the equation - for a single point source
category is in the range of $1 million to $5 million, based on recent experience. (The sampling,
analytical, and reporting costs for a typical wastewater sampling event in developing a guideline
range from $50,000 to $250,000 and depend on which analytes are sampled, the number of
analytes sampled, the number of sample points, the site of the operation, the diversity of the
industry, and the geographical location of the industry.) We estimate that the cost of collecting
effluent sampling data after an effluent guideline has been implemented would be an additional
$200,000 based on an estimated $20,000 per facility and 10 facilities sampled in a given industry.
The number of facilities could change based on several factors, including the quality of data
already available in PCS, the size of the industry, and statistical considerations. The post-
guideline estimated cost is less man the pre-guideline sampling because there would be fewer
analytes to collect and fewer sample points.
In the absence of funds of this magnitude we have identified some alternative approaches that may
have similar merit For example, we could use the data collected during the development of the
effluent guideline and compare that data to self-monitoring data, which facilities report to the
PCS. I have directed the Engineering and Analysis Division (BAD) to conduct a retrospective
analysis of the effectiveness of several effluent guidelines, similar to the approach in your draft
report. Their focus will be on three or four point source categories with recently-collected and
sufficiently-robust data. I have also directed BAD to determine the usefulness of comparing the
sampling data collected during the revision of an effluent guideline with that collected during the
original promulgation. EAD will report back to me on their findings by mid-FY05.
Recommendation 2 (Report Recommendation 4.1):
Develop performance measures that are based on actual pollutant discharge data rather than
discharge estimates gathered before the effluent guideline was effective.
The Office of Water agrees that using actual data is the ideal approach for developing
performance measures. Actual data gathered during effluent guideline development is used as the
basis for projections in the performance measures used currently by the Office of Science and
Technology. These projected values are based on wastewater samples collected from industrial
facilities and carefully extrapolated using statistically-valid methods to determine industry-wide
pollutant loadings before and after an effluent guideline is promulgated. I agree that being able to
validate these projections would enhance the reliability of our performance measures. I believe
that the work EAD is undertaking in response to Recommendation 1 will help address this
recommendation.
30
-------
Recommendation 3 (Report Recommendation 4.2):
Work-with the Office of Enforcement and Compliance Assurance to ensure that it develops a
required data field in the modernized PCS to capture the effluent guideline or guidelines that
apply to each permitted facility.
I agree that adding a data field to PCS would be a useful way to link reporting facilities with the
appropriate effluent guidelines. A new effluent data element has been included in the Office of
Enforcement and Compliance Assurance's (OECA) Integrated Compliance Information System
(ICIS). You may recall the memo J. P. Suarez sent you in November 2002 about this effort.
Adding this information presents the potential to make self-monitoring data available for
systematically evaluating the effectiveness of effluent guidelines, at least for major facilities that
are direct dischargers. The Office of Water is continuing, through the Permitting for
Environmental Results Strategy, to fully develop and implement the tools described in that
Strategy in consultation with OECA, Regions and States.
Clarifications
The draft report covered a range of subjects related to effluent guidelines development and
implementation. We identified some clarifications that we ask you to consider before issuing the
report in final form. If your staff would like to review or discuss these clarifications, please ask
them to contact Pat Harrigan (202/566-1666) in the Office of S cience and Technology and Jan
Pickrel (202/564-7904) in the Office of Wastewater Management.
Attachment
31
-------
32
-------
Appendix E
Distribution
Acting Assistant Administrator, Office of Water (4101M)
Director, Office of Science and Technology (4301T)
Director, Office of Wastewater Management (4201M)
Comptroller (2731 A)
Agency Followup Official (the CFO) (2710A)
Agency Audit Followup Coordinator (2724A)
Associate Administrator for Congressional and Intergovernmental Relations (1301 A)
Associate Administrator, Office of Public Affairs (1101 A)
Audit Liaison, Office of Water (4.101 M)
Inspector General (2410)
33
-------
U S EPA Headquarters Library
' Mai! code 3404T
1200 Pennsylvania Avenue NW
Washington, DC 20460
202-566-0556
------- |