United States
Environmental Protection
Agency
Office of
Solid Waste and
Emergency Response
vvEPA
DIRECTIVE NUMBER:
9481.06(83)
TITLE: Guidance on Implementation of Subpart F Requirements for
Statistically Significant Increases in Indicator Parameter
Values
APPROVAL DATE: 11-30-33
EFFECTIVE DATE: 11-30-83
ORIGINATING OFFICE: office of solid waste
0 FINAL
D DRAFT
STATUS' ^ ;-J- A~-Pending OMB approval .
[ r] B-"Pending "AA-OSWER approval -'=
[ :], C-". For review &/or conmenf":
h J..- tD.T.In development or circulati
REFERENCE (other documents): ; headquarters
OSWER OSWER OSWER
E DIRECTIVE DIRECTIVE
-------
PARTS 264 AND 265 SUBPART F - GROUND-WATER PROTECTION DOC: 9481.06(83)
Key Words: Student's T-Test, Detection Monitoring
Regulations: 40 CFR 265.93, 265.94, Parts 260, 261
Subject: Guidance on Implementation of Subpart F Requirements for
Statistically Significant Increases in Indicator Parameter
Values
Addressee: EPA Regional Directors, Air and Hazardous Materials
Divisions, Regions I-X
Originator: John H. Skinner, Director, Office of Solid Waste
Source Doc: #9481.06(83)
Date: 11-30-83
Summary:
Owners and operators complying with the interim status ground water moni--
toring provisions should have completed several semi-annual comparisons of the-
indicator values. All facilities should now either: (1) have records available
to inspectors showing that the statistical comparisons indicate no significant
change (§265.94(a)); or (2) have notified the Regional Administrator that the
facility may be affecting ground water quality and have submitted a ground
water assessment plan. (§265.93(d)(1) and (2)). There is no provision for
alternative comparison techniques for upgradiant and downgradiant indicator
parameter values; the student's t-test must be used.
The purpose of the assessment plan is to determine the extent of contamina-
tion. There are potential variations with the student t-test which can cause
some degree of "false positives," i.e., indications of possible contamination
where none is actually present. This problem and the problem of incomplete
monitoring data should be overcome through proper assessment plans.
All notifiers who detect some statistically significant differences using
the statistical test prescribed in the regulations must submit such assessment
plans (§265.93 (d)(2)). Where the owner or operator contends that "false
-positives" have resulted from the comparisons, the suitability and quality
indicators should be examined for observable patterns, e.g. an upward trend in
one or more parameters at both upgradiant and downgradiaat wells may indicate
contamination by another source. However, large sustained differences .between
upgradiant and downgradiant values support the indication of contamination.
To avoid some of the problems with comparing replicate values statistically,
it is useful to review plans from notifying facilities, to average the replicated
values in each sample and to compare the average downgradiant value against the
four upgradiant averages. Because this approach does reduce the power of the
test, it should not be allowed to replace the test for regulatory purposes.
Its effect is to raise the threshold level at which an increase becomes signifi-
cant, thus reducing the likelihood of a "false positive," but increasing the
possibility of failing to detect a real contaminant "false negative." For this
-------
Continued from Document 9481.06(83)
reason, this procedure is not recommended as the initial screening statistical
test. It may, however, help you decide which of these notifiers may have some
basis to their claim of "false positive."
Notifiers claiming that the required test has resulted in a false positive
should compare the averages of the replicates and proceed as follows:
1. Where replicate values do not show statistically significant increases,
the assessment plans should focus on whether there has been any discharge
from the facility before attempting to measure concentration rate, and
extent of migration. The content of the assessment plan is the respon-
sibility of the owner or operator and it is his right to propose plans
for demonstrating that a "false positive" was experienced.
2. Where replicate values exhibit significant increases, "false positives"
are less likely to occur. Reviewers should expect these assessment
plans to specify whatever new wells and parameters are necessary to
comply with §265.93(d)(4). While all notifiers can be expected to
take full advantage of the provision of §265.93(d)(6) and show that
hazardous waste constituents have not, in fact, entered ground water, -
this should not be allowed to delay full implementation of the plan.
Theso assessment plans should include a schedule for placement of new
wells as necessary and performance of further analyses to show concen-
tration, rate, and extent of migration of all hazardous waste consti-
tuents (HWCs). At a minimum, where existing.systems are judged adequate
to assure detection, these assessment plans should call for resampling
all downgradiant wells that exhibited a statistically significant
difference, and all upgradiant wells as appropriate for making compari-
sons with downgradiant wells. These samples should be analyzed for the
presence of HWCs.
o If these results do not show significant increases for one or more
HWCs, the owner or operator may return to detection monitoring.
o If these results do indicate that a change has occurr-ed , the owner/
operator must continue implementation of the comprehensive assessment
plan including whatever new wells and new analyses are necessary to
fully characterize concentration, rate, and extent of migration of
all HWCs from the facility.
For more detail on these procedures, please refer to the original memorandum.
Where there is incomplete data, the following statistical analyses should
be performed:
A. Where there is at least one upgradiant and one downgradient well,
statistical comparisons between upgradiant and downgradiant well(s)
should still be made. If data exists for three quarters at a well
that is only seasonally dry, a statistical comparison should still be
made (See B below).
-------
Continued from Document 9481.06(83)
B. If only one quarter is missing, values should be assigned for the four
• missing replicates. These values may be obtained by averaging values
of the other three quarters. If more than one quarter is missing, no
comparisons should be made.
C. If there are missing replicates within a quarter, assign values
obtained by averaging those replicates which are available for that
quarter.
After values have been assigned for the missing data, the owner or operator
should perform the comparisons as if all replicates and quarters were recorded,
and should notify the Regional Administrator if any of these findings indicate
significant increases.
-------
9431.06 (83)
NGV 20 •'?<•' M ^
Guidance on Ir.plT^ntation o? Subp-art T Ht^qui r">~«?nt3 -for
Statistically Siqr.i f icant Increases in Indicator ?.iran«t"r Values
John H. Skinner, Director
Office of Solid Waste CvTE-552)
EPA Regional Directors
Air & Hazardous Materials Divisions
EPA Regions. I - X
Thf* ground-vatsr nor. i to ring provisions of the interim
status re-gulations require comparisons of the upgradisnt and
the dcwngradient indicator parameter values with th«n first-
7«ar, or background, values using the Student's t test (40 CF?.
265.93(b)(2)). Ovners and ooerators complying vith the regula-
tions should have completed the second semi-annual comparison
r- cf the indicator valuas by "ovenber 13, 13S3: currently, all
\ facilities should either: (1) have records available to inspectors
^ showing that the? statistical comparisons indicate no sicr. i f icar.t
\ change ($ 265.34(a)), or (2) have notified the Regional Pdninistratcr
ZI (!?.A. ) that the facility nay be arfoctinq ground-wat^r quality
j: and have subnitted a ground-water assessment plan ( 5263 .93 (d ) ( 1)
« and (2)). There is no provision for alternative comparison
? techniques; Student's c must be used, and assessment plans
j= must be submitted for all facilities found by that tsst to show
vo statistically significant increases.
(J The purpose oc the aasesauwnt plan is to determine the
ri extent of contamination.. As a first step in this determination,
ri sotw* owners and operators are proposing activities aimed at
2 determining whether contamination has truly occurred. T^is
2j ^ay be approoriatc in som>» cases. As stated in r.y r».«»r»o of
1 August 2S, 1933, there are potential problens with the Studen-'s
^ t comparison technique prescribed in the regulations and with
^ -the procedure rGccanended in the draft Ground -Wa t'p'r ' Mon i to r i nq
^ Guidane? for Ovners and Operators of Interim Status .33ciligi?¥
~ . (SW-963), called the Cochran's Approxiaation of the Behrin's
s Fisher Student t test. JThera is a high probability that some*
'.'. owners and operators who will be required to notify will be
•£ reporting "false positives", or indications of possible contamin-
c at ion where none is present. This problem is compounded by
c inadequate monitoring data._ Tor example, it is likely that "any
> claims of a false positive nay be directly attributable to
£ insufficiencies in inonitorir.g programs. This is likely the case
w where owners or operators iapleacnted the minimum system required
2 in the regulations (e.g. *our wells, minimum frequency of
^ sampling) when the site is not extremely sLapie. The guidance
= and the preanbls clearly indicate that rsor* than those rniniau^s
2 will be necessary at the typical facility. No statistical
-------
comparison technique will enable meaningful comparisons of
insufficient data. Where the ownor or operator wants to add
wells or increase sanpling frequency etc. to improve the data
base after a statistically significant increase has been
indicated, they nay do so as part of the ground-water quality
assessment.
This memorandum establishes guidance for reviewing assess-
nent plans at facilities where there ia reasonable probability
that the statistical comparison techniques have resulted in a
false positive; it also provides guidance for determining whether
assessments should b*» required wh«?n op-rators have incomplete
monitoring data.
I. ASSESSING CLAIMS OF "FALSE POSITIVE' RESULTS OP COMPARISONS.
A. All notifiers who detect statistically significant
differences ssuat subnit assessment plans(S_2(55.93 (d H 2 ) ) . The
R.A. should assure that the Agency f iles~~"cohtaTn~aii "relevant
ground-water monitoring data for these sites. The fom attached
at Appendix A nay serve to identify nissing data. When the
owner or operator contends that the Student's t coraparison of
the *265.92 (b)(3) indicators has resulted in a false positive,
the suitability and quality indicators listed at fb)(l) and (2)
of that paragraph should be examined for observable patterns.
Tor ^xarapic, an upward trend in one or norc parannters at bo*.h
up and downgradient wells nay indicate contamination by another
sourca. (See discussion at 45 FT\ 331S4, May 19, 138C. ) However,
large sustained differences between upgradient and downgradient
values may support ihc indication cf ccntaninaticn.
""' B. _It is useful in reviewing plans ifron notifying facilities
to average the replicated values in each sample and to compare the
average downgradient value against the four upgradient averages^
This process avoids sowe of the problems with conparvng the reoii-
cate values statistically, bat it reduces the power of the test.
It has the effect of raising the threshold level at which an
increase becomes significant, thus reducing the likelihood of a
false positive but increasing the possibility of failing to
detect real contamination (a "false negative"). It is because
of this high potential to resalt in false negatives that this
procedure is not. reccsuaended as the initial screening statistical
teat. However, it is highly unlikely that those failing this
comparison of averages test are experiencing a false positlv,. •
due to shortcowings of the statistical tast itself. Instructions
for perfonaing this simple confutation, are provided at Appendix B
-fo this raerao. We are recomaending that all notifiers claiming
that the required test has resulted in a false positive should
cocrpare the averages of the replicates and proceed as follows:
(1) It nay be appropriate for a notifier whose averaged
replicate values do not show statistically significant increases
to focus the early phases of their assessment plans on the
-------
determination of whether ;-'iere has been any discharge frcra the
facility. A substantial -umber of these may be nalso positivra
under the Cochran Test and it is therefore reasonable to allow a
demonstration to that ereact as the first step in their assessment
plan before attempting to ar-asure concentration, rate, and extent
of migration. These demons' ratlor.3 may he based upon an evaluation
of the b(l) and (2) psrawet. -3 (it aay be appropriate to use the
procedure described In AppenU.t B), or upon an evaluation of
additional data obtained by taking more samples, etc. Of course
this only makes senae when it is clear that the monitoring system
is fully capable of detection ($265.90(a)). Facilities with
missing data,Insufficient or poorly located wells or inadequate
frequencies, must resolve these problems in order to make such
demonstrations.
The content of the assessment plan is the responsibility of
the owner or operator; it is his right to propose plans for demon-
strating that a false positive was experienced. Such plans nay
include increased sampling, installation of additional walls,
and selection of statistical comparison tests. In reviewing an
assessment plan, the Agency should assure that the statistical
test proposed is appropriate to the distribution of the data.
Initially, assessment plans to demonstrate 'false positives'
should bo referred to Headquarters (Bumell Vincent, 382-4638)
for review and comnent.
(2) Notifiers whose averaged replicate values also exhibit
statistically significant increases will be loss likely" to have
false positives. Reviewers should expect these assessment plans
to s-scify whatever r.-w wells and parameters ar:> necessary to
comply with <265. 93(d ) ( 4 ) . Xfrhile all notifiara can be expected
.•to take full advantage of the provision of 5265.93 (d) (5) and show
that hazardous waatr» constituents have not, In ffict, entered around
water, this should not be allowed to delay full implementation
of the plan. These assessment plans should include a schedule
for placement of new wells as necessary and performance of further
analyses to show concentration, rata, arid extent of migration
of ail hazardous waste constituents (HWCs) ySmjCs arc defined
in Part 260 as all constituents in Part 251 Ann-n^dix VII plus
all constituents listed in Table I of
-------
(a) If these results do not show significant increases for
one or acre HVCs, the owner or operator may return to detection
raonitoring. The next soni-annual sar.pl ir.q would be 5 months
from this negati%re determination.
( (b) If these results do indicate that a change has occurred,
| the owner/operator raust continue implementation of the comprehensive
j assessnent p_lan including whatever new wells and new analyses are
\ necessary to"" fully characterize concentration, rate, and extent
', of migration of all HWCs from the facility.
II. STATISTICAL ANALYSES OF INCOMPLETE DATA
Owners and operators are not relieved of the resoonsibilitv
to coraply with the requirements for statistical analyses and
assesaaents by their failure to fully comply with the aonitoring
requirements. It is appropriate that all owners or operators
perform the statistical analyses, when possible, even though
they may possess incomplete data (i.e., lesa than the aininun
required by the regulations). Incomplete data may result from:
(1) too few wells (i.e., leas than 4), (2) dry wells, (3) omitted .
paraoeters, (4) lost or ruined saaples, (5) insufficient replicates,
or (6) raisaing quarters. Where too few wella exist or wells are
dry for one or more quarters (cases (1) or (2) listed above),
the owner or operator is responsible to establish operable wells.
In the case of a dry well, possible solutions include: different
screening intervals, different well depth, use of an unclogging
tachnique (e.g., acid cleaning or detonation) vhsrs clogging is
a problem, or new w
------- |