600281124
A STATISTICAL EVALUATION OF OHMSETT TESTING
DJane M. Foster
I'.S, Environmental Proiection Agency
Ecison. N'e\v "ersev 0££37
and
Sol H. Schwartz and Gary F. Smith
Mason & Hanger-Silas Mason Co., Inc.
Leonardo, New Jersey 07737
Contract No. 68-03-2642
Project Officer
John S. Farlow
Oil and Hazardous Materials Spills Branch
Municipal Environmental Research Laboratory
Edison, New Jersey 08837
MUNICIPAL ENVIRONMENTAL RESEARCH LABORATORY
OFFICE OF RESEARCH AND DEVELOPMENT
U.S. ENVIRONMENTAL PROTECTION AGENCY
CINCINNATI, OHIO 45268
-------
DISCLAIMER
This report has been reviewed by the Municipal Environmental Research
Laboratory, U.S. Environmental Protection Agency, and approved for publication.
Approval does not signify that the contents necessarily reflect the views and policies
of the U.S. Environmental Protection Agency, nor does mention of trade names or
commercial products constitute endorsement or recommendation for use.
-------
FOREWORD
The U.S. Environmental Protection Agency was created because of increasing
public and government concern about the dangers of pollution to the health and
welfare of the American people. N'cxious air, foul water, and spoiled land are tragic
testimonies to the deieriorafion of our natural environment. The complexity of that
environment and the interplay of its components require a concentrated and integrated
attack on the problem.
Research and development is that necessary first step in problem solution; it
involves defining the problem, measuring its impact, and searching for solutions. The
Municipal Environmental Research Laboratory develops new and improved technology
and systems to prevent, treat, and manage wastewater and solid and hazardous waste
pollutant discharges from municipal and community sources, to preserve and treat
public drinking water supplies, and to minimize the adverse economic, social, health,
and aesthetic effects of pollution. This publication is one of the products of that
research and provides a most vital communiations link between the research and the
user community.
This report describes a program to statistically analyze test parameters used to
evaluate the oil and hazardous material spill control devices at the U.S. EPA's Oil and
Hazardous Materials Simulated Environmental Test Tank. Based on results presented
here, improved testing and evaluation of equipment can be developed. Further
information may be obtained through the Solid & Hazardous Waste Research Division,
Oil and Hazardous Materials Spills Branch, Edison, New Jersey.
Francis T. Mayo
Director
Municipal Environmental Research Laboratory
Cincinnati
in
-------
ABSTRACT
This program was cond'jcied 10 provide a statistical evaluation of performance
data generated at the L'SEPA's O>i and >;a :.-_r::o'js Materials Simulated Environmental
Test Tank (OHMSETT). The objective v. as to Investigate the value of replicate testing
in order to develop efficient test pro^ra;; s v'~: give the most reliable information
from the minirr'jrn nu:r:ber of Tests.
This study was set up in two separate programs, each consisting of 24 tests:
one program in which 2^ different co'-ditions were tested and a second program in
which three replicates each of eight different conditions were tested. A comparison
was then possible between the two types of programs. The 3-replicate test matrix was
duplicated to produce six replicates and the validity of the non-replicate and 3-
replicate programs was evaluated with respect to the 6-replicate data.
Parameters affecting device performance studied in this program were tow
speed, wave condition, oil type, and oil slick thickness. These parameters were tested
at various levels, and device performance was evaluated in terms of throughput
efficiency (the ratio of oil collected to oil encountered).
Comparisons between point estimates and confidence intervals, graphic trends
and analysis of variance were all examined. The results of this program indicate a
need for replicate testing to provide accurate estimates of performance parameters,
significant effects and performance trends.
These results are specific to the LPI-OSED skimmer and need not apply to all
equipment tested at OHMSETT.
This report was submitted in partial fulfillment of Contract No. 68-03-26^2, by
Mason & Hanger-Silas Mason Co., Inc., under the sponsorship of the U.S. Environmen-
tal Protection Agency. This report covers the period 31 July 1978 to 18 August 1978
and work was completed as of 13 October 1978.
IV
-------
CONTENTS
Foreword iii
Abstract iv
Figures and Tables vi
Abbreviations and Symbols vii
Conversion Factors viii
1. Introduction 1
2. Conclusions 2
3. Recommendations 3
4. Test Plan it
5. Procedures 8
6. Results 10
7. Discussion 23
Appendices
A. OHMSETT Description 25
B. Test Oils 27
C. Raw Data 28
-------
FIGURES
Number
1 Test sst-up for the LP!-OStD skimmer 9
2 Graphic comparison of 6-replicate vs. non-replicaie
data, calm water, light oil 13
3 Graphic comparison of 6-replicate vs. non-replicate
data, cairn water, heavy oil 14
i- Graphic comparison of 6-rsplicate vs. no'-.-rep'icate
data, 6.3 m harbor chop, light oil 15
5 Graphic comparison of 6-replicate vs. nor.-replicate
data, 0.3 m harbor chop, heavy oil 16
6 Graphic comparison of 6-replicate vs. 3-replicate
data, calm water, light oil 19
7 Graphic comparison of 6-replicate vs. 3-replicate
data, calm water, heavy oil 20
8 Graphic comparison of 6-replicate vs. 3-replicate
data, 0.3 m harbor chop, light oil 21
9 Graphic comparison of 6-replicate vs. 3-replicate
data, 0.3 m harbor chop, heavy oil 22
TABLES
Number
1 Non-replicate Test Matrix 6
2 Replicate Test Matrix 7
3 95% Confidence Intervals 11
4 Projected Optimal Sample Sizes . r 12
5 Analysis of Variance-Summary of Effects 17
6 Means of 3-Replicate Programs ; 18
VI
-------
ABBREVIATIONS
ABBREVIATIONS AND SYMBOLS
cm
dynes/cm
HC
IFT
m
m/s
mm
N/m
C.V.
-harbor chop wave condition
-interfacial tension
-meters
-meters per second
-millimeter
-Newton per meter
-coefficient of variation
SYMBOLS
y
X
a
s
n
a
-population mean
-sample mean
-population standard deviation
-sample standard deviation
-number of observations
-the probability of rejecting the null hypothesis when it's true
-percent
VII
-------
LIST OF CONVERSIONS
METRIC TO ENGLISH
To convert from
Multiply by
jouie
joule
kilogram
meter
meter -
meter_
meter-
rneter.,
meter
meter/second
meter^econd
rneter -/second
meter ./second
meter /second
newton
watt
ENGLISH TO METRIC
centistoke
degree Fahrenheit
erg
foot-
foot"1
foot/minute
foot /minute
foot-pound-force
gallon (U.S. liquid)
gallon (U.S. liquid)/minute
horsepower (550 ft Ibf/s)
inch-
incrT
knot (international)
liter
pound force (Ibf avoir)
pound-mass (Ibm avoir)
pound/foot
pourid-rri25.s ('DTI avoir)
foot
inch-
foot-
inch
gallon (U.S. liquid)
liter
foot/minute
knot
centistoke
foot /minute
gallon (U.S. liquid)/minute
pound-force (Ibf avoir)
horsepower (550 ft Ibf/s)
meter /second
Celsius
joule
meter-
meter
meter /.second
meter /second
joule _
meter-
meter /second
watt
meter-
meter
meter
newton
kilogram
pascal
2.205
3.2S1
3.937
1.076
\.5t9
2.6*2
1.000
1.969
1.000
2.119
1.587
2.248
1.3*1
£-07
E-01
E-00
E+00
E+01
E+01
E+03
E+02
E+03
E+02
E+00
E+06
E+03
E+0*
E-01
E-03
1.000 E-06
t = (tp-32)/1.8
IVOOO E-07
3.0*8 E-01
9.290 E-02
5.080 E-03
*.719 E-0*
1.356 E+00
3.785 E-03
6.309 E-05
7.*57 E+02
2.5*0 E-02
6.*52 E-0*
5.1** E-01
1.000 E-03
*.**8 E+00
*.535 E-01
*.788 E+01
VIII
-------
SECTION 1
INTRODUCTION AND BACKGROUND
In 197^, trie U.S. environmental Protection Agency began evaluating the
performance of oil and K?. ~arcous materials spill cleanup equipment at their OHMSETT
test facility in Leer'-; do. Ne\v "ersey. Tlv-se projects v.ere the direct result of the
Agency's determination to further the technology required to combat oil and hazardous
material waterborne pollution. By providing a facility to simulate environmental
conditions, equipment previously untested for oil spill containment and removal
potential could now be evaluated objectively. Together with qualitative information
such as extensive photographic and video coverage, it became possible to quantify the
performance of a wide range of prototype and production equipment.
The measured recovery parameters of oil recovery rate, recovery efficiency
(percent oil in total recovered mixture), and throughput efficiency (ratio of oil
collected to oil encountered, expressed as a percent) have served to define overall
device potential, performance trends, and sensitivities to various simulated
environmental conditions. These recovery parameters have historically been point
estimates (an estimate given by a single number). Due to cost and time limitations, it
has not always been possible to produce replicate data necessary to establish
confidence intervals.
In April 1975, the operating staff proposed to incorporate into the on-going
USCG-EPA hazardous materials project a method to determine interval estimates and
to produce a statement of their reliability. The results of that program are given in
Reference 1.*
This study investigates the value of replicate testing in developing an efficient
test program that gives the most reliable information from the minimum number of
tests. This study was funded and conducted in August 1978.
1. McCracken, W.E. and S.H. Schwartz. Performance Testing of Spill Control
Devices on Floatable Hazardous Materials, EPA-600/2-77-22, U.S. Environmental
Protection Agency, Cincinnati, Ohio, 1977. 139 pp.
-------
SECTION 2
CONCLUSIONS
The following conclusions are derived from this test program comparing
replicate vs. non-replicete testing at OHN'ScTT.
Replicate nesting is essential TO establish reliable e=.timai.es of performance
parameters and to determine the degree of precision of these estimates.
Replicate testing is necessary to estimate the significance of various control-
lable parameters on device performance.
Replicate testing is necessary to accurately establish performance trends over
various levels of a parameter.
-------
SECTION 3
RECOMMENDATIONS
Future testing at OHMSETT should include preliminary replicate programs to
establish best and worst c-.se confidence intervals and determine optimal performance
parameter considering precision.
Since test results serve to establish future research, development and procure-
ment programs, OHMSETT results should be defined in terms of their statistical
reliability and should supplement qualitative evaluations of device performance.
Test reports should include all available information concerning accuracy and
precision of laboratory procedures currently existing as control chart information and
should detail information concerning the operator control of oil distribution.
A standardized format for presenting such information should be established
and included as an Appendix to all reports.
-------
SECTION 4
TEST PLAN
GENERAL CONSIDERATIONS
The objective of an efficient test program is to get the most reliable
information from the rniriirnurn number of tests. \VhiJe covering the widest range of
test conditions, it is important that the reliability of the information obtained is not
sacrificed.
To evaluate means of achieving these objectives, this study has been set up in
two separate programs:
1. A non-replicate program of 24 tests run with 24 different conditions,
and,
2. A replicate program of 24 tests where three replicates each of eight
different conditions are tested.
A comparison can then be made as to which type of program provides the
greatest amount of reliable information.
Parameters affecting device performance may be either operator controllable
or not controllable in actual practice. Information on device response to controllable
parameters, such as forward speed and oil slick thickness, is necessary for optimal oil
collection. Information on device response to parameters beyond control, such as
wave condition and oil type, is beneficial when selecting the appropriate device for
particular oil spill conditions.
NON-REPLICATE TEST PLAN
It was chosen to test device performance with:
1. wave condition at 2 levels (calm and .3 m harbor chop)
2. oil type at 2 levels (light and heavy)
3. tow speed at 3 levels (.76 m/s, 1.02 m/s, and 1.52 m/s), and
4. slick thickness at 2 levels (3 mrn and 6 in in).
Each level of every parameter appears with each level of every other parameter
exactly once.
-------
THRbE-Kt--PLICA i E TEST PLAN
To allow for three replicates of each condition, the program was restricted to
testing with:
1. wave condition at 2 levels (cairn and .3 m harbor chop)
2. oil type at 2 levels (light and heavy)
3. tow sp-ed at 2 levels (1.02 rn/s and 1..52 m/s), and
4. slick thickness held constant at 3 mm.
Each level of e\ery parameter appears v Jth '~ach level of every other parameter
exactly three times.
Since additional time became available, it became possible to repeat the 3-
replicate test matrix, thus producing 6 replicates of each condition. This greatly
improves the estimate of the population's mean (y) and standard deviation (a, and is
extremely beneficial in estimating confidence intervals and projecting optimal sample
size.
A duplication of the 3-replicate matrix also makes possible a comparison
between two 3-replicate programs. This is of interest should it be found that (for this
particular skimmer) three replicates are sufficient for estimating throughput efficien-
cy within a desired bound (such as ± 5) at the 95% confidence level. This provides
additional information as to the reproducibility of test programs.
-------
TABLE 1. NON-REPLICATE TEST MATRIX
Test
no.
1
2
3
1
5
6
7
8
9
10
11
12
13
in
15
16
17
18
19
20
21
22
23
2*
Tow speed
(m/s)
0.76
1.02
1.52
0.76
1.02
1.52
1.02
1.52
0.76
1.02
1.52
0.76
1.52
0.76
1.02
1.52
0.76
1.02
0.76
1.02
1.52
0.76
1.02
1.52
Wave
condition
calm
calm
calm
.3 m HC
.3m HC
.3 rn HC
calm
calm
calm
.3 m HC
.3 m HC
.3 m HC
calm
calm
calm
.3 m HC
.3 m HC
.3m HC
calm
calm
calm
.3 m HC
.3 m HC
.3m HC
Oil
type
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
light
light
light
light
light
light
light
light
light
light
light
light
Slick thickness
(mm)
3
3
3
3
3
3
6
6
6
6
6
6
3
3
3
3
3
3
6
6
6
6
6
6
-------
TABLE 2. REPLICATE TEST MATRIX
7'rSt
1A
2A
IB
3A
4A
3B
2B
1C
2C
4B
3C
4C
5A
6A
5B
7A
8A
7B
6B
5C
6C
8B
7C
8C
Crn/sf ^
1.02
1.52
1.02
1.52
1.02
1.52
1.52
1.02
1.52
1.02
1.52
1.02
1.02
1.52
1.02
1.52
1.02
1.52
1.52
1.02
1.52
1.02
1.52
1.02
cono'i'Lion
calm
calm
cairn
.3 m HC
.3 m HC
.3m HC
calm
calm
calm
.3 m HC
.3 m HC
.3 m HC
calm
calm
calm
.3 m HC
.3 m HC
.3 m HC
calm
calm
calm
.3 m HC
.3 m HC
.3 m HC
Oil
type
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
light
light
light
light
light
light
light
light
light
light
light
light
Slick thickness
(mm)
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
This matrix was repeated with A, B, and C replaced with D, H, and F respectively.
-------
SECTION 5
TEST PROCEDURES
i'ie advancing skimmer tested \\as the LPI Corporation's OSED oil skimmer.
The device recovers spilled oil by gliding an encountered oil/water mixture along its
inclined bow and into the skimmer's entry slot, where the oil rises to the collection
area and \vater exits through the bottom. The device was positioned between the main
and auxiliary bridges and on trie west sloe of the video truss as shown in Figure 1. The
centerline was adjusted to match the centerllne of the oil distribution manifold,
approximately 7.3 m from the west :ank wall. Oil was guided to the bow of the device
as a 2.9-m wide slick at thicknesses of 3 and 6 mm. The tow speeds were selected on
the basis of previous testing of the LPI skimmer. They were 0.76, 1.02, and 1.52
meters per second. Surface conditions were calm water and 0.3-m harbor chop (HC),
and oil types included heavy and light oils with nominal viscosities of 570 and 150
centistokes, respectively. These viscosities relate to actual test tank water tempera-
ture and were extrapolated from a standard viscosity-temperature chart of the
American Society for Testing Materials (D-3^1).
The skimmer was brought to speed prior to oil distribution. On signal, a preset
rate of oil distribution was begun and continued for a distance of 3\.k m, after which
the tow speed was gradually reduced to minimize oil loss from the holding area of the
device. Collected fluid was transferred by pump to translucent barrels for subsequent
measurement. This operation included decanting free water, which rapidly settled,
and mixing the remaining mixture for grab sampling. The samples were then mixed
with an equal volume of water-saturated toluene and centrifuged for a period of 30
minutes. The percent water content was then multiplied by the volume of oil-water
mixture collected to obtain an estimate of the total oil collected.
-------
•Oil
'/E
U/W Video
Camera
1. Oil Distributor
2. Test Director
3. Test Fngineer
4. LPI Representative
5. Fire Hose Operators
6. Photographer
7. Oil Collector
8. VDU/rliter Operator*
9. Chemistry Lab Tech.*
10. Video Technician*
11. Bridge Operator*
Video Truss
>— Auxiliary Bridge
Figure 1. Test set-up for LPI -OShD skiir^er USEPA test program.
9
-------
j
SECTION 6
TEST RESULTS
Data were collected on both oil distribution and throughput efficiency for
statistics] analysis. The raw data are presented in Appendix C.
OIL DISTRIBUTION
Test fluids are typically distributed under operator control at OHi.'SETT from
onboard storage tanks through positive displacement pumps and meters equipped with
volume totalizers and rate tachogenerators. The true thickness of the slicks formed
on the water were not measured; the average thickness for each run was calculated
using the oil distribution flowrate, the forward speed, and the nominal 2.9-m slick
width. Accuracy and precision statements on this operation are based on the
calculated average slick thicknesses.
Sixty tests were run with a nominal oil slick thickness of 3 mm. The mean of
the average slick thicknesses was 3.06 mm, with a standard deviation of 0.120 rnm
The twelve 6 mm tests displayed a similar degree of repeatability with a mean
slick thickness of 5.92 mm and standard deviation of 0.298 mm.
THROUGHPUT EFFICIENCY-- REPLICATE VERSUS NON-REPLICATE
The best available estimate of throughput efficiency for a given tested
condition is given by the mean of the six replicates. The reliability of the non-
replicate and 3-replicate data is, therefore, evaluated with respect to the 6-replicate
program.
Confidence Intervals
__ For each of the eight conditions occurring in the 6-replicate program, the mean
(X) and standard deviation (s) is calculated. To establish the precision of these
estimates of throughput efficiency, a 95% confidence interval is computed for each
mean. There is a 95% probability that the true mean, y, lies within the limits
established by this interval. The 95% confidence intervals were determined by the
following formula:
C.I. - x±t(0-975>n_;
10
-------
v, Nere n - nurroer of observations
t = Student's t-distribution evaluated at a = .05 (two-tailed) with n-1
degrees of freedom
The mean, standard deviation and 95% confidence intervals for each condition are
presented in the following table.
TABLE 3. 95% CONFIDENCE INTERVALS FOR THROUGHPUT EFFICIENCY
Tc\v speed
r~- /o
'.' ' ••/ S/
1.02
1.54
1.02
1.54
1.02
1.5*
1.02
1.5*
Oil
type
light
light
heavy
heavy
light
light
heavy
heavy
Wave
condiijon
calm
calm
calm
calm
.3 rn HC
.3 rn HC
.3 m HC
.3 m HC
X
93.6
78.3
95.2
89.5
34.6
37.7
45.7
46.0
s
10.07
4.32
8.56
7.95
7.68
3.91
6.82
6.95
95% Confi
interval
83.1 to
69.3 to
86.2 to
81.2 to
26.6 to
33.6 to
37.2 to
37.4 to
dence
104.2
78.4
104.2
97.8
42.7
41.8
54.2
54.6
These intervals are presented graphically in Figures 2 through 5 in addition to
the individual data points.
The 95% confidence intervals ranged in size from ±4.1% to ± 10.6% throughput
efficiency, with an average interval size of ±7.7%. It is of interest to know how many
replicates would be needed in order to trim this interval down to ± 5% while remaining
at the 95% confidence level. The projected necessary sample size, n, can be estimated
using the following equation:
1? = (ts/B)2
where ft = projected number of necessary replicates
t = Student's t-distribution at t/, /_ ,x
(l-a/2, n-1)
(in this case a = .05)
s = sample standard deviation
B = desired bound on the error estimate (in this case B = 5)
The results are presented in the following table.
11
-------
TABLE k. PROJECTED SAMPLE SjZES,1\, NECESSARY TO TRIM CONFIDENCE
INTERVAL SIZE TO ±5% WHILE REMAINING AT 95% CONFIDENCE LEVEL
To\v speed
(m/s)
1.02
1.52
1.02
1.52
1.02
1.52
1.02
1.52
Oil
type
Jight
light
S-:.avy
^v.y
i^ht
J.;£l''t
re-ivy
heavy
Wave
condition
calm
Cc.jin
cairn
c-lM-i
.3 rr, >C
7 - , : , r^
,j n. r.iv-
.3 rn HC
.3 m hC
Interval size
at n = 6
10.6
-------
a
z
LLl
K
O
H
UJ
Qi.
- NON-REPLICATE DATA.
- REPLICATE DATA.
MEAN OF NON-REPLICATES.
~ MEAN OF REPLICATES.
K CONFIDENCE INTERVAL.
— -•
CD
D
O
a
Q
1.2S
TOW SPEED CM/S)
1.5
Figure 2. Graphic comparison of 6-replicate vs. non-replicate data, calin water, liqht oi!
-------
.
• I • » • I
- NGN-REPLICATE DATA.
~ REPLICATE DATA.
MEAN OF NQN~REPLICA7£3.
MEAN OF REPLICATES.
QS% CONFIDENCE INTERVAL.
>
£
!-i
q
5~i
IL,
J-
0,
O
/•s/
«~v
X
t-
wl-
a
a
--A
1.25
TOW SPEED CM/S>
1.5
Figure 3. Graphic comparison of 6-replicate vs. non-replicate data, cairn water, heavy oil
-------
o>
O
T*
rfC*.
!ju
H
u_
CD
D
c
Si-
- NOiN-REPLICATE DATA.
- REPLICATE DATA.
iXEAN OF NON-p:EPLICATES.
MEAN OF REPLICATES.
QS'A CONFIDENCE INTERVAL
TOW
1.25
CM/S)
'..5
Figure 4. Graphic comparison of non-replicate vs. 6-repiicate data, 0.3-m harbor chop, light oil,
-------
O
sr\
2(4-
G
A- NCN-REPLICATE DATA.
a- REPLICATE DATA.
NCN-REPLICATE DATA.
— REPLICATE DATA.
$H 85K CONFIDENCE INTERVAL.
'—;
d
H
n.
x
CO
D
O
w
//••// //
^M
1
ccl
-
1.25
TOW SPEED CM/S>
1.5
Figure 5. Graphic comparison of non-replicate vs. 6-replicate data, 0.3-m harbor chop, heavy oil.
-------
T
respective ccodit!;>':$.
source, where necessary.
I wo oe^rees
v. '.re
d from the residual
For each set of data (non-replicate, 3-re-plicate set and 6-replicate sets) a
cross-classification analysis of variance (ANOVA) was performed to test the signifi-
cance of each factor on device performance as determined by throughput efficiency.
The sources of variation tested were the main factors: tow speed, wave condition, oil
type, and slick thickness: and in the replicate sets, the interactions between tow speed
and wave condition, tow speed and oil type, and tow speed and slick thickness, wave
condition and oil type, etc. (It is only possible to test the significance of interactions
when replicate data is available). Three-way and four-way interactions are not of
particular interest a"c)ec with the r?ciduaj source.
Th = rssuh> of the four ANOVA's are p-^e.v.ed in the following table.
TABLE 5. ANALYSIS OF VARIANCE - SUMMARY OF SIGNIFICANT EFFECTS AT
THI
% CONFIDENCE LEVEL
Source Non-replicate
Tow speed
Wave condition *
Oil type
Slick thickness *
Tow speed x wave** not tested
3-replicate
Set I
•*
*
not tested
*
3-replicate
Set II
*
*
*
not tested
*
6-replicate
*
*
*
not tested
*
^Significant at the 95% confidence level
**No other interactions were found significant at a = .05 for any data set
A comparison can now be made between the ANOVA's for the replicate vs.
non-replicate programs. The non-replicate data were not sufficient to detect a
significant effect (at a = .05) of tow speed or oil on throughput efficiency. While one
of the 3-replicate data sets failed to detect the significance of tow speed, oil type was
found to be very significant (at a = .01) in each replicate ANOVA. This effect of_oil
type can be seen by referring back to Table 3, where mean throughput efficiency (X) is
consistently higher in heavy oil than in light oil. Note, however, that oil viscosity
varied systematically over time (see Appendix B), and the possible effects of this have
not been addressed.
A tow speed by wave interaction was found significant in all replicate
ANOVA's. This implies that wave condition should be taken into account when
determining optimal tow speed for device performance. Referring to the graphs in
Figure 2, this interaction is illustrated. Whereas throughput efficiency in calm water
decreases as tow speed is increased from 1.02 to 1.52 rn/s, in a .3 m harbor chop the
device performs as well at 1.52 m/s as at 1.02 rn/s.
17
-------
3-R^'i'^i.ej.^. Nco-Replicate
The non-replicate program has not proven to be a sufficient and reliable
evaluation of device performance when compared to the 6-replicate program in either
analysis of variance, 95% confidence intervals, or graph analysis. However, the 6-
replicate program required twice the number of tests as in the non-replicate program.
For this reason, a comparison follows between the non-replicate program and the 3-
replicate program, each consisting of 24 tests.
As mentioned in the previous section, the second 3-replicaie program (tests
with the suffixes O. F, sod F) detected all of the significant effects found in the 6-
replicate program. \vhi'e :he first 3-rtp'iCc ;e set (t''-?ts with the suffixes A, B. and C)
failed only to delect the -to;', speed effect. The non-repljcate program, however,
failed to detect both the ".c .v sooed effect b~:d the highly significant oil type effect.
The 3-replicate moans provide a much better estimate of throughput efficiency
than the non-replicate data points, when compared with the 6-replicate estimates.
Three out of eight non-replicate observations fall outside of the 95% confidence limits
established by the 6-replicate data, whereas, for both sets of 3-replicate data, all
means fall within the 95% confidence intervals. These data are presented in the
following table.
TABLE 6. MEANS OF 3-REPLICATE PROGRAMS
3-Replicate 3-Replicate 6-Replicate
Tow speed Oil
(m/s)
1.02
1.52
1.02
1.52
1.02
1.52
1.02
1.52
type
light
light
heavy
heavy
light
light
heavy
heavy
Wave
condition
calm
calm
calm
calm
.3m HC
.3 m HC
.3 m HC
.3 m HC
Set I
X
96.0
76.6
99.3
96.6
38.5
39.5
46.2
47.6
Set II
X
91.3
71.1
91.1
82.4
30.8
36.0
45.0
43.5
95% confidence
interval
(81.1,
(69.3,
(86.2,
(81.2,
(26.6,
(33.6,
(37.2,
(37.4,
104.2)
78.4)
104.2)
97.8)
42.7)
41.8)
54.2)
54.6)
It should be noted, however, that there is an average bias of 696 between the
two 3-replicate sets of means. Despite this bias, which is clearly depicted in the
graphs in Figures 6 through 9, the 3-replicate means closely follow the trends
established by the 6-replicate means. This was not the case with the non-replicate
data.
18
-------
A_ 6-REPLICATE DATA.
—-MEAN OF 6-REPLICATES.
MEAN OF FIRST S-REPLICATLCo.
MEAN OF SECOND 3-REPLICATES.
'%%$% 95% Confidence interval.
A
'A
1.25
TOW SPEti) CM/S5
1 .b
Figure 6. Graphic comparison of 6-replicate vs. 3-replicate data, cairn water, light oil.
-------
Q-
- 6-REPLICATE DATA.
—'MEAN OF 6-REPLICATES .
MEAN OF FIRST G-REPLICAT;^.
MEAN OF SECOND o-RcPLICAY£S,
Confidence interval.
NJ
O
H
Z>
A
*~W
00'
ll
TOW
1.25
CM/S)
l.S
Figure 7. Graphic comparison of 6-replicate vs. 3-replicate data, calm water, heavy oil,
-------
A- 6-REPLICATE DATA.
™- H£AN OF 6-REPLICA7cS.
,v,cAN OF FIRST S-KEPL^ATSS.
MEAN OF SECOND 3-REPLICATES,
95% Confidence Interval.
>
^ ; ^^ A
i: .^ N,\^
& v"- ;^ .................. A
1 1.25 1.5
TOW SPEc«> CJVS>
Figure 8. Graphic comparison of 6-replicate vs. 3-replicate data, 0.3-m harbor chop, light oil.
-------
A- 6-REPLICATE DATA.
- MEAN OF 6~REF'LICA7.7S
MEAN OF FIRST 3-R^L.
MEAN OF SECOND 3-REP
95% Confidence Interval.
X;
O
A
..4
1 1.25 ', .S
TOW SPEE*> CM/S)
Figure 9. Graphic comparison of 6-repIicate vs. 3-replicate data, 0.3-ni harbor chop, heavy oil,
-------
SECTION 7
DISCUSSION
The data c'eariy eit-.blisr the ~eed for statistical evaluation of OHNAStTT
results. Vi'hile it may rot be rossible, due to cost and time limitations, to reproduce
tftch test co'>diticr; ?ix or • ~iore times throughout a program, it is evident that certain
renditions must be repHrated. This may take the form of establishing confidence
estimates based on replicate testing of proposed best and worst conditions at the
outset of all test programs. \Vhile this procedure will not produce absolute estimates
of confidence intervals for all test conditions, it will establish an overall range of
intervals on which to base statements of precision and accuracy for each test device.
The use of preliminary replicate runs will also produce information on the
precision of each performance parameter. By comparing the coefficient of variation
for the estimated recovery rate, recovery efficiency, and throughput efficiency, the
test engineer will be able to establish which parameter has the greatest reliability, and
might therefore be used most effectively to establish device performance and trends.
The following is an example of results from such a preliminary test sequence.
Test Device-- Advancing Skimmer
Slick Thickness-- 3 mrn
Test 1-- 0.51 m/s, heavy oil, calm water
A
B
C
D
E
F
mean, X
standard
deviation, s
coefficient of
variation, s/X
range of 9596 confidence
limits about X
Throughput
Efficiency (%)
85
87
92
82
90
94
88.3
4.502
5.1%
±4.73
Recovery
Efficiency (%)
90
92
89
91
90
93
90.8
1.472
1.696
±1.55
Recovery
Rate (m^/s)
3.2
3.0
2.9
3.5
2.7
3.0
3.1
0.274
8.8%
± 0.29
-------
This information reveals that, for this example, recovery efficiency has the
greatest precision as an estimator of device performance. Depending upon how well
this parameter reflects a true measure of device performance, it may be considered
the choice parameter to be observed throughout the test program. The same number
of replicates should also be conducted for a potential worst case test condition, and
these data considered when choosing the optimal performance parameter.
c
-------
APPENDIX A
OHV.SETT TEST FACILITY
Figure A-l. OHMSETT Test Facility.
GENERAL
The U.S. Environmental Protection Agency maintains the Oil and Hazardous
Materials Simulated Environmental Test Tank (OHMSETT) located in Leonardo, New
Jersey (Figure A-l). This facility provides an environmentally safe place to conduct
testing and development of devices and techniques for the control of oil and hazardous
material spills.
The primary feature of the facility is pile-supported, concrete tank with a
water surface 203 meters long by 20 meters wide and with a water depth of 2A
meters. The tank can be filled with fresh or salt water. The tank is spanned by a
bridge capable of exerting a force up to 151 kilonewtons, towing floating equipment at
speeds to 3 meters/second for at least 'i5 seconds. Slower speeds yield longer test
runs. The towing bridge is equipped to Jay oil or hazardous materials on the surface of
25
-------
the water several meters ahead of the device being tested, so that reproducible
thicknesses and widths of the test fluids can be achieved with minimum interference
by wind.
The principal systems of the tank include a wave generator and beach, and a
filter system. The wave generator and adsorber beach have capabilities of producing
regular waves to 0.7 meter high and to 2S.O meters long, as well as a series of 1.2
meters high reflecting, complex waves meant to simulate the water surface of a
harbor or the sea. The tank water is clarified by recirculation through a 0.13 cubic
meter/second ciatornaceous earth filter system to permit full use of a sophisticated
underwater photography and video imagery system, and to remove the hydrocarbons
that enter the tank water as a result of testing. The lowing bridge has a built-in
skimming barrier which can rr>o\e oil onto the North erid of the tank for cleanup and
recycling.
When the tank must be emptied for maintenance purposes, the entire water
volume, or 98^2 cubic meters is filtered and Treated until it meets all applicable State
and Federal water quality standards before being discharged. Additional specialized
treatment may be used whenever hazardous materials are used for tests. One such
device is a trailer-mounted carbon treatment unit for removing organic materials from
the water.
Testing at the facility is served from a 650 square meters building adjacent to
the tank. This building houses offices, a quality control laboratory (which is very
important since test fluids and tank water are both recycled), a small machine shop,
and an equipment preparation area.
This government-owned, contractor-operated facility is available for testing
purposes on a cost-reimbursable basis. The operating contractor, Mason & Hanger-
Silas Mason Co., Inc., provides a permanent staff of fourteen multi-disciplinary
personnel. The U.S. Environmental Protection Agency provides expertise in the area
of spill control technology, and overall project direction.
For additional information, contact: Richard A. Griffiths, OHMSETT Project
Officer, U.S. Environmental Protection Agency, Research and Development, MERL-
Ci, Edison, New Jersey 08837, 201-321-6629.
-------
APPENDIX B
The following Table details the physical properties of test oi!s used during this
program.
TABLE B-l. TEST OIL PROPERTIES
Designation
Date
Circo X Heavy (2 Aug)
Circo X Heavy (3 Aug)
Circo X Heavy (18 Aug)
Circo 4X Light (31 3uly)
Circo 4X Light (1 Aug)
Circo 4X Light (14 Aug)
Circo 4X Light (16 Aug)
Viscosity
cSt (a °C
697 (3
772 @
900 @
14.7 @
16.1 @
16.8 @
19.8 @
22.1
23.3
20.0
24.1
22.8
24.6
19.9
Specific
Gravity
0.936
0.937
0.938
0.898
0.900
0.901
0.904
Surface
Tension
dynes/cm
34.2
35.5
35.5
27.5
28.3
30.9
32.4
Interfacia
Tension
dynes/cm
11.4
13.3
14.8
5.5
5.7
6.3
6.9
27
-------
APPENDIX C
RAW DATA
This section presents the complete set of data as the secv^ntiaj conduct of
individual tests. Excluding test 1, all Tests provided successful!}' :'_>r IOC9.- encounter
of test fluid by the device. Losses were observed fro;n the front of the d_-vice daring
Test 20 at 1.02 m/s and test 3 at .76 m/s with 6 mm slicks. This was observed as
a device performance situation and not the inefficiency of the guide booms. Also.
in certain harbor chop wave tests, vessel response caused additional reduced encounter.
TABLE C-l. TEST RESULTS - NON-REPLICATE
Date
7/31
7/31
7/31
7/31
7/31
8/1
8/1
8/1
8/1
8/1
8/1
8/1
8/1
8/1
8/2
8/2
8/2
8/2
8/2
8/2
8/2
8/2
8/3
8/3
8/3
8/3
Test
no.
1
1R
2
3
4
5
5R
6
7
8
3
10
11
12
13
1*
15
16
17
18
19
20
21
22
23
24
Tow
speed
(m/s)
0.76
0.76
1.02
1.52
0.76
1.02
1.02
1.52
1.02
1.52
0.76
1.02
1.52
0.76
1.52
0.76
1.02
1.52
0.76
1.02
0.76
1.02
1.52
0.76
1.02
1.52
Wave
cond.
cairn
calm
calm
calm
.3 m HC
Oil
type
light
light
light
light
light
Slick
thickness
(mm)
3.2
3.0
3.1
3.0
3.1
Total
oil .poll.
(m3)
.079
.076
.081
.060
.019
Throughput
efficiency
(%)
93.6
95.3
97.7
75.7
23.8
RUN ABORTED
.3 m HC
.3 m HC
calm
calm
calm
.3 m HC
.3 rn HC
.3 m HC
calm
calm
calm
.3 m HC
.3 m HC
.3 m HC
calm
calm
calm
.3 rn HC
.3 m HC
.3 rn HC
light
light
light
light
light
light
light
light
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
3.1
3.1
6.2
5.6
6.1
5.2
6.1
6.2
3.1
2.9
3.1
3.0
2.9
3.0
5.7
6.0
6.1
5.9
6.1
5.8
.036
.029
.149
.081
.146
.044
.059
.067
.062
.074
.083
.O'f7
.041
.036
.094
.143
.127
.046
.067
.073
44.2
35.5
90.4
54.9
90.1
37.4
36.8
41.1
76.7
96.2
102.6
61.7
53.5
45.7
61.7
88. 9
79.7
29.5
41.5
^6.8
28
-------
TABLE C-2. TEST RESULTS - REPLICATE
Daie
8/3
8/3
8/3
8/3
S/*
8/*
s/*
S/*
g/il
S/fc
S/*
S/*
8/*
8/1*
8/1*
8/1*
8/1*
8/1*
8/1*
8/1*
8/1*
8/1*
8/1*
8/15
8/16
8/16
8/16
8/16
8/16
8/16
8/16
8/16
8/16
8/16
8/16
Test
no.
1A
2A
IB
3A
*A
38
2B
1C
2C
*B
3C
*C
1CR
5A
6A
5B
7A
8A
7B
6B
5C
6C
8B
7C
7CR
8C
5D
6D
5E
7D
8D
7E
6E
5F
6F
Tow
speed
(m/s)
J .02
1.52
I .02
1.52
1.02
1.52
; .52
1.02
1.52
1.02
1.52
1.02
1.02
1.02
1.52
1.02
1.52
1.02
1.52
1.52
1.02
1.52
1.02
1.52
1.52
1.02
1.02
1.52
1.02
1.52
1.02
1.52
1.52
1.02
1.52
Wave
cond.
calm
calm
cairn
.3 rn HC
.3 rr. HC
.3 rn HC
cairn
calm
cairn
.3 m HC
.3 m HC
.3 rn HC
calm
calm
calm
calm
.3 m HC
.3 m HC
.3 m HC
calm
calm
calm
.3 m HC
.3 m HC
.3 rn HC
.3 m HC
calm
calm
calm
.3 rn HC
.3 m HC
.3 m HC
calm
calm
calm
Oil
type
heavy
h-avv
heavy
he aw
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
light
Slick
thickness
(mm)
3.1
3.6
3.2
2.9
3.2
3.0
2.9
3.0
3.0
3.0
3.1
3.0
3.1
3.0
3.0
3.2
3.1
3.1
3.1
3.1
3.0
3.0
2.9
3.3
3.2
3.1
3.0
3.1
2.9
3.0
3.2
3.1
2.9
3.1
3.0
Total
oil .coll.
(m3)
.086
.078
.083
.032
.037
.0*5
.075
N/A
.077
.030
.037
.0*5
.079
.086
.060
.077
.032
.030
.03*
.062
.069
.062
.03*
.031
.029
.028
.071
.055
.06*
.032
.021
.027
.052
.082
.060
Throughput
efficiency
(%)
10*. 7
95.*
97.5
*1.3
*3.6
56.8
5*7.1
97.3
38.*
**.S
56.6
95.6
110.1
75.8
91.1
39.9
37.0
*2.7
76.6
86.8
77.*
**.3
35.9
3*. 5
3*.l
90.8
68.3
82.*
*0.8
2*. 7
33.*
68.3
100.6
76.6
(Continued)
N/A --- not available
29
-------
TABLE C-2. CONT1NL-ED
Date
8/17
8/17
8/17
8/17
8/17
8/17
8/17
8/17
8/18
8/18
8/18
8/18
8/18
Test
no.
8E
7F
8F
ID
2D
IE
2E
2F
3D
*D
3E
*E
IF
Tow
speed
(m/s)
1.02
1.52
1.02
1.02
1.52
1.02
1.52
1.52
1.52
1.02
1.52
1.02
1.02
Wave
cond.
.3 m HC
.3 m HC
.3 rn HC
calm
calm
calm
calm
calm
.3 rn HC
.3 m HC
.3 m HC
.3 m HC
calm
Oil
type
light
light
light
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
heavy
Slick
thickness
(mm)
3.2
2.9
3.2
3.1
3.1
3.1
3.0
3.1
2.9
3.1
3.0
3.1
2.9
Total
oil.ro!!.
(m )
.022
.026
.03*
.069
.068
.083
.06*
.067
.037
.038
.030
.03*
.065
T:.r-.-eL,rut
e-;-?;.-'-"irv
1%)
26.9
33.8
*0.7
8*.0
8*. 3
103.1
79.7
83.2
*8.0
*7.1
39.0
*2.9
86.1
30
-------
TECHNICAL REPORT DATA
P.'<:ast' n?aJ ln±tsiictiis • >'< the reverse before completing!
-i f.J • J * r N k
~ i 7 L £ A N O
5 REPORT DATE
A Statistical Evaluation of OHMSETT Testing
6. PERFORMING ORGANIZATION CODE
I. RECIPIENT'S ACCESSIOI>NO
7 AUTHOR(S)
8. PERFORMING ORGANIZATION REPORT NO
)iane M. Foster
Sol H. Schwarz and Gary F. Smith
9 PERFORMING ORGANIZATION NAME AND ADDRESS
U.S. EPA
Idison, NJ
08837
Mason & Hanger-Silas Mason Co., Inc.
P.O. Box 117
Leonardo, NJ 07737
10. PROGRAM ELEMENT NO.
1NE623
ILCONfRACf/GRANtNO.
68-03-2642
12. SPONSORING AGENCY NAME AND ADDRESS
Municipal Environmental Research Laboratory
)ffice of Research and Development
U.S. Environmental Protection Agency
incinnati, OH 45268
13. TYPE OF REPORT AND PERIOD COVERED
Final
14. SPONSORING AGENCY CODE
EPA/600/14
15. SUPPLEMENTARY NOTES
John S. Farlow. Project Officer f201-3?1-fifi3n
16. ABSTRACT
This program was initiated to provide a statistical evaluation of performance
iata generated at the USEPA's Oil and Hazardous Materials Simulated Environmental Test
ank (OHMSETT). The objective was to investigate the value of replicate testing in
teveloping efficient test programs giving the maximum reliable information from the
ninimum number of tests.
This study was set up in two separate programs, each consisting of 24 tests: one
)rogram where 24 different conditions were tested and a second program where three
replicates each of eight different conditions were tested. A comparison was then pos
n'ble between the two types of programs. The 3-replicate test matrix was duplicated to
jroduce six replicates and the validity of the non-replicate and 3-replicate programs
vas evaluated with respect to the 6-replicate data.
Parameters affecting device performance studied in this program were tow speed,
vave condition, oil type, and oil slick thickness. These parameters were tested
it various levels, with device performance evaluated in terms of throughput efficiency
the ratio of oil collected to oil encountered).
Comparisons between point estimates and confidence intervals, graphic trend
analysis, and analysis of variance were all examined. The results of this program
indicate a need for replicate testing to provide accurate estimates of performance
)arameters, significant effects and performance trpnds.
17.
KEY WORDS AND DOCUMENT ANALYSIS
DESCRIPTORS
b.IDENTIFIERS/OPEN ENDED TERMS c. COSATI Field/Group
Experimental Design
jtatistical Analysis
erformance Tests
i/ater Pollution
Oil Spill Cleanup
Oil Skimmer Evaluation
Maximizing Test Effec-
tiveness
Repetitive Testing
13. ^DISTRIBUTION STATEMENT
Release to Public
19. SECURITY CLASS (This Repon)
UNCLASSIFIED
21. NO OF PAGES
39
20. SECURITY CLASS /This page!
UNCLASSIFIED
22 PRICE
EPA Form 2220-1 (9-73)
31
------- |