EPA-650/4-75-009
EVALUATION AND COLLABORATIVE
STUDY OF METHOD
FOR VISUAL DETERMINATION
OF OPACITY OF EMISSIONS
FROM STATIONARY SOURCES
by
Henry F. Hamil, Richard E. Thomas,
arid Nollie F. Swynnerton
Southwest Research Institute,
8500 Culebra Road
San Antonio, Texas 78284
Contract No. 68-02-0626
ROAP No. 26AAG
Program Element No. 1HA327
EPA Project Officer: M. Rodney Midgett
Quality Assurance and Environmental Monitoring Laboratory
National Environmental Research Center
Research Triangle Park, North Carolina 27711
Prepared for
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF RESEARCH AND DEVELOPMENT
WASHINGTON, D.C. 20460
January 1975
-------
EPA REVIEW NOTICE
This report has been reviewed by the National Environmental Research
Center - Research Triangle Park, Office of Research and Development,
EPA, and approved for publication. Approval does not signify that the
contents necessarily reflect the views and policies of the Environmental
Protection Agency, nor does mention of trade names or commercial
products constitute endorsement or recommendation for use.
RESEARCH REPORTING SERIES
Research reports of the Office of Research and Development, U.S. Environ-
mental Protection Agency, have been grouped into series. These broad
categories were established*to facilitate further development and applica-
tion of environmental technology. Elimination of traditional grouping was
consciously planned to foster technology transfer and maximum interface
in related fields. These series are:
1. ENVIRONMENTAL HEALTH EFFECTS RESEARCH
2. ENVIRONMENTAL PROTECTION TECHNOLOGY
3. ECOLOGICAL RESEARCH
4. ENVIRONMENTAL MONITORING
5. SOCIOECONOMIC ENVIRONMENTAL STUDIES
6. SCIENTIFIC AND TECHNICAL ASSESSMENT REPORTS
9. MISCELLANEOUS
This report has been assigned to the ENVIRONMENTAL MONITORING
series. This series describes research conducted to develop new or
improved methods and instrumentation for the identification and quanti-
fication of environmental pollutants at the lowest conceivably significant
concentrations. It also includes studies to determine the ambient con-
centrations of pollutants in the environment and/or the variance of
pollutants as a function of time or meteorological factors.
This document is available to the public for sale through the National
Technical Information Service, Springfield, Virginia 22161.
11
-------
SUMMARY AND CONCLUSIONS
This report presents the results of statistical analyses on data obtained in the collaborative testing of
EPA Method c> (Visual Determination of Opacity). There were 5 separate tests of Method 9 at three dif-
ferent sites: a training smoke generator, a sulfuric acid plant and a fossil fuel-fired steam generator.
For these tests, the opacity measure used is the average opacity, defined as the average of 25 read-
ings made at 15-second intervals. These determinations are analyzed for their accuracy and their precision.
For this report, accuracy is measured by the deviation of the observer's determination from the true opacity
as measured by the in-stack transmissometer. The precision of the method is measured in terms of within-
observer, observer bias and between-observer terms.
• Training Generator-Thirty-six runs were made over a two-day period using two training smoke
generators with 9 qualified observers participating. There were 20 runs using white smoke and 16 using
black smoke with plume opacity ranging from just above zero to 40 percent. The two cases are treated sep-
arately.
• White Smoke—There were a total of 170 determinations made using a white plume.
These values showed a low bias with respect to the transmissometer, and the deviation from the meter is
estimated by
deviation = 3.46 - 0.33 (meter average).
The within-observer standard deviation is estimated as 2.38 percent opacity with 133 degrees of freedom.
The observer bias standard deviation is 0.95 percent opacity with 7 df. This gives a between-observer
standard deviation of 2.56 percent opacity,
• Black Swofce- There were 133 determinations of the average opacity of black smoke. The
results from these values were similar to those of the white smoke. The predicted deviation from the trans-
missometer for this case is
deviation = 3.74 — 0.34 (meter average).
The within-observer standard deviation is 1.84 percent opacity with 105 degrees of freedom. The observer
bias standard deviation is 1.00 percent opacity with 7 df. This gives a between-observer standard deviation
of 2.09 percent opacity.
• Sulfuric Acid Plant-Thirty observation runs were made over two days at a sulfuric acid plant.
There were 11 observers on the first day for 14 runs and 9 on the second day for the remaining 16 runs. The
plume opacity was varied around the compliance limit for sulfuric acid plants, ranging from 2 to 15 percent
average opacity. The observers were chosen from both a local enforcement agency and from private con-
cerns to allow a comparison of the two groups.
The predicted deviation for these data is -2.0 percent opacity for the entire range studied. The
within-observer standard deviation is 2.12 percent opacity with 232 degrees of freedom. The observer bias
standard deviation is 0.96 percent opacity with 8 degrees of freedom. This gives a between-observer stan-
dard deviation of 2.33 percent opacity. It is shown for this test that the enforcement observers read a
higher, and more accurate, average opacity than did the private sector observers.
• Steam Station-Three separate tests were conducted at a power plant with plume opacity rang-
ing up to 35 percent average opacity. On the first two days of testing, the weather conditions prevented
the accurate determination of plume opacity. In the third test period, viewing conditions were ideal and
the accuracy of the method is evaluated.
Hi
-------
• Test 1 -There were 10 runs using 10 observers at the first test. The determinations were
well below the concurrent meter averages due to the sky and wind conditions. The deviation from the meter
average is represented by
deviation = —3.01 — 0.40 (meter average).
A total of 60 determinations were used to obtain the precision estimates. The within-observer standard
deviation is estimated as 1.82 percent opacity with 45 df. The observer bias standard deviation is estimated
as 1.39 percent opacity with 8 df, which gives a between-observer standard deviation estimate of 2.29 per-
cent opacity.
• Test 2—Eighteen runs were made at the second test using 10 observers. The determinations
made were more accurate than the previous test, but still were well below the meter average. The deviation
from the true opacity is represented by
deviation = —1.02 — 0.19 (meter average).
A total of 118 determinations are used to obtain the precision estimates. The within-observer standard
deviation is 1.84 percent opacity with 88 degrees of freedom. The observer bias standard deviaton is esti-
mated as 1.50 percent opacity with 8 degrees of freedom. The estimated between-observer standard devia-
tion, then, is 2.37 percent opacity.
• Test 3—Twenty-four runs were made using eight observers over a two-day period. The
resulting 192 determinations have a negative bias with respect to the metered opacity that can be repre-
sented by
deviation = 2.27 - 0.24 (meter average).
The within-observer standard deviation is estimated as 1.89 percent opacity with 90 degrees of freedom.
The observer bias standard deviation is 1.77 percent opacity with 6 degrees of freedom, which gives a
between-observer standard deviation of 2.59 percent opacity.
Also included for this test are evaluations of two variations to the method: reading to the
nearest percent, and averaging two observers' results.
* Composite Estimation^Using the results from the three test sites, it is possible to make com-
posite estimates of the accuracy and precision that can be expected with field use of Method 9. The ex-
pected deviation from the average metered opacity is given by
deviation = 3.13 - 0.31 (meter average)
for the range from 5 to 35 percent average opacity. This equation is obtained from the training generator
and steam station test data.
The composite precision estimates are obtained using the estimates from all tests. The estimated
within-observer standard deviation is 2.05 percent opacity with 693 df. The estimated observer bias standard
deviation is 1.29 percent opacity with 44 df. This gives a composite between-observer standard deviation
estimate of 2.42 percent opacity.
For each case, estimates are made of the range of determinations that would be expected from a single
observer, and of the maximum difference that would be expected between two observers when determining
the average opacity of a plume.
-------
TABLE OF CONTENTS
Page
LIST OF TABLES vii
I. INTRODUCTION J
II. COLLABORATIVE TESTING 2
A. Purpose 2
B. Smoke Generator Test 2
C. Stauffer Chemical Company Test 2
D. Riverbend Steam Station • 3
E. Collaborators and Test Personnel 4
III. DEFINITIONS AND STATISTICAL TERMINOLOGY 7
A. Definitions 7
B. Statistical Terminology 7
IV. SMOKE GENERATOR TEST
9
A. White Smoke 9
B. Black Smoke 12
V. STAUFFER CHEMICAL COMPANY TEST 16
A. Accuracy 1°
B. Precision 16
C. Predicted Results 18
D. Comparision of Enforcement and Private Observers 19
VI. RIVERBEND STEAM STATION TEST 20
A. Test 1 20
B. Test 2 23
C. TestS 26
VII. COMPOSITE ESTIMATION 34
A. Accuracy 34
B. Precision 34
C. Predicted Results 35
APPENDIX A-EPA METHOD 9 FOR VISUAL DETERMINATION OF OPACITY 37
A.1 Federal Register, Vol 36, No. 247-Thursday, December 23, 1971 39
A.2 Federal Register, Vol 39, No. 177-Wednesday, September 11, 1974 40
APPENDIX B-STATISTICAL METHODS 45
B.1 Linear Regression and Test for Significance 46
B.2 Precision Estimation for Training Generator Test 47
B.3 Precision Estimation for Sulfuric Acid Plant and Tests of Hypotheses 49
B.4 Analysis of Variance and Tests of Hypotheses from Riverbend Steam Station Test 1 . 52
-------
TABLE OF CONTENTS (Cont'd)
Page
B.5 Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 2 . 54
B.6 Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 3 . 56
B.7 Precision Estimation for Riverbend Steam Station Test 3, 1-Percent Increment
Runs 59
B.8 Significance of Reduction of Frequency of Large Deviations 61
B.9 Multiple Comparison Tests and Composite Estimation 62
LIST OF REFERENCES 63
VI
-------
LIST OF TABLES
Table
1 Certification Data on Collaborators, Riverbend Steam Station Tests 4
2 Test Log, Riverbend Steam Station, Test 1 4
3 Test Log, Riverbend Steam Station, Test 2 4
4 Test Log, Riverbend Steam Station, Test 3 5
5 Determinations of Average Opacity of Training Generator, White Smoke 10
6 Deviation From Training Generator, White Smoke 10
7 Predicted Deviations, Training Generator, White Smoke 11
8 Predicted Determinations, Training Generator-White Smoke 12
9 Determinations of Average Opacity of Training Generator, Black Smoke 13
10 Deviation From Training Generator, Black Smoke 13
II Predicted Deviation From Metered Opacity, Training Generator-Black 14
12 Predicted Determinations of Average Opacity, Training Generator-Black 15
13 Determinations of Average Opacity, Stauffer Chemical 17
14 Deviations From Meter Average, Stauffer Chemical 18
15 Predicted Determinations, Sulfuric Acid Plant 19
16 Observer Means, Sulfuric Acid Plant 19
17 Average Opacity Determinations, Steam Station Test 1 20
18 Deviations From Metered Opacity, Steam Station Test 1 21
19 Predicted Deviation From Metered Opacity, Steam Station Test 1 21
20 Predicted Determinations, Steam Station Test 1 23
21 Observer Means For Steam Station Test 1 23
22 Average Opacity Determinations, Steam Station Test 2 24
23 Deviations From Metered Opacity, Steam Station Test 2 25
24 Predicted Deviation From Metered Opacity, Steam Station Test 2 25
25 Predicted Determinations, Steam Station Test 2 26
26 Observer Means For Steam Station Test 2 26
vii
-------
LIST OF TABLES (Cont'd)
Table Page
27 Average Opacity Determinations, Steam Station Test 3 27
28 Deviations From Metered Opacity, Steam Station Test 3 27
29 Predicted Deviation From Metered Opacity, Steam Station Test 3 28
30 Predicted Determinations, Steam Station Test 3 29
31 Observer Means For Steam Station Test 3 30
32 Predicted Deviations, 1-Percent Increment Data 30
33 1-Percent Data Summary 31
34 Predicted Determinations, 1-Percent Data 32
35 Opacity Determinations Using Paired Observers 32
36 Deviations From Opacity, Paired Observers 33
37 Predicted Deviation From Metered Opacity, Composite Estimate 34
38 Predicted Determinations of Average Opacity, Composite Estimate 35
Bl Analysis of Variance From Training Generator Test, White Smoke 47
B2 Analysis of Variance From Training Generator Test, Black Smoke 48
B3 Analysis of Variance From Sulfuric Acid Plant Test 50
B4 Analysis of Variance From Steam Station Test 1 53
B5 Analysis of Variance From Steam Station Test 2 55
B6 Analysis of Variance From Steam Station Test 3 57
B7 Analysis of Variance From Steam Station Test 3, 1% Variation 60
B8 Analysis of Variance From Steam Station Test 3, 1% Runs 61
Vlll
-------
I. INTRODUCTION
This report describes the work performed and results obtained on Southwest Research Institute Project
01-3462-006, EPA Contract No. 68-02-0626, which includes collaborative testing of Method 9 for visual determi-
nation of the opacity of emissions from stationary sources as given in "Standards of Performance for New Stationary
Sources"! 1)* and "Stationary Sources, Proposed Emission Monitoring and Performance Testing Requirements."(2)
This report describes the statistical analysis of the data from collaborative tests conducted on two EPA-type
training smoke generators, and at a sulfuric acid plant and a coal-fired power plant.
The results of the data analyses and the conclusions based on these analyses are given in this report.
'Superscript numbers in parentheses refer to the List of References at the end of this report.
-------
II. COLLABORATIVE TESTING
A. Purpose
/ i -y\
Collaboralive testing of the method for visual determination of opacity of emissions from stationary sources' • '
was conducted in order to obtain data using certified observers which would allow statistical evaluation of the method.
Results from these tests are presented in this report.
Three collaborative test sites were used: a training smoke generator, a sulfuric acid plant, and a fossil fuel-
fired steam generatoi. Tests at the first two sites mentioned above were designed to evaluate the determination of
average opacity of emissions^), while the test at the third site was designed to allow evaluation of both the deter-
mination of minutes of noncompliance with a standard and determination of average opacityX-^) The results in
this report are all based upon average opacity determinations.
The initial test on the training smoke generator was conducted to provide background information on the use
of the method. The tests at the sulfuric acid plant and the fossil fuel-fired steam generator were conducted to obtain
information on the use of the method on applicable sources under field conditions. At no time during any
of the test were warm-up or practice runs allowed prior to the test itself.
B. Smoke Generator Test
The smoke generator test was conducted on 19-20 November 1973 at the National Environmental Research
Center, Research Triangle Park, North Carolina. Two training smoke generators were used, one belonging to the EPA
and one belonging to the State of North Carolina. A total of nine collaborators from enforcement agencies in the
State of North Carolina were available for the tests. All nine collaborators participated in the first fifteen runs—only
eight participated in the remaining twenty-one runs, since one collaborator operated the State of North Carolina's
smoke generator. Time since last certification for the collaborators varied from one to thirteen weeks.
During the two-day test, a total of thirty-six runs were made, with a run consisting of twenty-five readings
taken at the fifteen-second intervals by each collaborator. The collaborators read the plume on a time signal from
the test supervisor in order to assure that the readings were as nearly simultaneous as possible. The opacity of the
plume as indicated by the in-slack transmissometer of the smoke generator was recorded at the time signal to pro-
vide "true" opacity data. Twenty runs were made with white plumes, and sixteen runs were made with black plumes.
The range of opacity studied was from just above 0 to 40 percent. During some of the runs, plume opacity was held
essentially constant while on other runs the opacity of the plume was varied randomly to determine how
well the observers could follow changing opacity during the course of a set of observations.
Due to the relatively short stack height on the smoke generators, in some runs the observers had objects in
the background (buildings, trees, power lines, etc.), while on other runs only a sky background was available.
Since use of background objects in evaluating plume opacity is permitted in most training courses, no differentiation
was made in this test between runs with or without background objects. During both days of the test, viewing con-
ditions were ideal, with blue sky and bright sunshine. On all runs, the observers were allowed to select the opti-
mum viewing position with respect to sun and wind angles.
C. Stauffer Chemical Company Test
The sulfuric acid plant test was conducted on 29-30 January 1974 at Stauffer Chemical Company's facilities
in Houston. Texas. The unit was equipped with a 300-foot-high stack. In-stack monitoring of opacity was accom-
plished by use of a Lear Siegler Model 610A transmissometer*.
A total of eleven collaborators were available for the first day of the test. Nine collaborators were available for
the second day of the test. Four of the collaborators were from the City of Houston Department of Public Health,
•Mention o! truck names or speiitic products doos not constitute endorsement by the Lnvironmculul Protection Agency
-------
six were from area industries, and one was from a local health association. All collaborators' most recent certifica-
tion was on 24 August 1973 at the State of Texas Training Session conducted at Houston, Texas.
During the two-day test, a total of thirty runs were made, fourteen rum on the first day and sixteen runs on
the second day. Each run consisted of twenty-five readings taken at 15-second time intervals by each observer.
The collaborators began each run on a time signal from the test supervisor. The average opacity as indicated by
the in-stack transmissometer was obtained from the instrument strip chart recorder for the time period correspond-
ing to each run.
Arrangements were made with plant personnel to change unit operating conditions during the test to provide
plumes of varying opacity. The range of opacity studied was from slightly above 0 to 15 percent. Opacity of the
plume was essentially constant during any single run. During both days of the test, good viewing conditions pre-
vailed, with blue sky and a slight haze. Wind was light and variable throughout the test. Color of the plume was
white.
The observers were allowed to select the optimum viewing position with respect to wind direction and sun
angle.
D. River bend Steam Station
Three tests were conducted at the Duke Power Company's Riverbend Steam Station near Charlotte. N.C.
Tests were conducted on 27 August, 5 September, and 30 September-1 October, 1974. The tests involved use of
two units at the power plant, number 9 and number 10 boilers. Each boiler was equipped with two stacks (A&B).
Readings for the first two tests and the first day of the third test were made on Stack B of boiler number 9. On
1 October 1974, the number 9 boiler was shut down, and the third test was completed using Stack A of number 10
boiler. Both stacks were fitted with Lear Siegler Model RM4 transmissometers*.
The plan for the tests at the Riverbend Steam Station varied from the plans for the previously described tests
This set of tests was designed to provide information on some variations to the method as written.(' >-' The col-
laborators were divided into two equal groups with regard to size and experience. One group was used as a control
and made their readings of opacity within the constraints of the published method, with the sun in the quadrant
to their rear, assigning opacity to the plumes in 5-percent increments. The other was used as the experimental
group. One variation under study by this group was reading at 45- and 90-deg angles with respect to the sun.
Opacity was assigned in 5-percent increments during this portion of the experiment. The other variation was read-
ing the plume in 1-percent increments from the same position as the control group
From lhis test, the effect on a method test result induced by reading outside some of
the constraints of the method may be estimated. By altering the angle of view with respect
to the sun, it may be determined whether this is an important consideration for the observer.
By requiring that the plume be read in I-percent increments, it may be determined if this
variation is more accurate or more precise. In addition to these, the relative performance of
an inexperienced to an experienced observer may be assessed, and the possible improvement
of the method by averaging two observers" results may be evaluated.
In the first two tests, a run consisted of forty observations by each observer taken at 15-second intervals. This
was done for the purpose of evaluating the method when used to determine minutes of noncornpliance with a
standard. The average opacity presented in the data tables represents the average of the first twenty-five readings
by an observer in each run. In the third test, a run consisted of twenty-five observations by each observer taken at
15-second intervals. In all three tests, the observers read the plume on a time signal from test personnel. Transmis-
someter readings were recorded to the nearest percent on the time signal to provide "true" opacity readings.
'Trade rumc.
-------
I ABM' 1. CERTIFICATION DATA ON COLLABORATORS,
RIVKRBENDSTEAM STATION TESTS
Collaborator
Code Number
,
1
~! *
4
5
6
7
8
9t
II)
1 1
12
1 j
Duleof
Most Recent
Certification
7-16-74
7-16-74
4-5-74
4-5-74
4-5-74
4-5-74
4-5-74
4-5-74
3-19-74
8-27-74
9-11-74
9-11-74
9-11-74
Average Deviation
from True Opacity
During Certification, %
4.7
5.7
4.6
5.4
5.9
6.2
5.9
4.4
4.3
3.8
3.1
2.9
3.9
*Not qualified. One reading during qualification deviat-
ed from the opacity by more than 15 percent. Not
determined until after test due to clerical error.
•^Certification expired 9-19, recertified 10-22.
TABLE 2. TEST LOG, RIVERBEND STEAM STATION, TEST 1
27 August 1974
Run
No.
1
•>
3
4
5
6
7
S
9
10
Time Run
Began
1015
1055
1130
1330
1 350
1410
1430
1450
1510
1530
Position
Angle, deg
45
90
90
0
0
0
45
45
90
90
Sky Conditions
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, tow haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Wind
NE, 0-5 mph
Light, variable
Light, variable
SW, 5 -10 mph
SW, 5- 10 mph
Light, variable
Light, variable
Light, variable
W, 3-5 mph
W, 3-5 mph
TABLE 3. TEST LOG, RIVERBEND STEAM STATION, TEST 2
5 September 1974
Run
No.
1
-)
3
4
5
6
7
«
9
10
H
i?.
1 3
14
15
16
1 •
1 S
Time Run
Began
0910
0930
0950
1010
11130
1050
1 130
1 150
121(1
1 230
1 ."50
1410
I4.ui
1450
1530
155' )
1 r, i n
I Mi:
Position
Angle, deg
0
0
45
45
90
90
45
45
90
90
0
0
9o
90
11
ft
45
45
Sky
Conditions
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcas
Ovcrca>
Overea.x
Ovcrcas
Overeat
Ovcrcas
Wind
NE, 5-10 mph
NE, 5-10 mph
NE, 5-10 mph
NE, 5-10 mph
NE, 5 -10 mph
NE. 5-10 mph
NE, 5-10 mph
NE, 5- 10 mph
NE.5 -10 mph
NE, 5 -10 mph
NE, 5 lOniph
NE.5 10 mph
NE. 510 mph
NE. 5 10 mph
NE.5 lOtnph
NE.5 lOmpli
M . 5 10 mph
NE.5 10 mph
The range of opacity studied in all
three tests was from just above 0 to 40 per-
cent. Opacity of the plume was varied by
cutting off one or more stages of the elec-
trostatic precipitator prior to the run.
With the exception of occasional short-
duration transient high values, plume opac-
ity was essentially constant during a given
run for the low opacities. Variation in the
"true" opacity within a run increased as
the average plume opacity increased.
The collaborators for each test came
from State of North Carolina enforcement
agencies, EPA offices, and private contrac-
tors. The first two tests utilized four state,
four EPA, and two contractor personnel.
The third test utilized four state, two EPA,
and two contractor personnel. The date of
the most recent certification of the collab-
orators is given in Table 1. Also included
for each of the collaborators is the average
deviation from the true plume opacity for
the twenty-five white and twenty-five black
plumes read during certification.
The three tests consisted of ten,
eighteen, and twenty-four runs, respectively.
Viewing conditions during the first test were
poor, mostly cloudy and low haze. There
was also interference from another plume
during portions of the test. Conditions
for the second test were marginal, with a
solid overcast, while for the third test con-
ditions were ideal, with a cloudless sky and
bright sunshine. The plume color was white.
Test logs for the three tests are pre-
sented in Tables 2,3, and 4. In Tables 2
and 3, the position of the experimental
group is given relative to the control group.
In Table 4, the positions of both the con-
trol and experimental groups are given
relative to the sun, with the angles calcu-
lated as shown in Figure 1.
E. Collaborators and Test Personnel
The collaborators for the smoke gener-
ator test were Mr. Sammy L. Amerson.
Mr. George Lawson, Mr. Dewey D. Johnson,
Mr. Roy T. Gorman, Mr. J.R. Kirk, Mr. D.Y.
Daniel, Jr., Mr. B.J. Foust and Mr. Bill
Proctor from the State o1 N'orth Carolina
-------
TABLE 4. TEST LOG, RIVERBEND STEAM STATION, TEST 3
30 September - 1 October I 974
Run
No.
I
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Time Run
Began
1315
1330
1345
1400
1415
1430
Viewing Angle, 8 (Fig. 1)
Control
Group.
deg
0
0
10
10
15
15
1445 45
1500 45
1515 0
1530 0
0930
0945
1000
1015
1130
1145
1200
1215
1230
1245
1 300
1315
1330
1345
30
30
25
25
20
20
10
10
0
0
0
0
0
0
Experimental
Group,
deg
0
0
45
45
60
60
45
45
0
0
30
30
75
75
90
90
45
45
0
0
45
45
90
90
Sky
Conditions
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Gear
Gear
Gear
Clear
Clear
Clear
Clear
Wind
Light, variable
Light, variable
Light, turiubic
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
N. 8-12 mph
N. 8 - ' 2 mph
N, 5 10 mph
N, 5 -10 mph
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
Ligh'. variable
Ligh'. variable
Light variable
Light . variable
Light, variable
\ /
\
6 - Angle formed by lin* between sun and
plume being observed and line between
observing group and plume.
CONTROL GROUP
* • EXPERIMENTAL
GROUP
\ '
OO O'O OO OO
POWER PLANT
HGURL 1. SCHEMATIC OF OBSERVER POSITIONS.
RIVTRBEND STEAM STATION. TEST 3
-------
Depai imunt of Natural and Economic Resources; and Mr. William M. Edsel from the County of Forsyth Air Quality
Control Division, Forsyth County, North Carolina.
The collaborators for the sulfuric acid plant test, all from the Houston, Texas, area, were Mr. Charles L. Owen
fiom Annco Steel; Mr. James S. Corbin, Mr. Robert J. Stahl, Mr. Ronald F. Stockunas and Mr. Wendell La Foe
trum the City of Houston Department of Public Health; Mr. B. L. Bolton from Rollins Environmental Services;
Mr.T. M. Walker and Mr. Klaus R. Gerlach of Stauffer Chemical Co.; Mr. Mel Remley from Jefferson Chemical Co.;
Mr. William Mooie from the San Jacinto Lung Association; and Mr. Larry P. Stoltz from GAP Corporation.
The collaborators for the power plant tests were Mr. Dewey D. Johnson, Mr. Ron Jernigan, Mr. John Leather-
man and Mr. Don Shepherd from the Sfate of North Carolina Department of Natural and Economic Resources;
Mr. William M. Hdsel from the County of Forsyth Air Quality Control Division; Mr. David Da Crema and
Mr. John Yates. from Engineering Science Inc.; Mr. Alan Luther and Mr. Peter F. Burnette from Environ-
mental Science and Engineering, Inc.; and Mr. Alfred Vervaert, Mr. J.W. Brown. Mr. Joseph Peoples, Mr. James
E. Casey, Mr. Jack Siegel and Mr. John Hund from the Environmental Protection Agency.*
The tests were conducted under the general supervision of Mr. Nollie F. Swynnerton. The collaborators from
the Stale ot North Carolina were provided through the assistance of Mr. James A. McColman, Chief, Air Quality
Division, State of North Carolina Department of Natural and Economic Resources. The collaborators for the sulfuric
acid plant test were selected by Mr. Swynnerton from a list of people who had certified at the most recent State of
Texas certification school in the Houston area. The list was provided by Mr. Thomas Jay McMickle, Environmental
Health Specialist. Texas State Department of Health. The collaborators from EPA, Engineering Science, and
Environmental Science and Engineering were provided by Mr. Roy Neulicht, Environmental Engineer.
Emissions Standards and Engineering Division, Office of Air Quality Planning and Standards, EPA. Mr. Dennis
Holzschuh, Pysical Science Technician. Engineering and Enforcement Section. Air Pollution Training Institute,
Control Programs Development Division, Office of Air Quality Standards and Planning. EPA, assisted in the
conduct of the smoke generator test at the National Environmental Research Center. Mr. William D. Conner,
Research Physicist. Stationary Source Measurements Research Section. Emission Measurements Research
Branch. Chemistry and Physics Laboratory, National Environmental Research Center. EPA. made the neces-
sary arrangements with Duke Power Company personnel for the Riverbend Steam Station Tests.
t llv Kiiijmdcr ot thi\ n.-pm I. I!K- . ilixerv
:o i!u- .ilmvi- ordered IMIMS> ol ntiM.':ver-.
ode nunilvrs
d dn not
-------
III. DEFINITIONS AND STATISTICAL TERMINOLOGY
To facilitate the understanding of this report and the utilization of its findings, this section explains the
statistical terms used in this report and defines certain terms used in presenting the data and results.
A. Definitions
1. Run -A 6-minute period during which the observers determined the opacity of the plume.
2. Determination-The arithmetic mean of 25 consecutive opacity readings taken at 15-second
intervals during a run.
3. Meter Average-The average opacity of the plume as determined by the in-stack transmissometer.
4. Deviation-The difference between a determination and the corresponding meter average; deviation =
determination — meter average.
B. Statistical Terminology
1. x = — ^fXi -the sample mean. This statistic estimates the mean, or center, of a given population.
"•-1
1 "
2. s2 = - y^ (Xf - ;T)a —the sample variance. This statistic estimates the dispersion in a dis-
» - 1 ,' = j
tribution around the mean value.
3. x = v^7 -the sample standard deviation. The standard deviation is an alternative estimate of
dispersion.
4. Analysis of Variance (ANOVA)— A statistical technique for testing whether different groups or
factors perform in an equivalent manner, and for obtaining estimates of different variance terms.
5. SS— sums of squares. The sum of squared deviations from the mean for a component. The SS are
used to estimate variability.
6. df— degrees of freedom. The df are an indication of the amount of confidence in the estimate. A
larger number of df implies a greater degree of confidence.
7. MS-the mean square. The MS is the basis for hypothesis testing in the ANOVA and estimating a
variance component. MS = SS/df
8. EMS-the expected mean square. The EMS determines which mean squares are used to test which
hypotheses, and to estimate the particular variance components.
9. F-Ratio— the ratio of two mean squares. The F-ratio is the test statistic for determining the relative
performance of two groups.
-------
All precision components in this report are estimated using an ANOVA procedure. The statistical
model is given separately for each data set, but, in general, consists of an observer term, a run term and an
error term. The MS and EMS from these ANOVA tables are used to determine the significance of the vari-
ous factors, and to estimate the following precision components.
* Within-observer. The within-observer, or error, component measures the dispersion that would be
expected in replicate determinations by the same observer at the same true opacity. The within-
observer variance is denoted by a2.
• Observer bias. The observer bias component measures the dispersion that would be expected in
replicate determinations due to the use of the method by independent observers. These dif-
ferences can result from such factors as the training generator on which the observers qualified
and the relative frequency with which they use the method. The observer bias variance is denoted
•i-
by a?
Between-observer. The between-observer component is estimated from the within-observer and
observer bias terms. The between-observer term measures the variability associated with the dif-
ference between results by independent observers. The between-observer term is composed of
within-observer and observer bias terms, and the between-observer variance, a| is defined as
-------
IV. SMOKE GENERATOR TEST
The initial test of Method 9 was conducted using two EPA-approved smoke generators to produce plumes,
both white and black, covering the range of opacities that could be expected by observers seeking to determine
compliance with the new source performance standards. These included plumes from just above zero up to
40 percent. On each run, the collaborators recorded their observations to the nearest 5 percent for 25 readings
taken at 15-second intervals. The observers read on a signal given by test personnel, so that all determinations
were comparable. The 25 readings were averaged by SwRI personnel after the test to insure that the deter-
mination was correctly computed.
To determine to what extent the observers were able to follow changes in the plume opacity, the generated
opacity was subject to change during a run. During the runs, the opacity was either held constant, increased or
decreased according to a test plan. In addition, the number of times the plume opacity was changed, the point
at which it was changed and the amount it changed all were varied from run to run so that the observers would
not be able to anticipate these changes.
The observers were allowed to choose their position for observing the generated plume, consistent with
the constraints of the method. On some runs, they had a background (e.g., trees, buildings) to read against,
while on others, there was only blue sky. No record of the background was kept, so any differences in this
regard cannot be determined.
There were a total of 36 sampling runs, 20 with white smoke and 16 with black, made during the two
day period. One observer participated in only the first 15 runs then operated the second training generator for
the remainder of the test. This resulted in a total of 303 determinations, 170 with white smoke and 133 with
black. The analyses of the data will concern itself with two aspects: the accuracy, as measured by the deviation
from the metered opacity, and the precision, expressed in terms of standard deviations. The results are pre-
sented below separately for the white and black plumes.
A. White Smoke
The 170 determinations are shown in Table 5, along with the concurrent meter averages. To evaluate the
performance of the observers, the determinations will be looked at in two ways. First, they will be compared
to the meter averages to determine how accurately the observers were able to read the opacity; and second, they
will be compared to each other to determine how much variation can be expected both from a single observer,
and between two independent observers when reading the same opacity plume.
1. Accuracy
The deviations from the plume opacity are shown in Table 6. As can be seen from the data, there
is a tendency for the deviation to be greater and negative as the plume opacity increases. Because of this, a
linear regression is used, with opacity as the independent variable and deviation as the dependent variable.
The technique is presented in Appendix Bl.
For these deviations, the slope of the least-squares fit to these points is significantly different
from zero. This indicates that there is a linear relationship between the two variables. The equation of the
line derived is
Yj = 3.46 - 0.33*,
where
Yj is the predicted deviation
Xj is percent opacity
-------
TABLE 5. DETERMINATIONS OF AVERAGE OPACITY OF TRAINING
GENERATOR, WHITE SMOKE
(Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
27
28
29
30
31
32
33
34
35
36
Observer
1
12.8
10.6
8.8
27.0
11.2
5.6
8.8
31.2
22.6
9.4
17.4
8.6
10.0
13.0
18.2
32.2
19.0
9.0
14.8
12.4
2
12.0
9.0
5.8
28.2
9.6
4.6
8.4
22.6
15.4
7.6
17.4
6.4
8.8
14:8
20.8
29.4
18.8
8.8
17.0
10.2
3
10.8
11.2
8.6
23.8
12.0
5.0
7.0
24.6
13.6
8.4
19.2
8.6
12.4
15.8
17.4
30.8
20.8
13.6
20.4
9.8
4
13.2
11.0
9.6
22.6
12.0
6.6
7.2
24 .4
16.6
7.8
19.6
7.4
10.6
19.4
18.4
28.6
19.6
9.6
16.4
10.0
5
10.6
8.2
6.0
13.6
11.4
5.8
7.6
20.2
19.6
10.0
18.8
8.0
11.4
15.4
19.6
27.6
16.6
7.2
17.0
9.4
6
17.4
10.6
8.6
22.2
12.8
5.8
8.2
21.0
17.4
7.8
17.4
6.2
11.4
12.0
12.8
18.2
15.0
7.2
13.0
7.4
7
11.0
8.4
6.6
14.6
11.8
6.0
7.2
21.4
20.6
9.0
16.4
5.2
8.7
13.6
13.8
23.6
18.8
7.8
15.8
9.2
8
10.6
12.8
8.6
25.0
11.4
.5.0
7.6
27.4
17.0
7.2
16.4
5.2
8.2
12.6
10.8
23.2
13.0
7.0
11.0
8.0
9
13.0
14.8
10.0
24.0
11.2
6.6
7.4
27.4
16.0
8.6
Meter
10.0
13.6
9.2
23.8
12.3
4.7
7.6
34.0
25.4
12.4
13.1
8.6
9.0
11.2
14.3
31.3
13.2
16.8
18.7
14.6
TABLE 6. DEVIATION FROM TRAINING GENERATOR,
WHITE SMOKE
(Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
27
28
29
30
31
32
33
34
35
36
Observer
1
2.8
-3.0
-0.4
3.2
1.1
0.9
1.2
-2.8
-2.8
-3.0
4.3
0.0
1.0
1.8
3.9
0.9
5.8
-7.8
-3.9
-2.2
2
2.0
-4.6
-3.4
4.4
-2.7
-0.1
0.8
-11.4
-10.0
-4.8
4.3
-2.2
-0.2
3.6
6.5
-1.9
5.6
-8.0
-1.7
-4.4
3
0.8
-2.4
-0.6
0.0
-0.3
0.3
-0.6
-9.4
-11.8
-4.0
6.1
0.0
3.4
4.6
3.1
-0.5
7.6
-3.2
1.7
-4.8
4
3.2
-2.6
-0.4
-1.2
-0.3
1.9
-0.4
-9.6
-8.8
-4.6
6.5
-1.2
1.6
8.2
4.1
-2.7
6.4
-7.2
-2.3
-4.6
5
0.6
-5.4
-3.2
-10.2
-0.9
1.1
0.0
-13.8
-5.8
-2.4
5.7
-0.6
2.4
4.2
5.3
-3.7
3.4
-9.6
-1.7
-5.2
6
7.4
-3.0
-0.6
-1.6
0.5
1.1
0.6
-13.0
-8.0
-4.6
4.3
-2.4
2.4
0.8
-1.5
-13.1
1.8
-9.6
-5.7
-7.2
7
1.0
-5.2
-2.6
-9.2
-0.5
1.3
-0.4
-12.6
-4.8
-3.4
3.3
-3.4
-0.3
2.4
-0.5
-7.7
5.6
-9.0
-2.9
-5.4
8
0.6
-0.8
-0.6
1.2
^0.9
0.3
0.0
-6.6
-8.4
-5.2
3.3
-3.4
-0.8
1.4
-3.5
-8.1
-0.2
-9.8
-7.7
-6.6
9
3.0
1.2
0.8
0.2
-1.1
1.9
-0.2
-6.6
-9.4"
-3.8
Meter
10.0
13.6
9.2
23.8
12.3
4.7
7.6
34.0
25.4
12.4
13.1
8.6
9.0
11.2
14.3
31.3
13.2
16.8
18.7
14.6
10
-------
and
3.34 and -0.33 are the least-squares estimates of the intercept and slope, respectively.
The f-statistic for these data is tc = ^8.59 with 168 degrees of freedom, which is significant at the 5-percent
level. This equation holds over the range of average opacities studied, from 5 to 30 percent. By substituting
values into this equation, it is possible to predict what the deviation would be at a particular true average
opacity. This is done for selected opacities in Table 7.
TABLE?. PREDICTED DEVIATIONS,
TRAINING GENERATOR,
WHITE SMOKE
2. Precision
The precision of the determination of the average
opacity of a white plume from a training generator is determined
through use of an Analysis of Variance (ANOVA) technique.
From this, it is possible to estimate both the within-observer and
the observer bias components, and using these, to estimate the
between-observer component.
The precision estimates are obtained using only the
determinations from the eight observers who completed all 20
runs, which enables an estimate of the within-observer variance
to be made without blocking the data. The details of the ANOVA are given and the estimate obtained in
Appendix B.2.
The within-observer variance, o2 , is estimated as
a2 = 5.67
with 1 33 df . This gives an estimated within-observer standard deviation of
Percent
Opacity
5
10
15
20
25
30
Estimated
Deviation
1.81
0.16
-1.49
-3.14
-4.79
-6.44
= 2.38 percent opacity.
The F-ratio for observers isFc = 4.16. This exceeds the tabled value of 2.75, approximately,
from the F distribution with 7 and 133 df, at the 5-percent significance level. Thus, there is a significant
observer bias variance, a\ , and it is estimated by
CT? =0.90
lj
with 7 df. The estimated observer bias standard deviation, then, is
&L= \AX90
= 0.95 percent opacity.
The between-observer variance, o\, is estimated from the components above as
a| = 02-f&*
= 5.67 + 0.90
= 6.57.
11
-------
This gives a between-observer standard deviation of
= 2.56 percent opacity.
Predicted Results
By combining the above results, it is possible to predict confidence limits for the average opacity
determined by an observer. Let k be any true opacity level. Then the average opacity reported by the
observer could be expected to be within
expected range: k + (a + bk) ± (1 .96) a percent opacity
: k + (3.46 - 0.33*) ± (1 .96) (2.38) percent opacity
: (0.67k + 3.46) ± 4.66 percent opacity
at the 95-percent confidence level. The ranges for selected values of k are given in Table 8.
TABLES. PREDICTED DETER-
MINATIONS, TRAINING
GENERATOR-WHITE
SMOKE
Percent Opacity
5
10
15
20
25
30
35
Expected Range
2.15-11.47
5.50-14.82
8.85-18.17
12.20-21.52
15.55-24.87
18.90-28.22
22.25-31.57
Similarly, an estimate can be made of the difference
expected between two determinations of the same average plume
opacity made by independent observers. The difference between
determinations has a variance of 2 o|. Thus, the maximum expected
difference would be no more than
maximum difference: ± (1.96) \/2 a\,
: ±(2.77)dft
: ± (2.77) (2.56)
± 7.09 percent opacity
at the 95 percent confidence level.
B. Black Smoke
The 133 determinations made using a black plume are shown in Table 9 along with the average of the
transmissometer readings taken. These determinations are examined for accuracy and precision in the same
manner as were the white plume determinations.
1. Accuracy
The deviations from the transmissometer average are shown in Table 10. As before, there is a
tendency for the deviation to be greater as the true value increases. To obtain the best estimate of the devi-
ation, a linear regression of the deviations on the true opacity is done.
For these data, the line that best fits these points is
YI = 3.74 -0.34 Xi
12
-------
TABLE 9. DETERMINATIONS OF AVERAGE OPACITY OF TRAINING
GENERATOR, BLACK SMOKE
(Percent Opacity)
Run
No.
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Observer
1
24.4
1.4
20.0
17.6
7.4
9.6
13.6
13.2
12.6
12.0
18.6
29.2
14.8
7.8
15.0
22.0
2
20.4
9.6
20.2
14.4
11.4
5.2
11.0
14.0
10.2
10.8
18.2
17.0
11.4
8.0
12.2
20.2
3
22.0
10.0
22.6
15.6
12.6
5.8
12.0
13.4
16.4
13.8
17.2
25.2
15.8
11.4
15.2
23.2
4
23.2
10.4
24.0
19.6
14.0
4.6
11.0
14.2
16.2
14.0
21.0
29.4
17.4
11.4
20.4
24.2
5
19.0
10.6
22.2
17.2
12.8
8.8
10.4
10.0
11.4
11.6
17.2
29.4
15.0
10.0
15.8
23.0
6
21.4
7.6
20.8
19.6
12.0
5.6
12.0
13.8
14.8
12.4
23.0
31.2
17.4
11.6
16.6
24.4
7
21.6
11.2
23.2
20.0
13.0
4.8
11.2
11.6
11.2
11.8
17.8
30.0
13.4
8.4
15.8
22.0
8
21.8
8.4
21.8
20.2
13.0
4.8
12.4
11.8
15.0
13.6
19.2
28.6
13.0
8.6
15.6
19.8
9
25.4
12.0
23.0
17.0
12.4
Meter
35.0
10.8
26.0
20.0
12.3
16.0
11.0
17.6
17.6
13.6
18.0
28.2
13.8
9.0
14.8
22.2
TABLE 10. DEVIATION FROM TRAINING GENERATOR, BLACK SMOKE
(Percent Opacity)
Run
No.
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Observer
1
-10.6
-3.4
-6.0
-2.4
-4.9
-6.4
2.6
-4.4
-5.0
-1.6
0.6
1.0
1.0
-1.2
0.2
-0.2
2
-14.6
-1.2
-5.8
-5.6
-0.9
-10.8
0.0
-3.6
-7.4
-2.8
0.2
-11.2
-2.4
-1.0
-2.6
-2.0
3
-13.0
-0.8
-3.4
-4.4
0.3
-10.2
1.0
-4.2
-1.2
0.2
-0.8
-3.0
2.0
2.4
0.4
1.0
4
-11.8
-0.4
-2.0
-0.4
1.7
-11.4
0.0
-3.4
-1.4
0.4
3.0
1.2
3.6
2.4
5.6
2.0
5
-16.0
-0.2
-3.8
-2.8
0.5
-7.2
-0.6
-7.6
-6.2
-2.0
-0.8
1.2
1.2
1.0
1.0
0.8
6
-13.6
-3.2
-5.2
-0.4
-0.3
-10.4
1.0
-3.8
-2.8
-1.2
5.0
3.0
3.6
2.6
1.8
2.2
7
-13.4
0.4
-2.8
0.0
0.7
-11.2
0.2
-6.0
-6.4
-1.8
-0.2
1.8
-0.4
-0.6
1.0
-0.2
8
-13.2
-2.4
-4.2
0.2
0.7
-11.2
1.4
-5.8
-2.6
0.0
1.2
0.4
-0.8
-0.4
0.8
-2.4
9
-9.6
1.2
-3.0
-3.0
0.1
Meter
35.0
10.8
26.0
20.0
12.3
16.0
11.0
17.6
17.6
13.6
18.0
28.2
13.8
9.0
14.8
22.2
where
A
y< — the predicted deviation
Xf — the average opacity,
in the range from 9 to 35 percent average opacity. This equation is almost identical to that of the white
smoke data. The value of tc is -7.20, with 131 degrees of freedom which is significant at the 5-percent
level.
13
-------
TABLE 11. PREDICTED DEVIATION
FROM METERED OPACITY,
TRAINING GENERATOR-
BLACK
Average
Opacity
10
15
20
25
30
35
Predicted
Deviation
0.44
-1.21
-2.86
-4.51
-6.16
-7.81
Using this equation, deviations may be estimated for
plumes within the applicable range. This is done in Table 11.
2. Precision
The precision estimation for the black smoke determina-
tions is similar to that done for the white smoke data. The results
from the collaborator who did not complete all 16 runs are not in-
cluded. The remaining 128 determinations of average opacity are
used in an ANOVA to estimate the precision components. The de-
tails are presented in Appendix B.2.
The within-observer variance, o2, is estimated as
ft3 = 3.38
with 105 df. This gives an estimated within-observer standard deviation of
= 1.84,
The F-ratio for observers is Fc - 5.73 which exceeds the critical value, F0.0 5 (7, 1 05) « 2.10 at
the 5-percent level. Thus the observer bias variance, a£, is significantly different from zero, and is estimated
by
al =1.00
with 7df. Thus, the observer bias standard deviation is estimated by
&L = 1.00.
The between-observer variance is estimated from the above as
= 3.38 + 1.00
= 4.38.
The estimated between-observer standard deviation, then, is
= 2.09,
3. Predicted Results
Using the results above, it is possible to predict, at the 95-percent confidence level, the range of
opacities that would occur in the use of the method at particular opacity levels. The predicted determination
at fc-percent opacity would be
k + (a + bk) = k + (3.74 - 0.34*)
= 0.66fe +3.74.
14
-------
The variability in a reading would be measured by the within-observer standard deviation. For the 95-percent
confidence level and 105 df, the factor 1.96 is used. This gives
expected range: (0.66A + 3.74) ± 1.96 a
: (0.66*+3.74) ±1.96 (1.84)
: (0.66* + 3.74) ±3.61.
The expected range is given for selected values of k in Table 12.
TABLE 12. PREDICTED DETERMIN-
The maximum difference between two determinations
ATIONS OF AVERAGE OPACITY, made by independent collaborators is determined from these data at
TRAINING GENERATOR- the 95.perCent confidence level. The factor 1.96 is again used, and
BLACK ,,. .
this gives
Percent Opacity
5
10
15
20
25
30
35
Expected Range
3.43-10.65
6.73-13.95
10.03^17.25
13.33-20.55
16.63-23.85
19.93-27.15
23.23-- 30.45
maximum difference: ± 1.96 v2 o\
±(2.77) ab
±(2.77) (2.09)
± S.79 percent opacity.
at the 95-percent confidence level.
15
-------
V. STAUFFER CHEMICAL COMPANY TEST
The second test of the method was conducted at the Stauffer Chemical Company sulfuric acid plant.
This test was specifically designed to test the ability of a qualified observer to accurately read the plume
opacity near the compliance limit for that source. The standard for sulfuric acid plants is 10 percent, mean-
ing that an opacity of 10 percent or greater is in violation. '
The observers were chosen both from government enforcement agencies and from the private sector.
This was done to investigate whether the affiliation of the individual influenced his determination of opacity.
There were 14 runs made on the first day of the test by 11 collaborators, and 16 runs on the second with
9 collaborators for a total of 298 determinations. The average opacity during each run was determined from
a strip chart readout of opacity as determined by an in-stack transmissometer. Because of this, the "true"
opacity coujd only be determined to the nearest percent, and the observers' determinations were rounded to
the nearest percent opacity for comparison.
The observers made one deviation from the published method on the second day. Since the plume was
light, they suggested that they read to the nearest 1 percent rather than the nearest 5 percent, and this was
permitted. However, since the method specified that 5-percent increments be used, the individual readings
subsequently were rounded to the nearest 5 percent prior to being averaged to obtain the determinations used.
While there may be a difference between these and a Method 9 determination, it is felt that it would be slight.
The determinations are shown in Table 13 along with the average meter opacity for that range. These
values are investigated for both their accuracy and precision, as for the smoke generator test.
A. Accuracy
The deviations from the transmissometer average are shown in Table 14 for the sulfuric acid plant data.
The deviations show a negative bias, falling generally 1 to 3 percent below the metered opacity. To determine
if there is a linear relationship between the deviations and the meter average, a regression line is fit to these
points.
The equation of the best fit to these points is
deviation = —2.36 + (O.OSXmeter average).
From the slope of the line, it can be seen that there is a slightly positive trend to these deviations compared
to the negative trend of the smoke generator test.
To determine if the slope is significant, a r-statistic is calculated as described in Appendix B.I. The
value of the statistic is tc = 0.68 with 296 df. This is not significant, which implies that the slope, 0.05,
is not significantly different from zero. Thus, no linear relationship can be said to exist between deviation
and opacity for these data. The best estimate of the deviation, then, is the mean deviation of -2.0 percent
opacity. This is valid in the range studied, from 2 to 15 percent opacity. However, since the majority of the
runs were made between 5 and 10 percent average opacity, it is possible that there is a relationship between
the two variables that would be noticeable if a wider range of opacities was studied.
B. Precision
The determinations are submitted to an ANOVA procedure to estimate the precision of the method at
a sulfuric acid plant. The ANOVA was performed using only the nine observers who completed all 30 runs.
The remaining data set is large enough to obtain the desired estimates, and the necessity for blocking is
eliminated by their exclusion.
16
-------
TABLE 13. DETERMINATIONS OF AVERAGE OPACITY,
STAUFFER CHEMICAL
(Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Observer
1
7
5
5
5
7
5
5
5
18
7
1
5
5
5
0
0
0
0
9
6
5
5
3
3
4
5
8
6
6
7
2
3
1
0
0
2
3
3
3
10
5
0
2
2
3
0
0
0
0
8
9
9
7
4
5
6
9
9
9
10
10
3
7
7
7
6
6
6
6
6
15
8
3
5
4
5
1
1
1
0
8
7
5
5
4
3
5
7
7
6
8
5
4
6
5
4
3
5
5
5
5
15
7
1
5
5
3
0
1
2
0
10
12
6
5
5
3
6
10
13
16
15
16.
5
5
5
3
1
4
2
5
5
12
10
1
4
3
5
0
3
1
0
12
13
10
5
6
5
7
13
13
17
13
15
6
5
5
S
5
5
S
5
5
14
8
4
5
5
5
0
4
3
1
14
14
10
6
7
5
7 •
13
16
17
16
15
7
4
4
4
4
5
5
5
5
11
9
2
6
5
7
3
4
1
1
7
6
5
5
2
2
2
4
5
4
5
5
8
5
4
4
3
5
3
3
4
9
5
0
3
4
4
0
1
0
0
9
10
6
5
5
5
9
11
8
9
14
14
9
5
4
4
4
6
6
6
6
14
8
5
5
5
5
3
5
3
4
9
8
6
5
5
5
5
6
7
5
7
6
10
7
7
3
3
6
6
6
5
12
9
4
5
5
5
11
3
2
3
2
4
3
2
2
13
8
1
2
1
3
Meter
9
8
8
8
8
8
8
8
14
7
5
6
6
6
2
2
5
4
10
10
10
10
7
7
7
9
9
9
9
9
The precision estimates are taken from the ANOVA table in Appendix B.3. The within-observer com-
ponent is estimated by:
o2 =4.51
which has 232 df associated with it. This gives an estimated within-observer standard deviation of
= 2.12 percent opacity
The observer term for these data is shown in Appendix B.3 to be significantly different from zero. The
observer bias variance, o£, is estimated from the ANOVA table as
&l = 0.93
with 8 df. This gives an estimated observer bias standard deviation of
= 0.96 percent opacity
17
-------
TABLE 14. DEVIATIONS FROM METER AVERAGE, STAUFFER CHEMICAL
(Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Observer
1
-2
-3
3
-3
-1
-3
^3
-3
4
0
-4
_ j
1
-1
_2
-2
-5
-4
-1
-4
-5
-5
-4
-4
-3
_4
_j
-3
—3
-2
2
-6
-7
g
-8
-6
_5
-5
_5
_4
-2
-5
-4
4
-3
-2
-2
-5
~4
-2
— 1
— 1
-3
-3
-2
-1
0
0
0
1
1
3
-2
-1
1
-2
-2
-2
-2
-2
1
1
-2
-1
2
-1
-1
-1
-4
-4
-2
-3
-5
-5
-3
-4
-2
-2
-2
-3
-1
-4
4
-3
__3
4
-5
-3
-3
—3
-3
1
0
-4
-1
i
i
-3
-2
_j
-3
-4
0
2
^4
-5
-2
-4
1
1
4
7
6
7
5
-4
-3
5
-7
-4
-6
-3
-3
-2
3
_ j
-2
3
-1
-2
1
-4
-4
2
3
0
-5
1
-2
0
4
4
8
4
6
6
-4
-3
3
-3
-3
-3
-3
-3
0
1
_j
-1
i
— i
-1
—2
2
_2
-3
4
4
0
-4
0
-2
0
4
7
8
7
6
7
-5
-4
4
-4
-3
-3
-3
-3
-3
2
-3
0
j
1
-1
2
-4
-3
-3
_4
-5
-5
-5
-5
-5
_5
-4
-5
^t
-4
8
-4
-4
4
— *T
_5
-3
_5
-5
-4
-5
-2
-5
-3
^
— t.
-2
-2
^1
-5
-4
-1
0
-A
-5
-2
-2
2
2
-I
0
5
5
9
-4
-4
4
— t
_4
—2
-2
-2
_2
0
1
0
-1
i
— i
-1
1
3
-2
0
__2
-2
-4
-5
-2
-2
_2
_3
-2
-4
-2
-3
10
-2
-1
-5
-2
-2
-2
-3
-2
2
-1
-1
i
— i
-1
11
-6
-6
-6
-4
-4
-6
-6
— 1
1
-4
-4
c
— J
-3
Meter
9
8
8
8
8
8
8
14
7
5
6
6
2
2
5
4
10
10
10
10
7
7
7
9
9
9
9
9
The between-observer variance, a$ , is the sum of the within-observer and observer bias components.
The estimated value is
afc = tf + a2
= 0.93+4.51
= 5.44.
This results in a between-observer standard deviation of
ab=^/5A4
= 2.33 percent opacity.
C. Predicted Results
The above results can be used to make statements concerning the results that would be obtained by an
independent observer. The range of opacities that could be expected is calculated by using the best estimate
of the deviation, along with the variation that could be expected.
18
-------
At k percent opacity, the expected result would be
(k -2)% ±(1.96)5
at the 95-percent confidence level. The factor 1.96 is used since with 232 df, the distribution of the t statistic
is essentially a standard normal curve. Substituting in the value of ogives:
expected range : (k - 1)% ± (1.96X2.12)
: (k-2) ±4.16
: it-6.16 tofc + 2.16
for plumes of from 2 to 15 percent average opacity. For selected values, the expected range is calculated and
shown in Table 15.
TABLE 15. PREDICTED DETERMINATIONS,
SULFUR1C ACID PLANT
Avg Opacity
5
10
15
Expected Range
0*-7.16
3.84-12.16
8.84-17.16
*Truncated to zero.
The maximum expected difference between deter-
minations by two independent observers is calculated using
the between-observer variance term. As before, the difference
is
maximum difference = (2.306) \/2al
TABLE 16. OBSERVER MEANS,
SULFURIC ACID PLANT
Observer
1
2
3
*4
*5
*6
7
*8
9
Mean
5.00
4.40
5.47
6.47
6.60
7.63
4.57
5.40
5.73
•Observer from enforcement
agency.
largest overall means and the sixth largest.
opacity than the private sector people.
at the 95-percent confidence level. The factor 2.306 comes
from Student's t distribution with 8 df, due to the observer
bias term. Substituting gives
maximum difference = (3.26X2.33)
= 7.60 percent opacity
D. Comparison of Enforcement and Private Observers
The test personnel were chosen from both an enforcement
agency and from private concerns to allow the determinations
by each group to be compared. The nine collaborators who
completed all 30 runs are used, and the mean over the 30 runs
is computed. The observer means are shown in Table 16.
The observers from the enforcement agency were num-
bers 4, 5, 6 and 8. As can be seen, this group gives the three
The indication, then, is that these people tend to read a higher
The hypothesis that the two groups read the same average opacity is tested in Appendix B.3. The
test is a contrast among the means, and the test statistic follows Student's t distribution. The statistic is
tc - 5.69 with 232 df. This is a significant value and the conclusion is that the enforcement observers did
indeed determine higher average opacities overall than the other group.
The average opacity for all 30 runs, however, was 7.6, so that the enforcement observers were closer
to the true opacity than those from other agencies.
19
-------
VI. RIVERBEND STEAM STATION TEST
The three tests at the Riverbend Steam Station are treated separately in the analysis since there were
different groups of collaborators and varying sky conditions for the three tests.
The original test plan was designed to allow the evaluation of the method as used for the determination
of minutes of noncompliance with a particular standard/ ' A test run was set as 10 minutes of observation,
and the reported variable was to be how many observations exceeded a certain standard.
The experimental factors to be studied included the angle of observation and the relative experience
of the observer. Variations to the method to be evaluated included reading in 1 percent rather than 5 percent
increments and averaging the responses of two observers as opposed to a single observer's result.
i
In the third testing period, the test personnel were advised that the average opacity, based on 25 read-
ings, was the test variable of interest. The other factors for the test remained unchanged. For purposes of
comparison, average opacities were calculated from the first two data sets using the first 25 observations of
each run.
Due to the adverse sky and wind conditions during tests 1 and 2, not all of the planned evaluations are
useful. There was an inability to read the low opacity plumes against the type of background that existed,
and as a result, the determinations were generally well below the concurrent meter average. The precision
estimated, however, is independent of the accuracy of the determination.
The observers at each test were divided into two groups for the test, a control and an experimental.
The control group observed the plume at all times from a position consistent with the method as written and
read in increments of 5 percent. The experimental group either read the plume from a more extreme angle,
in increments of 5 percent or from the same angle as the control but in increments of 1 percent. Each group
was composed both of observers who had considerable field experience with the method and of observers
who had relatively little such experience.
A. Test 1
There were 10 runs made at the first test by 10 observers. The determinations of average opacity
calculated from the observer record sheets are shown in Table 17. These determinations will be evaluated
with respect to their accuracy and precision. There are six missing determinations in the data set. These
TABLE 17. AVERAGE OPACITY DETERMINATIONS, STEAM
STATION TEST 1
Run
1
2
3
4
5
6
7
8
9
10
Control Group
1
6.4
13.4
11.6
5.4
8.4
14.8
17.6
9.4
1.2
*
2
4.4
8.2
11.0
4.4
6.4
7.6
9.6
3.2
0.2
*
3
6.2
11.2
10.6
6.0
7.8
12.4
12.4
4.0
2.4
*
4
5.2
9.6
13.8
6.8
6.6
14.6
14.2
9.6
5.6
*
5
5.8
12.8
12.6
4.8
11.4
9.0
11.4
1.8
0.6
*
Meter
16.0
23.5
26.0
11.6
15.7
26.1
24.4
14.2
13.0
13.4
Experimental Group
1
5.4
6.6
10.8
5.2
7.6
12.2
7.5
4.4
1.6
2.0
2
5.4
6.2
9.4
5.5
5.8
8.5
9.4
6.0
3.0
3.0
3 •
1.8
3.8
7.8
4.1
5.9
7.5
4.6
1.0
0.2
0.0
4
5.8
6.4
13.0
5.8
5.9
15.0
8.6
5.0
1.0
0.0
5
2.2
4.4
•8.8
6.5
9.5
8.9
7.0
2.2
0.4
*
* Incomplete data sheet.
20
-------
occur when due to sky and wind conditions and interference from another plume, the observer felt that at
times lie was unable to make a determination of the opacity, and in these instances, a direct comparison
would not be valid.
1. Accuracy
It is apparent that these determinations are generally considerably lower than the meter average.
The deviations from the meter average are shown in Table 18.
TABLE 18. DEVIATIONS FROM METERED OPACITY, STEAM STATION TEST 1
Run
1
•>
3
4
5
6
7
8
9
10
Control Group
I
-9.6
-10.1
-14.4
-6.2
-7.3
-11.3
-6.8
-4.8
11.8
*
2
-11.6
-15.3
-15.0
-7.6
-9.3
18.5
-14.8
- 1 1 .0
12.8
*
3
-9.8
-12.3
-15.4
-5.6
-7.9
- 13.7
-12.0
-10.2
-10.6
*
4
-10.8
-13.9
-12.2
-4.8
-9.1
-11.5
-10.2
-4.6
-7.4
*
5
-10.2
-10.7
-13.4
-6.8
-4.3
-17.1
-13.0
-12.4
-12.4
*
Meter
16.0
23.5
26.0
11.6
15.7
26.1
24.4
14.2
13.0
13.4
Experimental Group
1
-10.6
-16.9
-15.2
-6.4
-8.1
-13.9
-16.9
-9.8
-11.4
-11.4
2
-10.6
-7.3
-16.6
-6.1
-9.9
-17.6
-15.0
-8.2
-10.0
-10.4
3
-14.2
-19.7
-18.2
-7.5
-9.8
-18.6
-19.8
-13.2
-12.8
-13.4
4
-10.2
17.1
-13.0
-5.8
9.8
- 11.1
-15.8
-9.2
-12.0
-13.4
5
-13.8
-19.1
-17.2
-5.1
-6.2
-17.2
-17.4
-12.0
-12.6
*
"Incomplete Data Sheet.
To assess the accuracy of the method under these conditions the control group data is used.
These observers had taken the optimum viewing position consistent with the method, and thus are consid-
ered to best represent the accuracy that could be expected of the method.
The deviations are fit to a regression model as with the data from the previous tests. The least-
squares fit to these points is given by
deviation = -3.01 - 0.40 (meter average).
The / statistic for this fit is tc = 5.89 with 43 df. This far exceeds the critical value of t 05 (43)
* 2.02, and this equation may be said to be valid in the range up to about 25 percent opacity.
TABLE 19. PREDICTED DEVIATION
FROM METERED OPACITY, STEAM
STATION TEST 1
Substituting into this equation, it is possible to predict
what the deviation would be at selected opacities in the applicable
range. This is done for the 5 percent increments and the results
shown in Table 19. As can be seen, a plume opacity up to 10 percent
is barely distinguishable from zero under these conditions.
2. Precision
The precision estimates are obtained using both groups'
determinations from runs where the experimental group was read-
ing in 5-percent increments. The 1-percent increment data were
intended only to evaluate that variation as a possible improvement to the published method when working
on low opacity plumes, and due to the viewing conditions, no evaluation using these data is performed. The
precision estimation is shown in Appendix B.4, using runs 1, 2, 3. 7, 8 and 9, omitting run 10 where there
were 6 incomplete determinations.
Percent Opacity
5
10
15
20
25
Predicted Deviation
-5.01
-7.01
-9.01
-11.01
-13.01
21
-------
The estimated within -observer variance is
o2 =3.31
with 45 df. This gives an estimated standard deviation of
& = 1 .82 percent opacity
The observers are determined to be a significant source of difference between observed values. The F-ratio
is 4.50 which exceeds the critical value of 4.08 with 8 and 54 df. Thus, of, is estimated as
(£-1.93
with 8 df. The estimated observer bias standard deviation is
= 1.39 percent opacity.
Combining the above estimates, the between-observer component can be estimated. The between-observer
variance, cr|, is estimated by
= 3.31 -I- 1.93
= 5.24.
Thus, the estimated between-observer standard deviation is
ab = V5. 24
= 2.29 percent opacity
3. Predicted Results
Using the results of the previous two sections, it is possible to estimate the range of opacities
that could be expected from a qualified observer determining average opacity under these conditions.
The expected range at any level, k, of true opacity would be given by
expected range: k + [a + bk] ± ( 1 .96) 0
at the 95-percent confidence level. Substituting gives
expected range: k + [-3.01 + 0.40A:] ± 1.96(1.82)
: [0.60* -3.01] ±3.57.
The range is calculated for selected opacities and shown in Table 20. As expected, there are no
instances where an accurate determination can be expected under these conditions.
The maximum difference that could be expected between two observers is estimated using the
between-observer standard deviation. The difference would not be expected to exceed
22
-------
TABLE 20. PREDICTED DETER-
MINATIONS, STEAM STATION
TEST 1
Percent Opacity
5
10
15
20
25
Expected Range
0*
0*
2.42
5.42
8.42
3.56
-6.56
9.56
-12.56
-15.56
"Truncated to zero.
TABLE 21. OBSERVER MEANS
FOR STEAM STATION
TEST I
Group
Control
Experimental
Observer
1
2
3
4
5
1
2
3
4
5
Mean
Determination
9.93
6.10
7.80
9.67
7.50
6.05
6.57
3.20
6.63
4.17
maximum difference = ?o.05
= (2.306X1.414X2.29)
= 7.49 percent opacity
at the 95-percent confidence level.
4. Experience Effect
The means of the observers are shown in Table 21 and these are used to determine if there is any
significant difference between the average opacity as determined by an experienced as opposed to an inex-
perienced observer. The test statistic is a /'•statistic based upon a contrast among the means. The details are
shown in Appendix B.4.
The calculated r-statistic is
tc= 1.39
with 45 df. There can be said to be a difference between the two groups at the 5-percent level only if tc
exceeds the tabled value. Since /o,05 (45) * 2.016, the difference is not significant at the 5-percent level.
Thus, for these conditions, the relative experience of the observer had no noticeable effect.
B. Test 2
There were 18 runs made at the second test period with 10 collaborators participating. The personnel
breakdown was identical to the first test, with 4 experienced and 6 relatively inexperienced observers. One
run, run 10, could not be completed. During the course of the run, the opacity meter began its automatic
calibration check. During this time, the meter displays only the calibration opacities, 0 and 95 percent, and
no true value is available for comparison. Only ten readings were made before the run was halted, and thus
no determinations could be calculated.
The determinations made by the observers are shown in Table 22. As in the previous test, missing
determinations occur when an observer is unable to make a determination of the plume opacity due to wind
and sky conditions.
23
-------
TABLE 22. AVERAGE OPACITY DETERMINATIONS, STEAM STATION
TEST 2
Run
I
2
3
4
5
6
7
8
9
11
12
13
14
IS
16
17
18
Control Group
1
11.2
13.8
10.0
5.8
20.0
5.0
13.8
5.0
14.4
5.2
5.6
5.2
9.4
17.0
7.8
12.4
7.6
2
5.8
14.4
9.2
5.4
15.0
0.4
11.4
5.4
10.6
4.0
2.2
0.6
7.0
12.8
7.0
8.0
6.0
3
11.6
13.2
9.2
5.6
12.6
4.4
9.4
6.6
10.8
5.0
7.8
5.0
7.6
16.8
10.8
10.0
6.4
4
10.6
.14.8
12.0
6.2
16.2
1.0
14.2
5.2
15.2
7.6
6.4
0.0
11.8
21.8
9.8
12.4
7.6
5
4.2
5.6
5.2
3.4
13.6
0.0
10.4
0.8
10.6
4.4
2.6
0.0
7.4
15.2
3.2
8.2
5.0
MCtcr
14.4
13.2
11.2
5.6
16.4
6.2
11.6
9.5
15.0
7.5
10.8
4.7
10.1
23.6
13.5
13.4
13.6
Experimental Group
1
8.0
13.7
13.0
4.6
20.0
0.4
*
1.8
8.6
4.4
2.7
0.0
6.6
13.6
7.0
8.4
5.0
2
6.1
7.2
5.2
1.4
13.8
1.6
8.0
5.6
6.0
5.0
3.4
0.0
4.8
13.0
6.0
7.6
6.0
3
4.3
9.4
3.4
0.2
11.4
0.0
6.6
0.6
5.4
0.1
0.0
0.0
0.6
9.8
2.3
1.8
2.2
4
11.7
11.3
4.2
3.4
17.8
0.0
*
4.2
9.4
2.6
2.5
0.0
6.4
15.6
5.4
10.0
7.8
5
8.3
8.6
9.0
4.0
19.0
3.0
10.6
3.8
8.8
3.6
1.8
0.0
2.0
12.3
4.0
7.0
6.6
"Incomplete data sheet.
|Run 10 could not be completed.
1. Accuracy
The deviations of the determinations from the meter average are shown in Table 23. As before,
the determinations are generally below the meter due in large part to the viewing conditions. A linear regres-
sion of the deviations is used to assess the accuracy, and as in the first test, only the control group data is
used.
The best fit to these points is given by
deviation = — 1.02 — 0.19 (meter average)
for the range from 5 to 25 percent average opacity. The t statistic for this model is tc - 2.56 with 83 df,
which is significant at the 5-percent level.
Using the above equation, it is possible to predict what deviation would be expected at various
opacity levels. This is done in Table 24 for selected values in the applicable range. These are somewhat better
than those from the first test, but still have a strong negative bias at the low range of opacities.
2. Precision
The precision components for this test will be estimated using runs 3—9, 13, 14, 17 and 18. The
remaining runs were those where the experimental group was reading in 1-percent increments, and a direct
comparison between the two sets of readings is not proper.
The precision components are estimated by an ANOVA procedure comparable to that for the
first test. The details and ANOVA table are shown in Appendix B.5. The within-observer component is
estimated from the error term of the ANOVA.
24
-------
TABLE 23. DEVIATIONS FROM METERED OPACITY, STEAM STATION TEST 2
(Percent Opacity)
Run
1
2
3
4
5
6
7
8
9
11
12
13
14
15
16
17
18
Control Group
12345
-3.2 -8.6 -2.8 -3.8 -10.2
0.6 1.2 0.0 1.6 -7.6
1.2 -2.0 -2.0 0.8 -6.0
0.2 0.2 0.0 0.6 -2.2
3.6 1.4 -3.8 -0.2 -2.8
-1.2 -5.8 -1.8 -5.2 -6.2
2.2 -0.2 -2.2 2.6 -1.2
-4.5 -4.1 -2.9 -4.3 -8.7
-0.6 -4.4 -4.2 0.2 -4.4
-2.3 -3.5 -2.5 0.1 -3.1
-5.2 -8.6 -3.0 -4.4 -8.2
0.5 -4.1 0.3 -4.7 -4.7
-0.7 -3.1 -2.5 1.7 -2.7
-6.6 -10.8 -6.8 -1.8 -8.4
-5.7 -6.5 -2.7 -3.7 -10.3
-1.0 -5.4 -3.4 -1.0 -5.2
-6.0 -7.6 -7.2 -6.0 -8.6
Meter
14.4
13.2
11.2
5.6
16.4
6.2
11.6
9.5
15.0
7.5
10.8
4.7
10.1
23.6
13.5
13.4
13.6
Experimental Group
1 2345
-6.4 -8.3 -10.1 -2.7 -6.)
0.5 -6.0 -3.8 -1.9 -4.6
1.8 -6.0 -7.8 -7.0 -2.2
-1.0 -4.2 -5.4 -2.2 -1.6
3.6 -2.6 -5.0 1.4 2.6
-5.8 -4.6 -6.2 -6.2 -3.2
* -3.6 -5.0 * -1.0
-7.7 -3.9 -8.9 -5.3 -5.7
-6.4 -9.0 -9.6 -5.6 -6.2
-3.1 -2.5 -7.4 -4.9 -3.9
-8.1 -7.4 -10.8 -8.3 -9.0
-4.7 -4.7 -4.7 -4.7 -4.7
-3.5 -5.3 -9.5 -3.7 -8.1
-10.0 -10.6 -13.8 -8.0 -11.3
-6.5 -7.5 -11.2 -8.1 9.5
-5.0 -5.8 -11.6 -3.4 -6.4
-8.6 -7.6 -11.4 -5.8 -7.0
*lncomplete Data Sheet.
TABLE 24. PREDICTED DEVIATION
FROM METERED OPACITY, STEAM
STATION TEST 2
Percent Opacity
5
10
15
20
25
Predicted Deviation
-1.97
-2.92
-3.87
-4.82
-5.77
The within-observer variance, a2, is estimated by
62 = 3.37
with 88 df. This gives an estimated standard deviation of
= 1.84 percent opacity
The observer-within-group term is used to determine
if there is a significant observer bias component. The F-ratio is
Fc = 8.37, which is significant at the 5-percent level. This is
equivalent to saying that the observer bias component, a|, is greater than zero. The observer bias variance
is estimated from the observer mean square as
&l = 2.26
with 8 df. The observer bias standard deviation is estimated by
»L =
= 1.50 percent opacity
The between-observer component is estimated from the above. The between-observer variance,
a?, is estimated as
= 2.26 + 3.37
= 5.63.
This gives an estimated standard deviation of
6b =V5.63
= 2.37 percent opacity
25
-------
3. Predicted Results
The accuracy and precision statements above can be combined to predict the performance that
could be expected when using Method 9 under these conditions.
The range of opacities that could be expected from a qualified observer at fc-percent opacity, at
the 95-percent confidence level, is given by
expected range: k + [a + bk] ± 1.96 &
: fc +[-1.02-0.19*]+ 1.96(1.84)
: [0.81*- 1.02] ±3.61
TABLE 25. PREDICTED DETER-
MINATIONS, STEAM STATION
TEST 2
Percent Opacity
5
10
15
20
25
Expected Range
0*-6.64
3.47-10.69
7.52-14.74
11.57-18.79
15.62-22.84
*Truncated to zero.
Values for this expected range are shown in Table 25 for
the range of opacities studied. As can be seen, the observer would be
expected to be biased on the low side above 10 percent opacity.
The between-observer standard deviation can be used to
estimate the difference that could be expected between two observers.
The maximum expected difference is calculated as
maximum difference = fo.osW V2 Of,
at the 95-percent confidence level. The 8 df comes from the observer
bias term. Substituting gives
expected difference = (2.306X1.414X2.37)
= 7.73 percent opacity
4. Experience Effect
The relative experience of an observer is investigated using the data from the ANOVA in Appendix
B.5. The means of the observers over the 11 runs are calculated, and these are shown in Table 26.
TABLE 26. OBSERVER MEANS FOR STEAM
STATION TEST 2
Group
Control
Experimental
Observer
1
2
3
4
5
1
2
3
4
5
Mean
Determination
9.48
6.76
7.82
8.76
5.42
6.84
5.20
2.56
6.32
6.32
The 1% variation runs are excluded to avoid the
possibility of this variation influencing the results. Run 7 is
also excluded, due to the two missing observations.
The details of the test for significance are presented
in Appendix B.5. The value of the test statistic is tc = 3.19.
From a Student's t distribution with 88 df, the critical value
is approximately 1.99, and the hypothesis of equality between
experienced and non-experienced observers is rejected since
tc exceeds 1.99.
C. TestS
There were 8 collaborators for the third test. Of these,
4 were from state and county agencies, 2 were from EPA
offices and 2 were from a private contractor. The observers were divided into two equal groups as in the two
previous tests. There were 10 observation runs on the first day and 14 on the second for a total of 192 indi-
vidual determinations. These are shown in Table 27, along with the meter average concurrently obtained.
26
-------
TABLE 27. AVERAGE OPACITY DETERMINATIONS. STEAM STATION
TEST 3
Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Control Group
1
5.0
13.6
5.0
9.0
10.4
6.6
11.0
5.8
7.6
6.8
25.0
40.8
11.2
17.0
12.8
26.0
13.4
29.2
23.8
23.0
33.8
17.4
23.8
12.6
2
5.2
13.6
5.0
8.4
6.8
6.6
10.2
6.0
5.0
6.0
23.0
39.0
9.8
15.2
10.8
25.6
12.4
25.2
23.0
19.4
34.6
12.6
19.8
15.0
3
3.0
15.4
4.0
8.6
8.8
5.4
8.4
5.8
5.4
5.4
36.2
33.0
5.6
6.4
7.8
19.0
11.2
24.0
18.4
16.6
30.0
10.0
18.0
9.4
4
5.0
5.4
5.0
5.2
5.2
5.0
5.6
5.2
5.0
5.0
16.8
39.6
6.2
10.2
10.8
25.2
7.6
25.0
21.2
12.4
33.2
8.0
19.2
6.8
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
Experimental Group
1
4.2
12.2
4.2
7.8
9.2
5.6
5.4
3.2
3.0
3.8
15.3
25.2
10.2
18.0
14.4
29.8
16.6
30.0
21.8
22.8
34.6
14.0
21.2
15.0
2
4.6
8.6
3.0
7.0
6.4
6.0
5.6
2.0
1.6
2.0
13.5
28.7
9.0
14.2
16.6
26.6
16.4
30.6
25.6
21.6
40.2
13.4
25.2
9.2
3
2.1
6.1
2.4
1.8
3.8
4.0
4.0
2.0
3.3
2.3
16.5
25.7
8.6
12.2
9.6
24.2
10.0
22.0
15.8
13.5
31.6
11.8
22.2
8.8
4
3.1
10.8
5.0
6.8
7.8
4.8
6.0
1.6
4.5
5.3
10.2
18.1
7.4
8.8
12.8
21.6
12.8
27.2
21.6
18.8
31.8
12.2
24.8
12.8
To evaluate the observer performance at this test, the accuracy and precision will be estimated as for
the first two sites. In addition, the relative performance of the tested variations to the method will be evaluated.
1. Accuracy
The evaluation of the accuracy of the method is performed using the deviations from the meter
obtained from the control group data as shown in Table 28. These deviations were all from determinations
made within the constraints of the method, and thus offer the most applicable information.
TABLE 28. DEVIATIONS FROM METERED OPACITY, STEAM STATION TEST 3
Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Control Group
1
1.9
6.1
0.8
1.8
3.7
1.6
5.6
2.4
3.4
2.0
3.8
4.8
0.9
2.8
0.6
-3.6
-1.2
0.9
- 10.8
-4.2
-3.1
2.2
-4.0
-1.8
2
2.1
6.1
0.8
1.2
0.1
1.6
4.8
2.6
0.8
1.2
1.8
3.0
-0.5
1.0
-1.4
-4.0
-2.2
-3.1
-11.6
-7.8
-2.3
-2.6
-8.0
0.6
3
-0.1
7.9
-0.2
1.4
2.1
0.4
3.0
2.4
1.2
0.6
15.0
-3.0
-4.7
-7.8
-4A
-10.6
-3.4
-4.3
-16.2
-10.6
-6.9
-5.2
-9.8
-5.0
4
1.9
-2.1
0.8
-2.0
-1.5
0.0
0.2
1.8
0.8
0.2
-4. 4
3.6
-*.\
-^.0
-1.4
-4.4
-7.0
-3.3
-13.4
-14.8
-3.7
-7.2
-8.6
-7.6
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
Experimental Giou
1
1.1
4.7
0.0
0.6
2.5
0.6
0.0
-0.2
-1.2
-1.0
-5.9
-10.8
-0.1
3.8
2.2
0.2
2.0
1.7
-12.8
-4.4
-2.3
-1.2
-6.6
0.6
2
1.5
1.1
-1.2
-0.2
-0.3
1.0
0.2
-1.4
-2.6
-2.8
-7.7
-7.3
-1.3
0.0
4.4
-3.0
1.8
2.3
-9.0
-5.6
3.3
-1.8
-2.6
-5.2
3
-1.0
-1.4
-1.8
-5.4
-2.9
-1.0
-1.4
-1.4
-0.9
-2.5
-4.7
-10.3
-1.7
-2.0
-2.6
-5.4
-4.6
-6.3
-18.8
-13.7
-5.3
-3.4
-5.6
-5.6
4
0.0
3.3
0.8
-0.4
1.1
-0.2
0.6
-1.8
0.3
0.5
-11.0
-17.9
-2.9
-5.4
0.6
-8.0
-1.8
-1.1
-13.0
-8.4
-5.1
-3.0
3.0
-1.6
27
-------
As was the case with the smoke generator test, there appears to be a relationship between the
magnitude and direction of the deviation and the meter average. To test this, a linear regression model is fit
in the manner of Appendix B.I. The least-squares line obtained is
deviation = 2.27 - (0.24)(meter average).
The t statistic for this test is tc = 6.10 with 94 df, which is significant at the 5-percent level. This equation
is valid in the range from 5 to 35 percent average opacity.
By substituting into the equation, it is possible to predict what the deviation would be at
different levels of true average opacity. This is done in Table 29. These predicted deviations are
comparable to those obtained in the smoke generator test. The observers are fairly accurate in the low
opacity ranges, but acquire a low bias as the opacity increases.
For comparison, the deviations from the meter are calculated for the experimental group as
well. The three cases, 45°, 90°, and 1 percent are treated separately and regression lines calculated for
each case. These are:
TABLE 29. PREDICTED DEVIATION
FROM METERED OPACITY,
STEAM STATION TEST 3
Percent Opacity
5
10
15
20
25
30
35
Predicted Deviation
1.07
-0.13
1.33
-2.53
-3.73
-4.93
-6.13
deviation = -0.68 - 0.03 (opacity)
deviation = 1.34-0.19 (opacity)
deviation = 1.76 - 0.39 (opacity)
respectively, for the three cases. The first line does not
have a significant slope, while the second two do. However,
it is easy to see that the three slopes are different, and
thus that the observers' determinations differed when the man-
ner and position of observation differed.
It is interesting to note that the most accurate readings were made when the group was at an
approximate 45° angle to the sun. T-he mean deviation was —1.18 over the range from 5—32 percent average
opacity.
2. Precision
The model used for estimating the precision of the method is similar to that used in the two previous
tests. The variance components are estimated from an ANOVA of all determinations made by both groups
on the 45° and 90° runs. The ANOVA is presented in detail in Appendix B.6. Precision estimation using the
1 percent variation runs is contained in a separate evaluation.
The estimated within-observer variance is
o2 = 3.57
with 90 df. The estimated within-observer standard deviation, then, is
§=V/3.57
= 1.89 percent opacity
The F-ratio for observers/groups isFc = 14.99. From a table of the F distribution, the critical
value at the 5-percent significance level is FQ.OS (6,90) « 2.30. Thus, the observer/group term is significant,
28
-------
which is equivalent to saying that an observer bias variance term,
estimated by
, does exist. The observer bias variance is
with 6 df. The estimated observer standard deviation, then, is
= 1 .77 percent opacity.
The between-observer variance is estimated from the above two terms as
= 3.12 + 3.57
= 6.69.
This gives a between-observer standard deviation of
ab = V6 .69
= 2.59 percent opacity.
3. Predicted Results
The above results can be combined to give estimates of the type of average opacities that would
be expected from field use of the method. The expected range of opacities that would be reported by an
independent observer are obtained by using the predicted deviation and the within-observer standard deviation.
The range of opacities, at k percent opacity, is defined as
expected range: [k + predicted deviation] ±(1.96)a
at the 95-percent confidence level. The normal variate 1.96 is used since there are 69 df for the within-observer
component. Substituting gives
expected range: k% + 2.27 - (0.24)* ± (1 .96X1 .89)
(0.76)Jt + 2.27 ± (3.70) percent opacity
TABLE 30. PREDICTED DETERMINATIONS,
STEAM STATION TEST 3
Percent Opacity
5
10
15
20
25
30
35
Expected Range
2
6
37-9.77
17-13.57
9.97-17.37
13
17
21
25
77-21.17
57-24.97
37-28.77
1^-32.57 '
Values for the expected range are given in Table 30 for 5-per-
cent increments in the applicable range. Similar to the pre-
vious results, these ranges contain the true opacities at the
low end, but acquire a negative bias beginning at the 25-per-
cent level.
The between-observer component is used to
estimate the difference that would be expected between
two observers making independent observations of the plume
opacity. The maximum difference that would be expected
29
-------
maximum difference = (Q 05 (6) -v/lSj,
= (2.447)(1.414)(2.59)
= 8.96 percent opacity
at the 95-percent confidence level.
The t value has 6 df from the observer bias variance term. This maximum difference is slightly
larger than the numbers previously obtained.
4. Experience Effect
The observer means shown in Table 31, are used to test the hypothesis that the experienced
observers read the same average opacity as the inexperienced. The test is a contrast among the means, and
the details are given in Appendix B. 6.
TABLE 31. OBSERVER MEANS FOR
STEAM STATION TEST 3
Group
Control
Experimental
Observer
1
2
3
4
1
2
3
4
Mean
Determination
15.31
14.00
11.40
11.46
14.95
14.46
11.19
12.76
The test statistic is based on Student's t distribution,
and compares the difference between the four experienced ob-
servers, 1, 2, 5, and 6, and the four inexperienced, 3, 4, 7, and
8. The value of the statistic is tc = 8.89 with 69 df. The tabled
value is ro.os(69) ** 2.00, and thus the hypothesis is clearly re-
jected. The conclusion, then, is that the experience observers
read different, and in this case higher, average opacities than the
inexperienced. By inspection of the deviations, it appears that
this difference occurs mainly in the high opacity range ( > 25%)
where the inexperienced observers read much lower than the
experienced. As it was previously, though, the experienced
observers compare more favorably to the meter average.
5. 1'Percent Increment*
On runs 1,2,9, 10, 11, 12, 19 and 20, the experi-
mental group read from the same viewing position as the con-
trol, but recorded their observations to the nearest 1 percent opacity rather than the nearest 5 percent. These
runs are considered separately from the others in terms of their accuracy and precision.
a. Accuracy
As the regression line in Section VI.C.l above indicates, there was a negative bias in the deter-
minations. Using that line, predicted deviations are calculated for the applicable range, and these are calculated
and presented in Table 32.
TABLE 32. PREDICTED DEVIATIONS,
1-PERCENT INCREMENT DATA
Opacity
5
10
15
20
25
30
35
Expected Deviation
—0.19
-2.14
^.09
-6.04
-7.99
-9.94
-11.84
In addition, the runs are broken down into
single readings, and the opacity from the meter is used to group
the observations. All observations made at the same opacity
level are treated as a single sample, and the accuracy of each is
calculated. The results are summarized in Table 33.
The meter values ranged from 3 to 62 percent
opacity. For each, a mean is calculated and a standard deviation.
A 95-percent confidence interval is calculated using the expression
CIo.95
30
-------
TABLE 33. 1-PERCENT DATA SUMMARY
Opacity
3
4
5
6
7
8
9
10
11
13
17
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
44
45
46
50
58
62
.X
3.3
3.2
4.6
9.4
6.0
9.5
9.8
10.4
11.0
14.3
13.3
13.5
13.9
13.3
15.1
14.9
20.0
18.3
19.1
17.3
18.6
20.0
20.3
20.3
21.4
22.2
20.7
20.4
22.8
23.5
27.0
21.3
22.8
32.3
31.0
30.5
32.0
36.3
36.5
,s
2.0
2.0
3.1
3.7
3.3
2.9
5.0
2.7
2.3
4.0
2.2
5.0
3.8
3.4
4.5
2.7
1.6
4.0
6.2
4.2
4.6
4.2
4.0
6.6
3.9
6.7
6.8
6.1
3.6
4.4
8.8
6.1
1.7
5.4
4.7
4.9
5.4
4.8
6.6
n
108
100
112
20
28
4
8
8
8
4
4
4
24
44
16
12
4
16
32
20
32
12
16
8
16
20
32
36
8
4
4
8
4
4
4
4
4
4
4
t
1.96
1.96
1.96
2.093
2.052
3.182
2.365
2.365
2.365
3.182
3.182
3.182
2.069
2.021
2.131
2.201
3.182
2.131
2.042
2.093
2.042
2.201
2.131
2.365
2.131
2.093
2.042
2.031
2.365
3.182
3.182
2.365
3.182
3.182
3.182
3.182
3.182
3.182
3.182
CI0.95
UL
3.677
3.592
5.174
11.132
7.280
14.114
13.981
12.658
12.923
20.664
16.800
21.455
15.505
14.336
17.497
16.616
22.546
20.431
21.338
19.266
20.260
22.669
22.431
25.819
23.478
25.336
23.155
22.465
25.810
30.500
41.001
26.401
25.505
40.891
38.478
38.296
40.591
43.937
47.001
LL
2.923
2.808
4.026
7.668
4.720
4.886
5.619
8.142
9.077
7.936
9.800
5.545
12.295
12.264
12.703
13.184
17.454
16.169
16.862
15.334
16.940
17.331
18.169
14.781
19.322
19.064
18.245
18.335
19.790
16.500
12.999
16.199
20.095
23.709
23.522
22.704
23.409
28.663
25.999
Max
10
9
16
17
14
13
18
15
15
19
15
19
23
26
24
20
22
28
28
25
25
25
26
28
25
30
34
36
28
28
39
28
25
38
36
37
38
40
44
Min
0
0
0
4
1
6
5
8
8
10
10
7
8
8
8
12
18
12
5
11
9
12
12
8
16
7
5
5
19
18
21
12
21
25
25
25
25
30
30
where CQ.OS is the tabled value,
taken either from the standard
normal or from Student's / distribu-
tion, and s is the calculated standard
deviation. The observations are
accurate when the "true value" lies
within the interval. As can be seen
from the table, the 1 -percent data
are generally accurate up to 20 per-
cent opacity, but beyond 20 percent
the accuracy standard is met only
once. Also included in the tables
are the maximum and minimum ob-
servations at each opacity level.
b. Precision
The precision of the
1 -percent variation is investigated
using an ANOVA, as before. The
details are contained in Appendix
B.7. The estimated within-observer
variance is
ft2 = 8.29
with 21 df. This gives an estimated
within-observer standard deviation
of
= 2.88 percent opacity.
This value is somewhat
larger than that obtained from the
5-percent variation. To test whether
or not these two variance terms are
equivalent, an F-ratio is calculated comparing this estimate to the variance estimate from the 5-percent in-
crement data. The statistic is Fc = 2.32 with 21 and 90 df. From the tables, the critical value for this test
is FQ 05 (2 1 ,90) « 1.75. Since the calculated F exceeds the tabled F, the two variances may be said to be
different.
The F-ratio for observers is Fc = 1 .84 with 3 df. This is not significant at the 5-percent level,
which indicates that there was no observer effect in these runs. Thus, a£ is taken to be zero, and
0\ = 8.29,
with 21 df as above.
c. Predicted Results
Using the results obtained above, the range of opacities that would be expected from the use
of this variation can be predicted. As before, the range is calculated as
31
-------
expected range: A: + predicted deviation ± tQQ5 (21) a
: *+ [1.76-0.39*] ±(2.080)(2.88)
: [0.71 k+ 1.76] ±(5.99)
Values of this range are listed in Table 34 for selected opacity levels. The data indicate that there is fairly
good agreement at the lower opacities, but that a negative bias appears above 25 percent. This is a com-
parable result to the 5-percent increment data.
The maximum difference that would be expected between observations by two independent
observers is estimated using the between-observer standard deviation. As before, the equation is
maximum difference = fu.05
= (2.080)v/f6"38
= (2.080X4.07)
= 8.47 percent opacity
As expected, this value is greater than that for the 5-percent increment data. The apparent
conclusion is that there was more variability in the 1-percent data than the 5 percent. A contributing factor
to this was undoubtedly that the observers had not been trained to read to the nearest percent, but it is
impossible to determine how much effect this actually had. From a second ANOVA shown in Appendix
B.7, the two groups of observers cannot be shown to be reading different average opacities on these runs.
TABLE 34. PREDICTED DETER-
MINATIONS, 1-PERCENT DATA
TABLE 35. OPACITY DETERMINATIONS
USING PAIRED OBSERVERS
Opacity
5
10
15
20
25
30
35
40
Expected Range
0*-11.30
2.87-14.85
6.42-18.40
9.97-21.95
13.52-25.50
17.07-29.05
20.62-32.60
21.17-36.15
*Truncated to zero.
6.
Two Observers
A second variation to be
studied involved taking the average
of two observers results as opposed to
a single observer's average. The 6
possible pairings among the control
group are made, and the determinations
are shown in Table 35 with the devia-
tions shown in Table 36.
Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Observer Pair
(1,2)
5.1
13.6
5.0
8.7
8.6
6.6
10.6
5.9
6.3
6.4
24.0
39.9
10.5
16.1
11.8
25.8
12.9
27.2
23.4
21.2
34.2
15.0
21.8
13.8
(1,3)
4.0
14.5
4.5
8.8
9.6
6.0
9.7
5.8
6.5
6.1
30.6
36.9
8.4
11.7
10.3
22.5
12.3
26.6
21.1
19.8
31.9
13.7
20.9
11.0
(1,4)
5.0
9.5
5.0
7.1
7.8
5.8
8.3
5.5
6.3
5.9
20.9
40.2
8.7
13.6
11.8
25.6
10.5
27.1
22.5
17.7
33.5
12.7
21.5
9.7
(2,3)
4.1
14.5
4.5
8.5
7.8
6.0
9.3
5.9
5.2
5.7
29.6
36.0
7.7
10.8
9.3
22.3
11.8
24.6
20.7
18.0
32.3
11.3
18.9
12.2
(2,4)
5.1
9.5
5.0
6.8
6.0
5.8
7.9
5.6
5.0
5.5
19.9
39.3
8.0
12.7
10.8
25.4
10.0
25.1
22.1
15.9
33.9
10.3
19.5
10.9
(3,4)
4.0
10.4
4.5
6.9
7.0
5.2
7.0
5.5
5.2
5.2
26.5
36.3
5.9
8.3
9.3
22.1
9.4
24.5
19.8
14.5
31.6
9.0
18.6
8.1
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
32
-------
There are 15 deviations of
absolute magnitude greater than 7.5 per-
cent out of the 144 pairings, and also
15 out of the original 96 determinations.
Thus, there was a decrease in the fre-
quency of the large deviation. The per-
centages are 0.10 as opposed to 0.16.
In Appendix B.8, it is shown that this
is a significant decrease in frequency.
The precision that would be
expected from this variation is esti-
mated by taking one-half of the between-
observer variance term from the previous
analysis. The variance between two
observations, then, would be
o£ = 1/2 (6.69)
= 3.35
and a standard deviation of
dh = 1.83 percent opacity.
TABLE 36. DEVIATIONS FROM OPACITY, PAIRED OBSERVERS
Run
1
2
3
4
5
6
7
8
9
10
It
12
13
14
15
16
17
18
19
20
21
22
23
24
Observer Pail
(1,2)
2.0
6.1
0.8
1.5
1.9
1.6
5.2
2.5
2.1
1.6
2.8
3.9
0.2
1.9
-0.4
-3.8
-1.7
--1.1
-11.2
-6.0
-2.7
-0.2
-6.0
-0.6
(1,3)
0.9
7.0
0.3
1.6
2.9
1.0
4.3
2.4
2.3
1.3
9.4
0.9
-1.9
-2.5
-1.9
-7.1
-2.3
-1.7
-13.5
-7.4
-5.0
-1.5
-6.9
-3.4
(1,4)
1.9
2.0
0.8
-0.1
1.1
0.8
2.9
2.1
2.1
1.1
-0.3
4.2
-1.6
-0.6
-0.4
-4.0
-4.1
-1.2
-12.1
-9.5
-3.4
-2.5
-6.3
-4.7
(2,3)
1.0
7.0
0.3
1.3
1.1
1.0
3.9
2.5
1.0
0.9
8.4
0.0
-2.6
-3.4
-2.9
-7.3
-2.8
-3.7
-13.9
-9.2
-4.6
-3.9
-8.9
-2.2
(2,4)
2.0
2.0
0.8
-0.4
-0.7
0.8
2.5
2.2
0.8
0.7
-1.3
3.3
-2.3
-1.5
-1.4
-4.2
-4.6
-3.2
-12.5
-11.3
-3.0
-4.9
-8.3
-3.5
(3,4)
0.9
2.9
0.3
0.3
0.3
0.2
1.6
2.1
1.0
0.4
5.3
0.3
-4.4
-5.9
-2.9
-7.5
-5.2
-3.8
-14.8
-12.7
-5.3
-6.2
-9.2
-6.3
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
The 95-percent confidence limit on the difference between two determinations made this way
would be
max imum difference: t$ _ Q 5 (6) \/2aft
: (2.447X1-414X1.83)
: 6.33 percent opacity.
This is a 29-percent reduction from the single observer determination result.
33
-------
VII. COMPOSITE ESTIMATION
As a final evaluation, it is investigated whether the precision and accuracy from all the tests may be
combined to produce statements that are applicable to any site. The accuracy statements are derived from
the training generator and Steam Station Test 3 data, where good background was available. The sulfuric
acid test data is not used since it applies to a much narrower range of opacities.
A. Accuracy
The slopes of the regression lines for the training generator and steam station test are tested for equal-
ity. A test statistic is calculated for comparing the three estimated slopes using their standard deviations and
an uncertainty factor. The details are given in Appendix B.9.
The slopes can be said to be estimating the same true value if they differ from each other by no more
than 0.12. As can be seen, the difference between the largest and the smallest slope estimate is 0.10, and
thus the three values are insignificantly different from one another.
Because of this, a composite line is calculated using all the points from the three sets of data. The
equation thus calculated is
deviation = 3.13 — (0.31) (percent opacity)
for the range from 5 to 35 percent average opacity. The expected deviation calculated using this equation is
shown in Table 37 for values in that range.
TABLE 37. PREDICTED DEVIATION B. Precision
FROM METERED OPACITY,
COMPOSITE ESTIMATE
Opacity
5
10
15
20
25
30
35
Expected Deviation
1.58
0.03
-1.52
-3.07
-4.62
-6.17
-7.72
The six estimates of the within-observer variance, aj, are com-
pared using Bartlett's tesr3'. The test statistic is a chi-square, and is
calculated to be 13.77 with 5 df, and the significance level for this
value is approximately 0.02. This is sufficient to accept the hypoth-
esis that all are estimating the same true variance.
The combined estimate is obtained by "pooling" the variances,
as described in Appendix B.9. The pooled variance is
with 693 df. This gives an estimated within-observer standard deviation of
= 2.05 percent opacity.
Similarly, the 6 values of o/,are tested for equality using Bartlett's test. The test statistic is 4.72 with
5 df, which is not significant. Thus, for these components as well, the hypothesis of equality is accepted.
The pooled estimate of the observer bias variance is
al = 1.66
with 44 df. This gives an estimated observer bias standard deviation of
aL = 1.29 percent opacity.
34
-------
Combining the above estimates, the between-observer variance is estimated as
= 4.22 + 1.66
= 5.88.
This gives a between-observer standard deviation of
= 2.42 percent opacity.
C. Predicted Results
Using the results of the previous two sections, the expected range and maximum expected difference
can be calculated for the composite values. The expected range would be
expected range: k + [3.13 -0.3 Ik] ± 1.96 d
: (0.69fc + 3.13)±1.96(2,05)
: (0.69* + 3.13) ±4.02
for any true opacity k, at the 95-percent confidence level. The expected range is presented for selected
values of k in Table 38. As before, the results show accuracy up to about 25 percent opacity, then acquire
a low-side bias.
TABLE 38. PREDICTED DETERMIN-
ATIONS OF AVERAGE OPACITY,
COMPOSITE ESTIMATE
Percent Opacity
5
10
15
20
25
30
35
Expected Range
2.56-10.60
6.01-14.05
9.46-17.50
12.91-20.95
16.36-24.40
19.81-27.85
23.26-31.30
The maximum difference that could be expected between two
independent observers is calculated using the between-observer standard
deviation. At the 95 percent confidence level, the difference could not
be expected to exceed
maximum difference: fo.os (44) \/2 a\
: (2.021) (1.414) (2.42)
: 6.92 percent opacity.
The 44 df are due to the observer bias term.
35
-------
-------
APPENDIX A. EPA METHOD 9 FOR VISUAL DETERMINATION
OF OPACITY
37
-------
-------
A.1. Federal Register, Vol 36, No. 247-Thursday, December 23, 1971
RULES AND REGULATIONS
24895
Horn, Jerome J-, Maintenance. Calibration,
and Operation of Isokinettc Source Sam-
pling Equipment. Environmental Protection
Agency, Air Pollution. Control Office Publi-
cation No. APTD-O576.
Shell Development Co. Analytical Depart-
ment, Determination of Sulfur Dioxide and
Sulfur Trioxide in Stack Oases, Emeryville
Method Series, 4516/5»a.
MITTHOD 9 VISUAL DETERMINATION OF THE
OPACITY OF EMISSIONS FROM STATTONARIT
SOURCES
1. Principle and applicability.
1.1 Principle. The relative opacity of an
emission from a stationary source is de-
termined visually by a qualified observer.
1.2 Applicability. This method is appli-
cable for the determination of the relative
opacity of visible emissions from stationary
sources only when specified by teat proce-
dures for determining compliance with the
New Source Performance Standards.
2. Procedure.
2.1 The qualified observer stands at ap-
proximately two stack heights, but not more
than a quarter ol a mile from the base of
the stack with the sun to his back. Prom a
vantage point perpendicular to the plume,
the observer studies the point of greatest
opacity in the plume. The data required In
Figure 9-1 is recorded every 15 to 30 seconds
to the nearest 5'." opacity. A minimum of 25
readings is taken.
3. Qualifications.
3.1 To certify as an observer, a candidate
must complete a smokereadlng course con-
ducted by EPA, or equivalent; in order to
certify the candidate must assign opacity
readings in 5% Increments to 25 different
black plumes and 25 different white plumes,
with an error not to exceed 15 percent on
any one reading and an average error not to
exceed 7.5 percent in each category. The
smoke generator used to qualify the ob-
servers must be equipped with a calibrated
smoke indicator or light transmission meter
located in the source stack, if the smoke
generator is to determine the actual opacity
of the emissions. All qualified observers must
pass this test every 6 months in order to
remain certified.
4. Calculation*.
4.1 Determine the average opacity.
6. References.
Air Pollution Control District Rule* and
Regulations, Loa Angeles County Air Pollu-
tion Control District, Chapter 2, Schedule 6,
Regulation 4. Prohibition, Rule SO, 17 p.
Kudluk. Rudolf. Rlngelmann Smoke Chart,
VJS. Department of Interior, Bureau of Mines,
Information Circular No. 8333, May 1967.
•==^11:
1
1
1
1
1
t
7
*
»
29
21
22
23
24
2S
26
27
3»
2*
.0
IS
30
4§
Utt
1 30
31
32
31
34
35
36
31
39
39
40
41
«2
43
44
4«
47
4»
49
M
51
62
S3
6*
S&
&«
57
&|
59
0
IS
30
— 1
1
4S
O**»rv»l>a« 4*1* "v
n.i.
Tim* , _
m;~t Al.mft^» iu-ii , , , ,.„_
Vl^>p*W., . »
Sum of no*. f«CO*rf*4
* Tt>t*l no. t ***'»• ft
... •
Figure 9-1. Field data.
[FR Doc.71-13624 Filed 12-22-71:8:45 am)
FEDEHAL REGISTER, VOL 36, NO. 247—THURSDAY, DECEMBER 23, 1971
39
-------
A.2. Federal Register, Vol 39, No. 177-Wednesday, September 11, 1974
PROPOSED RULES
32857
(e) For the purpose of reports pursu-
ant to § 60.7 adju.i»d
where:
OF=dally conversion factor (mg frs/dscm
per ppm). and
%Oj = percent oxygen by volume (expressed
as percent), dry basis.
* • • t •
Subpart L—Standards of Performance for
Secondary Lead Smelters
§60.122 [Amended]
16. Section 60.122 is amended by de-
leting paragraph (c).
Subpart M—Standards of Performance for
Secondary Brass and Bronze Ingot Pro-
duction Plants
§60.132 r Amended)
17. Section 60.132 is amended by de-
leting paragraph (c).
Subpart O—Standards of Performance for
Sewage Treatment Plants
§60.152 [Amended]
18. Paragraph
-------
32858
In that portion of the plume where condensed
water vapor is not present. The observer shall
not look continuously at the plume, but in-
stead shall observe the plume momentarily at
15-second intervals.
2.3.1 Attached steam plumes. When con-
densed water vapor is present within the
ulume as It emerges from the emission outlet,
opacity observations shall be made beyond
the point in the plume at which condensed
water vapor is no longer visible. The observer
shall record the approximate distance from
the emission outlet to the point at which
the observations are made.
2.3.2 Detached steam plume. When water
vapor in the plume condenses at a distinct
distance from the emission outlet, the opacity
of emissions may be evaluated at the emission
outlet prior to the condensation of water
vapor and the formation of the steam plume.
2.4 Recording observations. Opacity obser-
vations shall be recorded to the nearest 5
percent at 15-second intervals on an obser-
vational record sheet (Figure »-2). Each
momentary observation recorded shall repre-
sent the opacity of emissions for that 15-
second time period.
2.5 Data reduction. Determine the opacity
standard applicable to the affected facility
being observed and compute minutes of non-
PROPOSED RULES
erator. Plumes within each set of 26 black
and 25 white runs shall be presented In ran-
dom order. The candidate assigns an opacity
value to each plume and records his observa-
tion on a suitable form. At the completion of
each run of 50 readings, the score of the
trainee Is determined. If a trainee falls to
qualify, the complete run of 60 readings must
be repeated In any retest. The smoke test may
be administered as part of a smoke school
or training program, and may be preceded by
training or familiarization runs of the smoke
generator during which candidates are shown
black and white plumes of known opacity.
3.3 Smoke generator specifications. Any
smoke generator used for the purposes of
paragraph 3.2 shall be equipped with a smoke
meter Installed to measure opacity across the
diameter of the smoke generator stack. The
smoke meter output shall display in-stack
opacity based :ipon a pathlength equal to
the stack exit diameter, on a full 0 to 100
percent chart recorder scale. The smoke meter
optical design and performance shall meet
the specifications shown In Table 9-1. The
smoke meter shall be calibrated as prescribed
In paragraph 33.1 prior to the conduct of
each smoke reading test. At the completion of
each test, the zero and span drift shall be
ch :cked and If the drift exceeds ±1 percent
soclated electronic circuitry Including the
chart recorder or output meter, or every 6
months, whichever occurs first.
TABIJE 9-1 BMOKM lOTTEX DESIGN AND
FXBrOKlOANCX SPECIFICATIONS
Parameter:
a. Light source
b. Spectral response
of photocell.
c. Angle of view
d. Angle of projec-
tion.
e. Calibration error-
span
Specification
Incandescent lamp
operated at nomi-
nal rated voltage.
Photopic (Daylight
spectral response
of the human eye:
reference 4-3) .
16
comoiiance using the appropriate procedure opacity, the condition shall be corrected prior
as follows- **> conducting any subsequent test runs. The
2 5 1 For all opacity standards e»ept aver- smoke meter shall be demonstrated, at the
jure ooacitv mark on the observation record time of Installation, to meet the speclncft-
sheet (Fleure 9-2) all observation* which tlona listed. In Table »-l. This demonstration
exceed the opacity level specified by the shall be repeated following any subsequent
standard Compute the aggregate number of repair or replacement of the photocell or as-
OL,(tuu«uu. vi*."I**."— DO D J_i , .- • - ,.4w»il+w in»*1,iH4«f» +Vi&
minutes during which the opacity of emis-
sions was observed to exceed the applicable
standard. Exclude from this computation the
number of minutes of operation exempted
by the applicable opacity standard or by
§60.11(c) (If any). Record the minutes of
noncompllance on Figure 9-1.
2.5.2 For average opacity standards, sum
on the observation record sheet (Figure 9-3)
all observations for any specific time period
(use time periods specified by the standard
when applicable), and compute the arith-
metic mean average bv dividing by the total
number of observations lui ihe time period
evaluated. Exclude from tills computation
any minutes of opefivtion exempted by the
applicable opacity standard or S«O.ll(c) (If
any). Record the miuutes of each time period
of noncompllance on Figure 9-1.
3. Qualifications and teattttf.
3.1 Certification requirements. To receive
certification as a qualified observer, a candi-
date must be tested and demonstrate the
ability to assign opacity reading* In 5-percent
increments to 25 different black plumes and
25 different white plumes, with sn error not
to «xo»ed 15-percent opacity on any one read-
Ing and an average error not to exceed 7.5-
percent opacity in each category. Candidate*
shan be tested according to the procedures
described In paragraph 8-2. Smoke generators
used pursuant to paragraph 3.2 shall be
equipped with a smoke meter which meet*
the requirements of paragraph 33.
The certification shall be valid for a period
of 6 months, at which time the qualification
procedure must be repeated by any observer
In order to retain certification.
8.2 Certification procedure. The (moke test
consists of showing th« candidate a complete
run of 60 plumes—26 black plumes and 25
white plumes—generated by a smoke gen-
15* maximum total
angle.
±8* opacity maxl-
±1* opacity,
minutes.
30
t. Zero and
drtrt.
f. Response ttme.
3.3.1 Calibration. The smoke meter is cali-
brated after allowing t. minimum of SO min-
utes warmup by alternately producing stara-
lated opacity of 0 percent and 100 percent.
When stable response at 0 percent or 100
percent is noted, the smoke meter !• ad-
justed to produce an output of 0 percent or
100 percent, as appropriate. This calibration
shall be repeated until stable 0 percent and
1OO percent readings an produced without
adjustment. Simulated 0 percent and 100
percent opacity values may be produced by
alternately switching the power to the light
source on and off while the smoke generator
Is not producing smoke.
8.8.2 Smoke meter evaluation- The smoke
meter design and perfonnanoe are to be
evaluated tut follows:
S.3.2.1 Light source. Verify from manu-
facturer's data and from voltage measure-
ments made at the lamp, as Installed, that
the lamp Is operated within ±B percent of
the nominal rated voltage.
8.3.2.2Spectral response of photocell. Verify
from manufacturer's data that the photocell
has a photoplc response; I.e., the spectral
sensitivity of the cell shall closely approxi-
mate the standard spectral-luminosity curve
for photoplc vision which Is referenced in
(b) of Table 9-1.
3.3.2.3 Angle of view. Check construction
geometry to ensure that the total angle of
view of the smoke plume, as seen by the
photocell, does not exceed 15°. The total angle
of view may be calculated from: 0 — 2 tan-1
d/2L, where 0 —total angle of view, d —the
sum of the photocell diameter 4- the diameter
of the limiting aperture: and L = the distance
from the photocell to the limiting aperture.
The limiting aperture Is the point In the
path between the photocell and the smoke
plume where the angle of view is most
restricted. In smoke generator smoke meters
this Is normally an orifice plate.
3.3.2.4 Angle of projection. Check construc-
tion geometry to ensure that the total angle
of projection of the lamp on the smoke
plume does not exceed 16*. The total angle
of projection may be calculated from: »=2
tan-1 d/2L, when »=total range of projection;
d—the sum of the length of the lamp fila-
ment + the diameter of the limiting aper-
ture; and L=the distance from the lamp to
the limiting aperture.
3.1.2.5 Calibration error. Using neutral-
density filters of known opacity, check the
error between the actual response and the
theoretical linear response of the smoke
meter. This check Is accomplished by first
calibrating the smoke meter according to
3.3.1 and then Inserting a series of three
neutral-density filters of nominal opacity of
2O. SO, and 75 percent In the smoke meter
patblength. Filters calibrated within ±2
percent shall be used. Gsre should be taken
when Inserting the filters to prevent stray
light from affecting the meter. Make a total
of five noneonseeutlye readings for each filter.
The maximum error on any one reading
Bhsfl be 8 percent opacity.
3.3.2 8 Zero and span drift. Determine the
zero and span drift by calibrating and oper-
ating the smfAft generator in a normal man-
ner over a 1-hour period. The drift is
measured by checking the eero and span
at the end of this period.
3.3.2.7 Besponse time. Determine the re-
sponse time by producing a series of five
simulated O percent and 100 percent opacity
values and observing Uw time required to
reach stable response. Opacity values of 0
percent and 100 percent may be simulated by
alternately switching the power to the light
source off and on while the smoke generator
is not operating.
4. References.
4.1 Air Pollution Control District Ruiei and
Regulations. Los Angeles County Air Pollu-
tion Control District. Regulations IV, Pro-
hibitions. Bule SO.
4.2 Welsburd. Kelvin I, Field Operations
and Enforcement Manual for Atr, US. En-
vironmental Protection Agency, Besetrch
Triangle Park. N.C, APTD-UOO. August 1972,
pp. 4.1-4.36.
43 Condon, K. U, and Odtahaw, E., Hand-
book of Phytia*. McGraw-Hill Co, New Tort,
N.T, 1868, Table 8.1. p. »-62.
FEOCtAl REGISTf*, VOL S9, NO. 177—WEDNeSOAY, SETTEMBM 11, 1974
41
-------
PROPOSED RULES
32859
Figure 9-1
UCOU) or vaatL DBTERHINATIOII OF OPACITY
CCMftRT
tOCATiaT
HOTKS OF CBSntVATION
OKBtm
MM acmrr
COBOL WVICB
cnnnciTioi D
OBSBRVZ2 APFILIATIOH
FODfT OF EMISSIONS
BEICBT OF DISCHARGE FOINT
Record th« folloving Infonutlon prior to and opoa ecu
FAtloot at each source.
ECordiogs should b« mada
CDSOITQI LOCATIOH
BlltBncc to Discharg*
Direction ft cm DUch.rjs
Itollht of Obiervatloa Poin
BiCCOLOtM) DBSCUFnCM
•KiTBB connnons
VlDd Direction
Vlnd 3p«e
-------
32860
PROPOSED RULES
FIGURE 9-2 OBSERVATION RECORD
(Cont.)
PAGt
OF
COMPANY
LOCATION
TEST NUMBElT
DATE
OBSERVER
TYPE FACILITY
POINT OF EMISSlW
Hr.
Min.
30
31
32
33
3 it
35
36
37
38
39
1*0
1»1
U2
"«3
>|l»
i»5
*9
50
51
52
53
5>t
55
56
57
58
59
Seconds
0
Ib
30
45
STEAM PLUME
(check if applicable)
Attached
.
-
Detached
COMMENTS
-
20. Appendix B is added as follows:
APPENDIX B—PZHFOBMANCI SPECIFICATIONS
PEKFOBMANCE SPECIFICATION 1 PERFORMANCE
SPECIFICATIONS AND SPECIFICATION TEST FBO-
CKDUUS FOB TaANSMISSOMETES SYSTEMS FOB
CONTINUOI7S MEASUREMENT OF THE OPACITY
OF STACK EFFLUENTS
1. Principle and Applicability.
1.1 Principle. TransmlaBometry Is a direct
measurement of attenuation of visible radia-
tion (opacity) by particular matter In a
stack effluent. Light from a lamp Is projected
across the stack of a pollution source to a
light sensor. The light Is attenuated due to
absorption and scatter by the particulate
matter In the effluent. The percentage of light
attenuated is denned as the opacity of the
emission. Transparent stack emissions that
do not attenuate light will have a traremit-
tance of 100 or an opacity of 0. Opaque stack
emissions that attenuate all of the light will
hare a transmlttance of 0 or an opacity of
100 percent.
: .2 Applicability. This method is applicable
to the instrument systems specified in the
subparts for continuously monitoring the
opacity of emissions. Specifications for con«
t.r.uous measurement of visible emissions are
given in terms of design, performance, and
installation parameters. Performance specifi-
cs: ;o as and test procedures are given for
transmissometer systems to test their capa-
bility before approving the systems Installed
by an affected facility.
2. Apparatus.
2.1 Neutral Density Filters. Filters with
neutral spectral characteristics and known
optical densities to visible light. One each
low, mid, and high range filters, 5- or 6-lnch
square or 6-lnch diameter, with nominal op-
tical densities of 0.1, 0.2, and 0.3 (20, 37, and
SO percent opacity) are required. Calibrated
niters with accuracies certified by the manu-
facturer to within 3 percent shall be used. It
is recommended that filter calibrations be
checked with a well-collimated photopic
transmissometer of known linearity prior to
use.
2.2 Chart Recorder. Analog chart recorder
with Input voltage range and performance
characteristics compatible with ttie measure-
ment system output.
2.3 Opacity Measurement System. An In-
stack transmissometer (folded or single path)
with the optical design specifications desig-
nated below, associated- control units and ap-
paratus to keep optical surfaces clean.
3. Definition*.
3.1 Measurement System. The total equip-
ment required for the continuous determi-
nation of a pollutant concentration In a
source effluent. The system consists of three
major subsystems:
3.1.1 Sampling Interface—That portion of
the measurement system that performs one
or more of the following operations: delinea-
tion, acquisition, transportation, and condi-
tioning of a sample of the source effluent, or
protection of the analyzer from the effluent.
3.1.2 Analyzer—That portion of the sys-
tem which senses the pollutant and generates
a signal output that is a function of the
pollutant concentration.
3.1.3 Data Recorder—That portion of the
measurement system that processes the ana-
lyzer output and provides a permanent rec-
ord of the output signal in terms of pollutant
concentration.
3.2 Span. The value of opacity at which
the measurement system is set to produce
the maximum data display output. For the
purpose of this method, the span shall be set
at an opacity of 50 percent for a pathlength
equal to the stack exit diameter of the source.
3.3 Calibration Error. The difference be-
tween the opacity reading indicated by the
measurement system and the known values
of a series of test standards. For this method
the test standards are a series of calibrated
neutral density niters.
3.4 Zero Drift. The change in measurement
system output over a stated period of time
of normal continuous operation when the
pollutant concentration at the time of the
measurements is zero.
3.5 Calibration Drift. The change in meas-
urement system output over a stated period
of time of normal continuous operation
when the pollutant concentration at the time
of the measurements is the same known up-
scale value.
3.6 System Response. The time interval
from a step change In opacity In the stack
at the input to the measurement system to
the time at which 95 percent of the cor-
responding final value is reached as displayed
on the measurement system data presen-
tation device.
3.7 Operational Test Period. A minimum
period of time over which a measurement
system is expected to operate within cer-
tain performance specifications without un-
scheduled maintenance, repair, or adjust-
ment.
3.8 Transmittance. The fraction of in-
cident light that is transmitted through an
optical medium of Interest.
3.9 Opacity. The fraction of incident light
that Is attenuated by an optical medium
of Interest Opacity (O) and transmittance
(T) are related as follows:
O=1-T
3.10 Optical Density. A logarithmic meas-
ure of the amount of light that Is attenu-
ated by an optical medium of Interest Op-
tical density (D) is related to the trans-
mittance and opacity as follows:
D= -log,8T
D= -log,. (1-0)
3.11 Mean Spectral Response The wave-
length which bisects the total area under
the curve obtained pursuant to paragraph
9.2.1.
3.12 Angle of View. The maximum (total)
angle of radiation that Is seen by the photo-
detector assembly of an optical tranamls-
someter.
3.13 Angle of Projection. The maximum
(total) angle of radiation that is projected
by the lamp assembly of an optical trans-
missometer.
3.1* Pathlength. The depth of effluent In
the light beam between the receiver and the
transmitter of the single-pass transmis-
someter. or the depth of effluent between the
transceiver and reflector of a double-pass
transmissometer.
4. Installation Specif cation.
4.1 Location. The transmissometer must
be located across a section of duct or stack
that will provide a particulate matter flow
through the optical volume of the trans-
FEDHAL REGISTER, VOL 39, NO. 177—WEDNESDAY, SEPTEMBEt 11, 1974
43
-------
-------
APPENDIX B. STATISTICAL METHODS
45
-------
APPENDIX B. STATISTICAL METHODS
B.1 Linear Regression and Test for Significance
In order to obtain the best estimate of the average deviation from the metered opacity, the relationship
between the deviation and the meter average is investigated. To investigate this, the method of least-squares
is used to give the equation of the best straight line that fits the data obtained. The line is defined by its
slope and intercept, estimated by:
and
a = y - bx
where
Xi - the Ith value of the independent variable
Yj - the /th value of the dependent variable
x , y - the sample means of X and Y , respectively
n - the number of paired observations.
To determine if the line is a good estimate of the relationship between the two variables, the slope of
the line is tested. A linear relationship exists if the slope is non-zero. To test this, a statistic is used that is
based on Student's t distribution. The statistic is
where
tc - the calculated value
and
sb - the standard deviation of the slope, b. This statistic has n - 2 df associated with it for
n pairs of observations.
The standard deviation of the slope, sb , is calculated by first calculating the standard deviation around
the regression line, sy.x . The equation for the variance, S*y.x is:
H-2
46
-------
The variance of the slope, b, is then given by
4-
iS y i
Taking the ratio of the slope to its standard deviation, gives the desired statistic. To determine if this
is significant, the value is compared to a Student's t distribution with n - 2 df.
B.2 Precision Estimation for Training Generator Test
1. White Smoke
An analysis of variance was performed on the average opacities determined by the 8 collaborators
who completed all 20 runs. The model is a random effects model represented by
where
y,-y - the determination by observer; in run/
p — overall mean
Pi - the effect of run i
7, - the effect of collaborator;
- the random error associated with Yn.
and
This is an additive or no-interaction model. This means that the collaborator and run effects are assumed to
be independent of each other, or that an observer reads consistently low or high with respect to the other
observers regardless of the opacity level.
TABLE Bl. ANALYSIS OF VARIANCE FROM TRAINING
GENERATOR TEST, WHITE SMOKE
Source
Runs
Observers
Error
Total
df
19
7
133
159
SS
5545.06
165.04
754 .51
6464.61
MS
_*
23.58
5.67
F
_*
4.16f
EMS
_*
CT2 + 20 a}
„' •*•
*Not of interest.
tSignificant. FC > FQ 05(7,133) * 2.09.
The ANOVA results are shown in
Table Bl. The expected mean squares indicate
the proper F-ratios and the means of obtaining
estimates of the desired precision components.
The EMS of the error term is a1, the within-
observer variance. Thus, the error mean square
is the estimated within-observer variance, and
this gives
a2 = 5.67
with 133 df. This gives an estimated within-
observer standard deviation of
= 2.38 percent opacity.
47
-------
The significance of the observer term is determined by comparing the observer mean square to the
error mean square. The F-ratio is Fc = 4.16 which exceeds the critical value, F0.0s (7, 133)^2.09. Thus
there are differences among the observers, and an observer bias variance can be estimated .
The EMS of the observer term is a2 + 20 a£, so that the observer bias variance estimate is
-2 =MSL-&2
°L 20
23.58-5.67
20
_ 17.91
20
= 0.90
with 7 df. The estimated observer bias standard deviation is
= 0.95 percent opacity.
2. Black Smoke
The ANOVA table for the black smoke determinations is in Table B2. The model is identical to
that for the white smoke.
TABLE B2. ANALYSIS OF VARIANCE FROM TRAINING
GENERATOR TEST, BLACK SMOKE
Source
Runs
Observers
Error
Total
df
15
7
105
127
SS
3963.43
135.58
354.81
4453.82
MS
_*
19.37
3.38
F
_*
5.73t
EMS
_*
a7 + 16 a)
a* L
*Not of interest.
tSignificant. F005(7,105) » 2.12.
The expected value of the error
mean square is a2, the within-observer variance.
Thus, the estimated value is
&2=3.38,
the error mean square. There are 105 df associ-
ated with this estimate. The within-observer
standard deviation is
= 1.84 percent opacity.
The ratio of the observer mean square to the error mean square tests for differences among the
observers. The calculated F-statistic is Fc - 5.73, which exceeds the critical value of 2.13, approximately,
with 7 and 105 df. Thus, there can be said to be differences among the observers.
The observer bias variance is estimated as
_MSL -a2
L 16
48
-------
as indicated by the EMS for observers. This gives
19.37-3.38
51 =
16
15.99
16
= 1.00
with 7 df. Thus the observer bias standard deviation is
aL = 1.00 percent opacity.
B.3 Precision Estimation for Sulf uric Acid Plant and Tests of Hypotheses
The precision estimates for use of the method at a sulfuric acid plant are obtained from an analysis of
variance. The model for the data is a random effects model,
10 = M + It + Pj + «(/
where
Y-- - the observation by observer / in run /
p — the overall mean
~fj - the effect due to collaborator i
p- - the effect due to run ;'
and
e« - the random error associated with YIJ.
The model assumes that there is no interaction between the run and the collaborator. This is equivalent
to saying that an observer's tendency to read higher or lower than other observers is independent of the true
opacity level. The error term, then, is taken from the interaction between the observers and the runs.
The ANOVA is summarized in Table B3. There are 29 df from the 30 runs and 8 df for the observer
term This leaves a remainder of 232 for error. The expected mean square (EMS) column gives the basis for
forming F-ratios to test for significance and for estimating the individual variance components.
The EMS of the error term is o2 , the within-observer variance term. Thus, the estimated within-observer
variance is
a2 =4.51,
the error mean square. This gives an estimated standard deviation of
= 2.12 percent opacity
49
-------
TABLE B3. ANALYSIS OF VARIANCE FROM
SULFURIC ACID PLANT TEST
Source
Runs
Observers
Error
Total
df
29
8
232
269
SS
2636.68
259.06
1046.56
3942.30
MS
90.92
32.38
4.51
F
__*
7.18J
EMS
_*
oa + 10o}
a2
*Not of interest.
f Significant. FO.OS (8,232) » 2.00.
Solving the EMS for a^ gives
EMSobs
The EMS of the observer term is a2 + 30a£.
The factor 30 results from the fact that each
observer mean is the average over the 30 runs.
The ratio of the observer mean square to the error
mean square provides the proper ratio for deter-
mining if there is a significant observer term
The value of Fc is 7.18, which exceeds the tabled
value for the F distribution at the 95-percent
confidence level. The tabled value is.Fo.05 (8,232)
=* 2.00 and, thus, there is a significant observer
effect.
• -a"
30
so that an estimate is given as
_ MSobserver - a2
L= 30 ~
_ 32.38-4.51
30
_27.87
30
= 0.93
with 8 df. The estimated observer bias standard deviation, then, is
= 0.96 percent opacity.
Combining these two estimates gives
= 0.93+4.51
= 5.44
and
= 2.33 percent opacity.
The test for significance of a\ is a test for the equality among the observers of the mean opacity
determined. A significant value of 0£ is equivalent to saying that not all the means are equal. The next
problem is to investigate where the differences lie.
50
-------
For this test, the only comparison of interest concerns the difference between the observers from the
enforcement agency and those from the private sector. The enforcement personnel were observers 4, 5, 6 and
The means for the 9 collaborators who participated in all 30 runs are shown in Table 16.
To determine if there is a significant difference between the two groups, a contrast among the means is
tested. A contrast is of the form
where
Hi - the jth true mean
Cj - the coefficient of the ith mean
and
The null hypothesis to be tested is that there is no difference between the two groups of observers. The
contrast that tests this hypothesis is written as
Mi) -
where 5 and 4 are chosen so that the sum of the coefficients is 0.
The test statistic follows Student's t distribution and is defined by
where
& - the standard deviation for error
jc / - the /th sample mean
and
n - the number of determinations used to obtain* £-
The degrees of freedom for this statistic are the df associated with the estimated variance.
For this test,
jX, = 5(6.47 + .. . + 5.40) - 4(5.00 +... + 5.73)
= 29.54.
o=2.12, from Table B3
51
-------
2c? = 4(52) + 5(42) = 180
n = 30
so that
29.54
A t value at the 0.05 level with 232 df is essentially a standard normal value or 1.96. Since tc exceeds
1.96, we reject the null hypothesis, and conclude that the observers from the enforcement agency read
a higher opacity than did the observers from the private sector.
B.4 Analysis of Variance and Tests of Hypotheses from Riverbend Steam Station Test 1
The determinations from the first test are used to obtain precision estimates for within- and between-
observer and observer bias components. The determinations used are taken from runs 1, 2, 3, 7, 8 and 9.
Runs 4, 5, and 6 are eliminated since the experimental group was reading in 1-percent increments. Run 10
was eliminated since there were no determinations from 6 of the 10 observers, including the entire control
group. This was due to the fact that sky conditions and interference from another plume at times prevented
the observer from making a reading.
The model for the AMOVA is represented by
a completely random model, where
Yjjk - the kth observation by collaborator /in group i
p. — the overall mean
(3,- — the effect due to group /
tj/i — the effect due to observer/, nested within group/'
pk - the effect due to run k
and
ei/k ~ the random error associated with Yjj/f
The observers are treated as a nested factor since the observers differ from one group to another.
52
-------
TABLE B4. ANALYSIS OF VARIANCE FROM
STEAM STATION TEST 1
Source
Groups
Runs
Observers/Group
Error
Total
df
1
5
8
45
59
SS
124.42
662.03
119.05
149.10
1054.60
MS
124.42
-t
14.88
3.31
f
8.36*
-t
4.50J
EMS
-t
-t
a2 + 6o*L
*Significant. FQ.QS d.8) = 5.32.
•(•Not of interest.
^Significant. F0.Q5 (8,45) « 4.08.
The ANOVA table is shown in Table B4.
The expected mean squares (EMS) determine
the proper F-ratio for each factor. The group
mean square is divided by the observer mean
square to test for a difference between groups.
The F-ratio is Fc = 8.36 with 1 and 8 degrees
of freedom. The critical value from the
F-distribution isFo.os (1,8) = 5.32, so that
the group term is significant. The means
for the two groups are 8.20 and 5.32
for the control and experimental,
respectively, so that the control group
was reading significantly higher opacities
during the test. Since the determinations
that the control group was able to judge the opacity
were generally biased low, however, this indicates
more accurately than the experimental.
The EMS of the error term is a1 , the within-observer component. This gives an estimated within-
observer variance of
a2 = 3.31
with 45 df. The estimated within-laboratory standard deviation, then is
0 =
= 1.82 percent opacity
To determine if there is a significant observer effect, the observer mean square is compared to
the error mean square. The ratio is Fc = 4.50. The critical value for this test is F0-05 (8,45) « 4.08,
and the observer bias term, Q\ , Js determined to be greater than zero.
The observer bias is estimated from the observer group mean square. The EMS is a2 + oj,, so the
estimated variance is
MSobserver ~ *
_ 14.88-3.31
6
= 1.93
with 8 df. The df are calculated as (5-1) df for each group.
Since there are differences among the observers, the experience levels of the observers are important
considerations. The test is based on the means of the collaborators, and a contrast among the means is
tested. The hypothesis is
HO •
= 0
0
where c/ is the coefficient of the ith mean, chosen so that
53
-------
Since there are 4 experienced observers and 6 inexperienced observers, the c/ are chosen as
HO- 6(Mi + ^2 +M6 +M7)-4(M3 + M4
is the hypothesis tested. The test statistic is based on Student's t distribution, and given by
o
«
where n is the number of determinations comprising each mean, in this case 6. Substituting gives
,)- 4 (* 3 + ...+*io)
/62 + . . . + 6* + 42 + . . . + 42
L82y_ - . -
6(28.65) - 4 (38.97)
144 + 96
1.82
6
171.90-155.88
1.82 v/40
16.02
= =1.39
11.50
This statistic has 45 df associated with it. These df came from the estimated variance, a, used in the calculation.
A f-value with 45 df must exceed 2.016, approximately, to be considered significant at the 0.05 level. Thus,
no difference is determined between the experienced and the inexperienced observer for these data.
B.5 Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 2
The deviations shown in Table 23 are submitted to an ANOVA procedure to estimate the precision of
the method. The model selected is a completely random model represented by
*ijk = M + ft + Ijli + Pk + etfk
where
Yijk - the observed value in run k by observer / in group i
p. — the overall mean
(3,- - the effect due to group i
"fj/j - the effect due to collaborator / in group i
pk - the effect due to run k
and
eijk - the random error associated with y,yjt
54
-------
The observer term is nested within groups, since a different set of observers comprised each group. The
run term is crossed with both the group and the observer terms. The error term is estimated from the inter-
action of the run and group, and of the run and observer, with the interactions assumed to be zero. The
resulting ANOVA table is in Table B5.
The EMS column tells which mean squares provide the proper denominator for the F-ratio, and how
the resulting variance components may be estimated.
The within-observer variance is estimated from the error term. Since the EMS is a2, the estimated variance
is
o2 = 3.37,
the mean square for error. There are 88 df for this estimate. The standard deviation is estimated as
= 1.84 percent opacity.
TABLE B5. ANALYSIS OF VARIANCE FROM STEAM
STATION TEST 2
Source
Groups
Runs
Observers/Groups
Error
Total
df
1
10
8
88
107
SS
156.77
1866.06
225.71
296.21
2544.75
MS
156.77
186.61
28.21
3.37
F
5.56*
-t
8.37*
EMS
-t
-t
a2 + lla£
*Significant. FQ.QS (9,8) = 5.32.
t Not of interest.
^Significant. F0.05 (8,88) ~ 2.10.
The observer bias term is estimated from
the observer term in the ANOVA. The
denominator for the observer F-ratio is
the error term, and the test statistic is
Fc = 8.37 with 8 and 88 df. The tabled
F value at the 5-percent level is approxi-
mately 2.10 and the observer term is
determined to be significant. The expected
value of the mean square is a2 4 11 a\.
Thus, we estimate the variance by
-a2
11
where MS/, is the observer/group mean square. Substituting gives
. 28.21-3.37
_ 24.84
11
= 2.26
with 8 df. The degrees of freedom are calculated asn-l=5-l=4 from each group. The estimated
observer bias standard deviation is
= 1 .50 percent opacity.
The F-ratio for groups is the ratio of the group mean square to the observer/group mean square. The
statistic is Fc = 5.56. To determine if this value is significant, Fc is compared to a tabled F value at the 5-per-
cent significance level. The tabled value is Frj.osO .8) = 5.32, so that a significant difference can be said
to exist between the two groups in the determination of average opacity. This is an indication that the angle
of view is an important consideration in the visual estimation of opacity.
55
-------
Since it has been determined that there are differences among the observers, the hypothesis that the
experienced observers differ from the inexperienced can be tested. This is done by use of a contrast among
the means. A contrast is represented by 2 c,-/i,-, where the Cj are coefficients applied to the means, constructed
so that 2c, = 0. For this test, the null hypothesis is stated as
H0. 6(^1 +JU
where
M,- - the mean for observer i.
Observers 1,2,6 and 7 are experienced, and 3,4, 5, 8, 9 and 10 are relatively inexperienced. The test
statistic is given by
n
where
x j - the sample mean for observer /
a — the estimate of the within-observer standard deviation
and
n — the number of determinations that comprise Xj.
For these data, the statistic is:
_ (6) (29.60) +(-4) (3 7.20)
C
_ 177.60-148.80
1.84-V/24
_ 28.80
9.02
= 3.19
There are 88 df for the statistic taken from the estimated variance. By comparing this statistic to a tabled t
with 88 df, tc = 3.19 exceeds /o.OsC88) *= 1 •" and thus a significant difference does exist. The conclusion
is that the experienced observers did determine a higher average opacity than the inexperienced.
B.6 Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 3
The determinations of average opacity made by the collaborators as shown in Table 27 are submitted
to an ANOVA procedure to estimate the precision components. The model for these data is a completely
random model,
56
-------
tjk =H+Pt+ tjli + Pk + Pfiik + etjk
where
Yjjk - the observation on run k by observer / in group i
p. - the overall mean
fa - the effect of the ith group
yj/j - the effect of the /th collaborator in group
pk - the effect due to run k
p0ik - the interaction between run k and group i
and
£ijk -
the random error associated with
The two groups are considered together with respect to precision under the assumption that a more
direct sun angle would not affect the precision of the individual observer. To test this further, separate esti-
mates of the error variance are made using the run by observer interaction within each group. The estimated
values of a2 are
6* = 3.49
4 = 4.09
for the control and experimental groups, respectively. To test whether the experimental estimate is signifi-
cantly larger, an F-ratio is used. The calcualted statistic is
4.09
F = ,. ,-. -
° 3.49
with 45 and 45 df. The critical value is F0.0s (45,45) « 1.69, and the two values may be said to be estimat-
ing the same true value.
The ANOVA is summarized in Table B6 for the third test. The evaluation is made using the 16 runs
where the experimental group read in 5-percent increments. The EMS column provides the basis for testing
for significant effects and estimating the variance components.
TABLE B6. ANALYSIS OF VARIANCE FROM STEAM STATION TEST 3
Source
Groups
Runs
Group X Run
Observer/Group
Error
Total
df
1
15
15
6
90
127
SS
2.88
9440.54
138.39
321.05
313.73
10216.59
MS
2.88
629.37
9.23
53.51
3.57
698.56
F
0.31*
_•}-
-t
14.99 J
EMS
oj + 16a£ + 32ajjG + 64ac
-t
aj + 16o£ + 32
-------
The error term comes from the observer-run interaction. The assumption made is that this effect
is equal to zero, which is equivalent to saying that if a particular observer reads higher opacities than the
others on one run, he would not read lower on other runs. The EMS of the error term is a2 , which gives
an estimated within-observer variance of
a2 =3.57
with 90 df. This results in a within-observer standard deviation estimate of
& = \/3~37
= 1.89.
The EMS for observers/group is a2 + 16a£ , since each observer mean is the average over the 16 runs.
The F-ratio is the ratio of the observer mean square to the error mean square. This gives Fc = 14.99 with
6 and 90 degrees of freedom. The critical value is Fo.os(6,90) « 2.30. Thus, there is a significant observer
effect at the 5-percent level.
To estimate the observer variance component, the EMS is again used.
16
where MS^ is the observer mean square, and a2 is the error mean square. Substituting gives
, 53.51-3.57
a? = -
L 16
_ 49.94
16
= 3.12.
There are 6 df for laboratories calculated as (4-1) for each gioup. The estimated observer bias standard
deviation is
= 1 .77 percent opacity
To test whether there is a significant group effect, an F-ratio is constructed for the group term. The
denominator is the group by run interaction, and the statistic is Fc = 0.31 with 1 and 15 df. The critical
value is F0-OS (1 , 15) = 4.54, and no group effect is found. This indicates that the angle of observation had
essentially no effect in this test.
The means of the observers over the 16 runs are used to determine if there was a significant difference
in this test between the experineced and the inexperienced observers. A contrast among the means is used
for the test. The hypothesis is of the form
HO '• 2
-------
and
to is the mean of collaborator i.
For this test, the q are chosen to be 1 for the experienced and -1 for the inexperienced observer means.
The test statistic is given by
where
xf is the sample mean for collaborator i
a is the estimated within-observer standard deviation
and
n is the number of determinations used to obtain AT,-.
Substituting from Table 31 gives
(15.31 + ... + 14.46) -(11.40 + ... + 12.76)
te = —
1.89 /—
(58.72)-(46.81)
(1.89) (0.71)
11.91
~ 1.34
= 8.89.
There are 90 df associated with this statistic, and this gives a critical value of f0.os (90) « 2.00. Thus,
there was a significant difference between the experienced and inexperienced observers.
B.7 Precision Estimation for Riverbend Steam Station Test 3,1-Percent Increment Runs
The determinations made in increments of 1 percent are submitted to an ANOVA procedure according
to the completely random model
where
YIJ - the observation in run j by observer /
H — the overall mean
Pi - the effect due to run i
f — the effect due to observer/
59
-------
and
fc'/
the random error associated with Y,
i]
The model assumes that there is no interaction between run and observer. This assumption is equivalent
to saying thai the observers will be consistently high or low with respect to each other, regardless of the density
of the plume. By making this assumption, blocking the runs in order to make an estimate of the within-ohserver
variance is not necessary.
TABLE B7. ANALYSIS OF VARIANCE FROM
STI: AM STATION TEST 3, \','i VARIATION
Source
Runs
Observers
Error
Total
df
7
3
21
31
SS
2105.62
45.75
174.07
2325.44
MS
__ *
15.25
8.29
F
_. *
1.84t
EMS
_ *
*Not of interest.
tNot signit
cant. F0()5 (3.21)= 3.07.
There were 8 runs by 4 the observers in the control
group for a total of 32 determinations. The ANOVA
is summarized for these data in Table B7. The EMS of
the error term is a2 , the within-observer variance. Thus,
the estimated value of o2 is
&2 =8.29
with 21 df. This gives a within-observer standard
deviation of
The EMS of the observer term is a2-I- Sa2. . Thus, the ratio of the observer term to the error term tests
for significance. The resultant f-ratio is Fc - 1.84. The critical value for this test is F0 05 (3.21) = 3.07.
and thus, the observer term is not significant. This implies that the between-observer term, a2h. is identical
to o2. and gives
ah = a = 2.88 percent opacity
as before.
The within-observer term for these runs is higher than that for the previous runs. An F-test is con-
ducted to determine if a significant difference exists between the true variances of the two sets. The F-ratio
is
8.29
3.57
= 2.32
with 21 and 90 df. The critical value for this test is F0 05 (21. 90) ~ 1.75. Thus, the hypothesis of equality
is rejected, and we can say that there is more variability reading in 1 percent increments.
To determine if there was any difference between the performance of the two groups, a second ANOVA
is done in which the control group data is included. The model is
ijk
Pk
eljh
where
the determination by observer/' of group / in run k
the overall mean
60
-------
Pk
the effect due to group i
the effect due to observer/' in group i
the effect due to run k
the run by group interaction
and
e,y£ - the random error associated with y^
As before, the run by observer interaction is assumed to be zero and used to estimate the error term.
TABLE B8. ANALYSIS OF VARIANCE FROM STEAM STATION
TEST 3,1% RUNS
Source
Groups
Runs
Group X Run
Observer/Group
Error
Total
df
1
7
7
6
42
63
SS
268.96
5549.59
408.16
127.69
476.61
6831.01
MS
268.96
792.80
58.31
21.28
11.35
F
4.61*
-t
-t
-t
EMS
o2 + 8<7/ + Ifi^fcfj + 32<7 G
-t
a2 + Sal + 16er)jG
a' +8o£
o1
*Not significant. Frj.05 d>7) = 5-59-
tNot of interest.
The ANOVA is summarized in Table B8. The only term of interest is the group term. The F-ratio is
formed by dividing the group mean square by the group by run mean square. The calculated value is Fc = 4.61
with 1 and 7 df. The critical value is Frj.osO ,7) =5.59, and so no group effect can be determined.
B.8. Significance of Reduction of Frequency of Large Deviations
To determine if a significant decrease is noted in the frequency of large deviations, a confidence interval
is set up around the observed proportion. The confidence interval is constructed using
-P)
where
P
and
the sample proportion
the number of items in the sample
1.96 - the standard normal deviate at the 95-percent confidence level.
The proportion of large deviations (absolute value greater than 7.5) in the sample is obtained by taking
the number of large deviations, and dividing by the size of the sample. This gives
__ J5_
P 144
= 0.10.
61
-------
Substituting this into the above formula gives
/(o.ioxi - o.io)
CI095: 0.10 ± 1.96V- —
144
: 0.10 ±1.96(0.03)
or from 0.05 to 0.15
The frequency of large deviations in the original data set is 15/96 or 0.16. Thus, since this falls outside
the above confidence interval, the two may be said to represent different true proportions of occurrence. Thus,
the frequency of large deviations was reduced by averaging two observers' result.
B.9. Multiple Comparison Tests and Composite Estimation
The slopes of the three regression lines obtained as representative of observer deviation from plume
opacity are compared for equality using the least significant difference (LSD) test. The test statistic provides
a number which must be exceeded in order for two sample values to be said to represent different true values.
The standard deviations of the slopes is calculated as in Appendix B.I for each regression line. For the
white training generator smoke, black training generator smoke and steam station data, these values are 0.04,
0.05, and 0.04, respectively. Since these are close to each other, it is reasonable to assume that there is a
common variance, s^, for all three slopes, and to estimate it by pooling the three estimates. A pooled variance
is of the form
1= 1
where
dfi - the number of degrees of freedom in the i'th sample
s] - the variance of the jth sample
k — the number of samples.
Pooling the variance gives an estimated value of s* = 0.0019 with 393 df. The df are obtained by sum-
ming the df for the individual variance estimates.
The LSD at the 5-percent significance level is given by
= 1.96V0.0038
= 0.12.
All three slopes can be said to be equivalent if they differ from each other by no more than 0.12. The dif-
ference between the greatest and least values is 0.34 - 0.24 = 0.10, so that all three slopes can be said to be
estimating the same true value.
62
-------
LIST OF REFERENCES
1 Environmental Protection Agency, "Standards of Performance for New Stationary Sources," Federal Register.
Vol. 36, No. 247, December 23, 1971, pp. 24876-24893.
2 Environmental Protection Agency, "Stationary Sources, Proposed Emission Monitoring and Perform
Testing Requirements," Federal Register, Vol. 39, No. 177, September 11, 1974, pp. 32852-32874.
3. Dixon, W.J. and Massey, F.J., Jr., Introduction to Statistical Analysis, 3rd Edition, McGraw-Hill, New York,
1969.
63
-------
-------
REPORT NO.
EPA-650/4-75-009
J.TTTL E~ANo SUBTITLE
TECHNICAL REPOBT DATA
(Please read Instructions on the reverse before completing)
2.
3 RECIPIENT
-Evaluation and Collaborative Study of Method for Visual Determination
of Opacity of Emissions from Stationary Sources."
5. REPORT DATE
January 1975 (date of approval)
6. PERFORMING ORGANIZATION CODE
~7. AUTHOR(S)
Henry F. Hamil,Richard E.Thomas and Nollie F.Swynnerton
8. PERFORMING ORGANIZATION REPORT
SwRl #01-3462-006
9TER~FORM!NG"ORT-:ANIZATION NAME AND ADDRESS
Southwest Research Institute
8500 Culebra Road
San Antonio. Texas 78284
12. SPONSORING AGENCY NAME AND ADDRESS
U.S. Environmental Protection Agency
Office of Research and Development
Washington. D.C. 20460
10. PROGRAM ELEMENT NO.
1HA327
11. CONTRACT/GRANT NO""
68-02-0626
13. TYPE OF REPORT AND PERIOD C.OVE.K
Task Order
14. SPONSORING AGENCY CODE
15. SUPPLEMENTARY NOTES
16. ABSTRACT"
This study presents the results of statistical analyses of determinations of the average opacity of emissions. The
determinations are based on 25 readings made by qualified observers at three types of sources. The tests were con-
ducted 1) using training smoke generators, 2) at a sulfuric acid plant and 3) at a coal-fired power plant. Observers
used in the tests came from both enforcement agencies and private companies, and varied in the amount of field
experience in the use of the method.
For each test, the accuracy and precision of the determinations is investigated. Accuracy is measured by the
deviation of the determination from a concurrent meter average, used as true opacity for these tests. The precision
of the method is measured as standard deviations for wi thin-observer, observer bias and between-observer terms.
For each test, the expected range of determinations by a single observer and the maximum difference expected be-
tween two observers are calculated.
The results of all the tests are used to estimate the accuracy and precision of the method independent of the
nature of the site or color of the plume. These estimates show a reasonable degree of accuracy and precision for
the method when used in the probable range of opacities that would occur in field use, 0 to 35 percent opacity.
17 KEY WORDS AND DOCUMENT ANALYSIS
a DESCRIPTORS
Emission
Field tests
Statistical analysis
Plume
Opacity
13. DISTRIBUTION STATEMENT
Unlimited
b.lDENTIFIERS/OPEN ENDED TERMS
EPA Method 9
Stationary Sources
Visual method
Smoke generator
Collaborative testing
19. SECURITY CLASS (This Report)
Unclassified
20. SECURITY CLASS (This page/
Unclassified
_;
c. COSATI Field/Group |
I
13B j
14B
70
22. PRICE
EPA Form 2220-1 (9-73)
-------
-------
ERRATA
EPA-650/4-75-009
EVALUATION AND COLLABORATIVE STUDY OF METHOD FOR
VISUAL DETERMINATION OF OPACITY OF EMISSIONS
by
Henry F. Hamil
Richard E. Thomas
Nollie F. Swynnerton
The following correction is applicable to the above cited report which was issued under Project No.
01-3462-006, dated January, 1975.
Page 32
Paragraph 2 is deleted in its entirety and the following paragraph is substituted:
This value is comparable to that for the 5-percent increment data. From a second ANOVA shown
in Append* B.7, the two groups of observers cannot be shown to be reading different average opacmes on
these runs.
-------
------- |