EPA-650/4-75-009
EVALUATION  AND COLLABORATIVE
         STUDY  OF METHOD
   FOR  VISUAL  DETERMINATION
    OF OPACITY  OF  EMISSIONS
   FROM STATIONARY  SOURCES
                    by
          Henry F. Hamil, Richard E. Thomas,
             arid Nollie F. Swynnerton
            Southwest Research Institute,
                8500 Culebra Road
             San Antonio, Texas 78284
              Contract No. 68-02-0626
                ROAP No. 26AAG
            Program Element No. 1HA327
         EPA Project Officer:  M. Rodney Midgett

   Quality Assurance and Environmental Monitoring Laboratory
         National Environmental Research Center
       Research Triangle Park, North Carolina 27711
                 Prepared for

        U.S. ENVIRONMENTAL PROTECTION AGENCY
         OFFICE OF RESEARCH AND DEVELOPMENT
             WASHINGTON, D.C.  20460

                 January 1975

-------
                        EPA REVIEW NOTICE

This report has been reviewed by the National Environmental Research
Center - Research Triangle Park, Office of Research and Development,
EPA, and approved for publication.  Approval does not signify that the
contents necessarily reflect the views and policies of the Environmental
Protection Agency, nor does mention of trade names or commercial
products constitute endorsement or recommendation for use.
                   RESEARCH REPORTING SERIES


Research  reports of the Office of Research and Development, U.S. Environ-
mental Protection Agency, have been grouped into series.  These broad
categories were established*to facilitate further development and applica-
tion of environmental  technology.  Elimination of traditional grouping was
consciously planned to foster technology transfer and maximum interface
in related fields. These series are:

          1. ENVIRONMENTAL HEALTH EFFECTS RESEARCH

          2. ENVIRONMENTAL PROTECTION TECHNOLOGY

          3. ECOLOGICAL  RESEARCH
          4. ENVIRONMENTAL MONITORING

          5. SOCIOECONOMIC ENVIRONMENTAL STUDIES

          6. SCIENTIFIC AND TECHNICAL ASSESSMENT REPORTS

          9. MISCELLANEOUS

This report has been  assigned to the ENVIRONMENTAL MONITORING
series. This series describes research conducted to develop new or
improved methods and instrumentation for the identification and quanti-
fication of environmental pollutants at the lowest conceivably significant
concentrations. It also includes studies to determine the ambient con-
centrations of pollutants in the environment and/or the variance of
pollutants as a function of time or meteorological factors.
This document is available to the public for sale through the National
Technical Information Service, Springfield, Virginia 22161.
                                11

-------
                               SUMMARY AND CONCLUSIONS

     This report presents the results of statistical analyses on data obtained in the collaborative testing of
EPA Method c> (Visual Determination of Opacity). There were 5 separate tests of Method 9 at three dif-
ferent sites:  a training smoke generator, a sulfuric acid plant and a fossil fuel-fired steam generator.

     For these tests, the opacity measure used is the average opacity, defined as the average of 25 read-
ings made at 15-second intervals. These determinations are analyzed for their accuracy and their precision.
For this report, accuracy is measured by the deviation of the observer's  determination from the true opacity
as measured by  the in-stack  transmissometer. The precision of the method is measured in terms of within-
observer, observer bias and between-observer terms.

     •     Training Generator-Thirty-six runs were made over a two-day period using two training smoke
generators with 9 qualified observers participating. There were 20 runs  using white smoke and 16 using
black smoke with plume opacity ranging from just above zero to 40 percent. The two cases are treated sep-
arately.

           •    White Smoke—There were a total of 170 determinations made using a white plume.
These values showed a low bias with respect  to the transmissometer, and the deviation from the meter is
estimated by

                               deviation = 3.46 -  0.33 (meter average).

The within-observer standard deviation is estimated as 2.38 percent opacity with 133 degrees of freedom.
The observer bias standard deviation is 0.95  percent opacity with 7 df. This gives a between-observer
standard deviation of 2.56 percent opacity,

           •    Black Swofce- There were 133 determinations of the average opacity of black smoke. The
results from these values were similar to those of the white smoke. The predicted deviation from the trans-
missometer for  this case is
                                deviation  = 3.74 — 0.34 (meter average).

The within-observer standard deviation is 1.84 percent opacity with 105 degrees of freedom. The  observer
bias standard deviation is 1.00 percent opacity with 7 df. This gives a between-observer standard deviation
of 2.09 percent opacity.

     •     Sulfuric Acid Plant-Thirty observation runs were made over two days at a sulfuric acid  plant.
There were 11 observers on the  first day for 14 runs and 9 on the second day for the remaining 16 runs. The
plume opacity was varied around the compliance limit for sulfuric acid plants, ranging from 2  to 15 percent
average opacity. The observers were chosen from both a local enforcement agency and from private con-
cerns to allow a comparison of the two  groups.

           The predicted deviation for these data is  -2.0 percent opacity for the entire range studied. The
within-observer standard deviation is 2.12 percent opacity with 232 degrees of freedom.  The observer bias
standard deviation is 0.96 percent opacity with 8 degrees of freedom. This gives a between-observer stan-
dard deviation of 2.33 percent opacity. It is shown for this test that the enforcement observers read a
higher, and more accurate, average opacity than  did the  private sector observers.

     •     Steam Station-Three separate tests were conducted at a power plant with plume  opacity rang-
ing up to 35 percent average opacity. On the first two days of testing, the weather conditions prevented
the accurate determination of plume opacity.  In the third test period, viewing conditions were ideal and
the accuracy of the method is evaluated.

                                                Hi

-------
           •     Test 1 -There were 10 runs using 10 observers at the first test.  The determinations were
well below the concurrent meter averages due to the sky and wind  conditions.  The deviation from the meter
average is represented by

                              deviation = —3.01  — 0.40 (meter average).

A total of 60 determinations were used to obtain the precision estimates. The  within-observer standard
deviation is estimated as 1.82 percent opacity with 45 df.  The observer bias standard deviation is estimated
as  1.39 percent opacity with 8 df, which gives a between-observer standard deviation estimate of 2.29 per-
cent opacity.

           •     Test 2—Eighteen runs were made at the  second test using 10  observers.  The determinations
made  were more accurate than the previous test, but still were well below the meter average. The deviation
from the true opacity is represented by

                              deviation = —1.02 — 0.19 (meter average).

A total of 118 determinations are  used to obtain the precision estimates.  The within-observer standard
deviation is 1.84 percent opacity with 88 degrees of freedom.  The observer bias standard deviaton is esti-
mated as 1.50 percent opacity with 8 degrees of freedom.  The estimated between-observer standard devia-
tion, then, is 2.37 percent opacity.

           •     Test 3—Twenty-four runs were made  using eight observers over a two-day period.  The
resulting 192 determinations have a negative bias with respect to the metered opacity that can be repre-
sented by

                                deviation = 2.27 - 0.24 (meter average).

The within-observer standard deviation is estimated as 1.89 percent opacity with 90 degrees of freedom.
The observer bias standard deviation is 1.77 percent opacity with 6 degrees of  freedom, which gives  a
between-observer standard deviation of 2.59 percent opacity.

                 Also included for this test are evaluations of two variations to the method: reading to the
nearest percent, and averaging two observers' results.

      *    Composite Estimation^Using the  results from the three test sites, it is possible to make com-
posite estimates of the accuracy and precision that can be  expected with field use of Method 9.  The ex-
pected deviation from the average metered opacity is given by

                                deviation = 3.13 - 0.31 (meter average)

for the range from 5  to 35 percent average opacity. This  equation is obtained from the training generator
and steam station test data.

           The composite precision estimates are obtained using the estimates from all tests. The estimated
within-observer standard deviation is 2.05 percent opacity with 693 df. The estimated observer bias  standard
deviation is 1.29 percent opacity with 44 df.  This gives a  composite between-observer standard deviation
estimate  of 2.42 percent opacity.

      For each case, estimates are  made of the range of  determinations that would be expected from a single
observer, and of the maximum difference that would be expected between two observers when determining
the average opacity of a plume.

-------
                                   TABLE OF CONTENTS
                                                                                  Page

LIST OF TABLES	     vii

I.    INTRODUCTION	       J

II.   COLLABORATIVE TESTING	       2

     A.   Purpose	       2
     B.   Smoke Generator Test	       2
     C.   Stauffer Chemical Company Test	       2
     D.   Riverbend Steam Station	 •       3
     E.   Collaborators and Test Personnel	       4

III.   DEFINITIONS AND STATISTICAL TERMINOLOGY	       7

     A.   Definitions	       7
     B.   Statistical Terminology	       7

IV.   SMOKE GENERATOR TEST	
                                                                                     9
     A.    White Smoke	       9
     B.    Black Smoke	      12

V.   STAUFFER CHEMICAL COMPANY TEST	      16

     A.    Accuracy	      1°
     B.    Precision	      16
     C.    Predicted Results	      18
     D.    Comparision of Enforcement and Private Observers	      19

VI.  RIVERBEND STEAM STATION TEST	      20

     A.    Test 1	      20
     B.    Test 2	      23
     C.    TestS	      26

VII.  COMPOSITE  ESTIMATION	      34

     A.    Accuracy	      34
     B.    Precision	      34
     C.    Predicted Results	      35

APPENDIX A-EPA METHOD 9 FOR VISUAL DETERMINATION OF OPACITY	      37

     A.1  Federal Register, Vol 36, No. 247-Thursday, December 23, 1971	      39
     A.2  Federal Register, Vol 39, No. 177-Wednesday, September 11, 1974	      40

APPENDIX B-STATISTICAL METHODS	      45

     B.1  Linear Regression and Test for Significance	      46
     B.2  Precision Estimation for Training Generator Test	      47
     B.3  Precision Estimation for Sulfuric Acid Plant and Tests of Hypotheses	      49
     B.4  Analysis of Variance and Tests of Hypotheses from Riverbend Steam Station Test 1 .      52

-------
                               TABLE OF CONTENTS (Cont'd)
                                                                                       Page

     B.5  Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 2  .       54
     B.6  Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 3  .       56
     B.7  Precision Estimation for Riverbend Steam Station Test 3, 1-Percent Increment
          Runs	       59
     B.8  Significance of Reduction of Frequency of Large Deviations	       61
     B.9  Multiple Comparison Tests and Composite Estimation	       62

LIST OF REFERENCES	       63
                                                VI

-------
                                      LIST OF TABLES




Table




   1     Certification Data on Collaborators, Riverbend Steam Station Tests	     4




   2     Test Log, Riverbend Steam Station, Test 1	     4




   3     Test Log, Riverbend Steam Station, Test 2	     4




   4     Test Log, Riverbend Steam Station, Test 3	     5




   5     Determinations of Average Opacity of Training Generator, White Smoke	    10




   6     Deviation From Training Generator, White Smoke	    10




   7     Predicted Deviations, Training Generator, White Smoke	    11




   8     Predicted Determinations, Training Generator-White Smoke	    12




   9     Determinations of Average Opacity of Training Generator, Black Smoke	    13




  10     Deviation From Training Generator, Black Smoke	    13




  II     Predicted Deviation From Metered  Opacity, Training Generator-Black	    14




  12     Predicted Determinations of Average Opacity, Training Generator-Black	    15




  13     Determinations of Average Opacity, Stauffer Chemical	    17




  14     Deviations From  Meter Average, Stauffer Chemical	    18




  15     Predicted Determinations, Sulfuric  Acid Plant	    19




  16     Observer Means, Sulfuric Acid Plant	    19




  17     Average Opacity Determinations, Steam Station Test 1	20




  18     Deviations From  Metered Opacity, Steam Station Test 1	    21




  19     Predicted Deviation From Metered Opacity, Steam Station Test 1	21




  20    Predicted Determinations, Steam Station Test 1	    23




  21     Observer Means For Steam Station Test 1	    23




  22     Average Opacity  Determinations, Steam Station Test 2	    24




  23     Deviations From Metered Opacity, Steam Station Test 2	    25




  24    Predicted Deviation From Metered Opacity, Steam Station Test 2	    25




  25     Predicted Determinations, Steam Station Test 2	    26




  26    Observer Means For Steam Station Test 2	    26





                                                vii

-------
                                 LIST OF TABLES (Cont'd)




Table                                                                                    Page




 27     Average Opacity Determinations, Steam Station Test 3	   27




 28     Deviations From Metered Opacity, Steam Station Test 3	27




 29     Predicted Deviation From Metered Opacity, Steam Station Test 3	28




 30     Predicted Determinations, Steam Station Test 3	29




 31     Observer Means For Steam Station Test 3	30




 32     Predicted Deviations, 1-Percent Increment Data	30




 33     1-Percent Data Summary	   31




 34     Predicted Determinations, 1-Percent Data	   32




 35     Opacity Determinations Using Paired Observers	   32




 36     Deviations From Opacity, Paired Observers	   33




 37     Predicted Deviation From Metered Opacity, Composite Estimate	34




 38     Predicted Determinations of Average Opacity, Composite Estimate	35







 Bl     Analysis of Variance From Training Generator Test, White Smoke	47




 B2     Analysis of Variance From Training Generator Test, Black Smoke	48




 B3     Analysis of Variance From Sulfuric Acid Plant Test	   50




 B4     Analysis of Variance From Steam Station  Test 1	   53




 B5     Analysis of Variance From Steam Station  Test 2	   55




 B6     Analysis of Variance From Steam Station  Test 3	   57




 B7     Analysis of Variance From Steam Station  Test 3, 1% Variation	60




 B8     Analysis of Variance From Steam Station  Test 3, 1% Runs	61
                                              Vlll

-------
                                         I. INTRODUCTION
     This report describes the work performed and results obtained on Southwest Research Institute Project
01-3462-006, EPA Contract No. 68-02-0626, which includes collaborative testing of Method 9 for visual determi-
nation of the opacity of emissions from stationary sources as given in "Standards of Performance for New Stationary
Sources"! 1)* and "Stationary Sources, Proposed Emission Monitoring and Performance Testing Requirements."(2)

     This report describes the statistical analysis of the data from collaborative tests conducted on two EPA-type
training smoke generators, and at a sulfuric acid plant and a coal-fired power plant.

     The results of the data analyses and the conclusions based on these analyses are given in this report.
 'Superscript numbers in parentheses refer to the List of References at the end of this report.

-------
                                   II.  COLLABORATIVE TESTING


A.    Purpose
                                                                                                        / i  -y\
      Collaboralive testing of the method for visual determination of opacity of emissions from stationary sources' • '
was conducted in order to obtain data using certified observers which would allow statistical evaluation of the method.
Results from these tests are presented in this report.

      Three collaborative test sites were used:  a training smoke generator, a sulfuric acid plant, and a fossil fuel-
fired steam generatoi. Tests at the first  two sites mentioned above were designed to evaluate the determination of
average opacity of emissions^), while the test at the third site was designed to allow evaluation of both the deter-
mination of minutes of noncompliance with a standard and determination of average opacityX-^)  The results in
this report are all based upon average opacity determinations.

      The initial test on the training smoke generator was conducted to provide background information on the use
of the method.  The tests at the sulfuric acid plant and the  fossil fuel-fired steam generator were conducted to obtain
information on the use of the method on applicable sources under field conditions. At no time during any
of the test were warm-up or practice runs  allowed prior to  the test itself.

B.    Smoke Generator Test

      The smoke generator test was conducted on 19-20 November 1973 at the National Environmental Research
Center, Research Triangle Park, North Carolina. Two training smoke generators were used, one belonging to the EPA
and one belonging to the State of North Carolina. A total of nine collaborators from enforcement agencies in the
State  of North Carolina were available for  the tests.  All nine collaborators participated in the first fifteen runs—only
eight participated in the remaining twenty-one runs, since one collaborator operated the State of North Carolina's
smoke generator. Time since last  certification for the collaborators varied from one to thirteen weeks.

      During the two-day test, a total of thirty-six runs were made, with a run consisting of twenty-five readings
taken at the fifteen-second intervals by each collaborator. The collaborators read the plume on a time signal from
the test supervisor in order to assure that the readings were as nearly simultaneous as possible. The opacity of the
plume as indicated by the in-slack transmissometer of the smoke generator was recorded at the time signal to pro-
vide "true" opacity data.  Twenty runs were made with white  plumes, and sixteen runs were made with black plumes.
The range of opacity studied was  from just above 0 to 40 percent. During some of the runs, plume opacity was held
essentially constant while on other runs the opacity of the plume was varied randomly to determine how
well the observers could follow changing opacity during the course of a set of observations.

      Due to the relatively short stack height on the smoke generators, in some runs the observers had objects in
the background (buildings, trees, power lines, etc.), while on other runs only a sky background was available.
Since  use of background objects in evaluating plume opacity is permitted in most training courses, no differentiation
was made in this test between runs with or without background objects. During both days of the test, viewing con-
ditions were ideal, with blue sky and  bright sunshine. On all runs, the observers were allowed to select the opti-
mum  viewing position with respect to sun  and wind  angles.

C.    Stauffer Chemical Company Test

      The sulfuric acid plant test was conducted on 29-30 January 1974 at Stauffer Chemical Company's facilities
in Houston. Texas.  The unit was equipped with a 300-foot-high stack. In-stack monitoring of opacity was accom-
plished by use of a Lear Siegler Model 610A transmissometer*.

      A total of eleven collaborators were available for the  first day of the test. Nine collaborators were available for
the second day of the test. Four of the collaborators were  from the City of Houston Department of Public Health,
•Mention o! truck names or speiitic products doos not constitute endorsement by the Lnvironmculul Protection Agency

-------
six were from area industries, and one was from a local health association.  All collaborators' most recent certifica-
tion was on 24 August 1973 at the State of Texas Training Session conducted at Houston, Texas.

     During the two-day test, a total of thirty runs were made, fourteen rum on the first day and sixteen runs on
the  second day.  Each run consisted of twenty-five readings taken at 15-second time intervals by each observer.
The collaborators began each run on a time signal from the test supervisor. The average opacity as indicated by
the  in-stack transmissometer was obtained from the instrument strip chart recorder for the time period correspond-
ing to each run.

     Arrangements were made with plant personnel to change unit operating conditions during the test to provide
plumes of varying opacity.  The range of opacity studied was from slightly above 0 to 15 percent.  Opacity of the
plume was essentially constant during any single run. During both days of the test, good viewing conditions pre-
vailed, with blue sky and a slight haze. Wind was light and variable throughout the test. Color of the plume was
white.

     The observers were allowed  to select the optimum viewing position with respect to wind direction and sun
angle.

D.   River bend Steam Station

     Three tests were conducted  at the Duke Power Company's Riverbend Steam Station near Charlotte. N.C.
Tests were conducted on 27 August, 5 September, and 30 September-1 October, 1974. The tests involved  use of
two units at the power plant, number 9 and number 10 boilers.  Each boiler was equipped with two stacks  (A&B).
Readings for the first two tests and the first day of the third test were made on Stack B of boiler number 9. On
1 October 1974, the number 9 boiler was shut down, and  the third test was completed using Stack A of number 10
boiler. Both stacks were fitted with Lear Siegler Model RM4 transmissometers*.

     The plan for the tests at the Riverbend Steam Station varied from the plans for the previously described tests
This set of tests was designed to provide information on some variations to the method as written.(' >-' The col-
laborators were divided into two equal groups with regard to size and experience. One group was used as a control
and made their readings of opacity within the constraints of the published method, with the sun in the quadrant
to their rear, assigning opacity to the plumes in 5-percent increments.  The other was used as the experimental
group. One variation under study by this group was reading at 45- and 90-deg angles with respect to the sun.
Opacity was assigned in 5-percent increments during this portion of the experiment.  The other variation was read-
ing  the plume in 1-percent increments from the same position as the control group


      From lhis test, the effect on a method test result induced by reading outside some of
the constraints of the method may be estimated. By altering the angle of view with  respect
to the sun, it may be determined whether this is an important consideration  for the observer.
By  requiring that the plume be read in I-percent increments, it  may be determined if this
variation is more accurate or more precise. In addition to these, the relative  performance of
an inexperienced to an experienced observer may be assessed, and the possible improvement
of the method by averaging two observers" results may be evaluated.


      In the first two tests, a run consisted of forty observations by each observer taken at 15-second intervals. This
was done for the purpose of evaluating the method when  used to determine minutes of noncornpliance with a
standard. The average opacity presented in the data tables represents the average of  the first twenty-five readings
by an observer in each run.  In the third test, a run consisted of twenty-five observations by each observer  taken at
 15-second intervals. In all three tests, the observers read the plume on a time signal from test personnel. Transmis-
someter readings were recorded to the nearest percent on the time signal to provide "true" opacity readings.

 'Trade rumc.

-------
  I ABM' 1. CERTIFICATION DATA ON COLLABORATORS,
           RIVKRBENDSTEAM STATION TESTS
Collaborator
Code Number
,
1
~! *
4
5
6
7
8
9t
II)
1 1
12
1 j
Duleof
Most Recent
Certification
7-16-74
7-16-74
4-5-74
4-5-74
4-5-74
4-5-74
4-5-74
4-5-74
3-19-74
8-27-74
9-11-74
9-11-74
9-11-74
Average Deviation
from True Opacity
During Certification, %
4.7
5.7
4.6
5.4
5.9
6.2
5.9
4.4
4.3
3.8
3.1
2.9
3.9
*Not qualified. One reading during qualification deviat-
ed from the opacity by more than 15 percent. Not
determined until after test due to clerical error.
•^Certification expired 9-19, recertified 10-22.
TABLE 2. TEST LOG, RIVERBEND STEAM STATION, TEST 1
                    27 August 1974
Run
No.
1
•>
3
4
5
6
7
S
9
10
Time Run
Began
1015
1055
1130
1330
1 350
1410
1430
1450
1510
1530
Position
Angle, deg
45
90
90
0
0
0
45
45
90
90
Sky Conditions
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, tow haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Mostly cloudy, low haze
Wind
NE, 0-5 mph
Light, variable
Light, variable
SW, 5 -10 mph
SW, 5- 10 mph
Light, variable
Light, variable
Light, variable
W, 3-5 mph
W, 3-5 mph
TABLE 3. TEST LOG, RIVERBEND STEAM STATION, TEST 2
                   5 September 1974
Run
No.
1
-)
3
4
5
6
7
«
9
10
H
i?.
1 3
14
15
16
1 •
1 S
Time Run
Began
0910
0930
0950
1010
11130
1050
1 130
1 150
121(1
1 230
1 ."50
1410
I4.ui
1450
1530
155' )
1 r, i n
I Mi:
Position
Angle, deg
0
0
45
45
90
90
45
45
90
90
0
0
9o
90
11
ft
45
45
Sky
Conditions
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcast
Overcas
Ovcrca>
Overea.x
Ovcrcas
Overeat
Ovcrcas
Wind
NE, 5-10 mph
NE, 5-10 mph
NE, 5-10 mph
NE, 5-10 mph
NE, 5 -10 mph
NE. 5-10 mph
NE, 5-10 mph
NE, 5- 10 mph
NE.5 -10 mph
NE, 5 -10 mph
NE, 5 lOniph
NE.5 10 mph
NE. 510 mph
NE. 5 10 mph
NE.5 lOtnph
NE.5 lOmpli
M . 5 10 mph
NE.5 10 mph
      The range of opacity studied in all
three tests was from just above 0 to 40 per-
cent. Opacity of the plume was varied  by
cutting off one or more stages of the elec-
trostatic precipitator prior to the run.
With the exception of occasional short-
duration transient high values, plume opac-
ity was essentially constant during a given
run for the low opacities. Variation in  the
"true" opacity within a run increased as
the  average plume opacity increased.

      The collaborators for each test came
from State of North Carolina enforcement
agencies, EPA offices, and private contrac-
tors. The first two tests utilized four state,
four EPA, and two contractor personnel.
The third test utilized four state, two EPA,
and two contractor personnel. The date of
the  most recent certification of the collab-
orators is given in Table 1. Also included
for each of the collaborators is the average
deviation from the true plume opacity for
the  twenty-five white and twenty-five black
plumes read during certification.

      The three tests consisted of ten,
eighteen, and twenty-four runs, respectively.
Viewing conditions during the first test were
poor, mostly cloudy and low haze. There
was also interference from another plume
during portions of the test.  Conditions
for the second test were marginal, with a
solid overcast, while for the third test con-
ditions were ideal, with a cloudless sky  and
bright sunshine. The plume color was white.

      Test logs for the three tests are pre-
sented in Tables 2,3, and 4. In Tables 2
and 3, the position of the experimental
group is given relative to  the control group.
In Table 4, the positions of both the con-
trol and experimental groups are given
relative to the sun, with the angles calcu-
lated as shown in Figure 1.

E.    Collaborators and Test Personnel

      The collaborators for the smoke gener-
ator test were Mr.  Sammy L. Amerson.
Mr. George Lawson, Mr. Dewey D. Johnson,
Mr. Roy T. Gorman, Mr. J.R. Kirk, Mr. D.Y.
Daniel, Jr., Mr. B.J. Foust and Mr. Bill
Proctor from the State o1  N'orth Carolina

-------
         TABLE 4.  TEST LOG, RIVERBEND STEAM STATION, TEST 3
                       30 September - 1 October I 974
Run
No.
I
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Time Run
Began
1315
1330
1345
1400
1415
1430
Viewing Angle, 8 (Fig. 1)
Control
Group.
deg
0
0
10
10
15
15
1445 45
1500 45
1515 0
1530 0
0930
0945
1000
1015
1130
1145
1200
1215
1230
1245
1 300
1315
1330
1345
30
30
25
25
20
20
10
10
0
0
0
0
0
0
Experimental
Group,
deg
0
0
45
45
60
60
45
45
0
0
30
30
75
75
90
90
45
45
0
0
45
45
90
90
Sky
Conditions
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Clear
Gear
Gear
Gear
Clear
Clear
Clear
Clear
Wind
Light, variable
Light, variable
Light, turiubic
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
N. 8-12 mph
N. 8 - ' 2 mph
N, 5 10 mph
N, 5 -10 mph
Light, variable
Light, variable
Light, variable
Light, variable
Light, variable
Ligh'. variable
Ligh'. variable
Light variable
Light . variable
Light, variable
\   /
       \
6 - Angle formed by lin* between sun and
  plume being observed and line between
  observing group and plume.
                                   CONTROL GROUP
                                                              * •   EXPERIMENTAL
                                                                   GROUP
                                 \  '
                            OO   O'O   OO   OO

                                    POWER PLANT
              HGURL 1.  SCHEMATIC OF OBSERVER POSITIONS.
                     RIVTRBEND STEAM STATION. TEST 3

-------
Depai imunt of Natural and Economic Resources; and Mr. William M. Edsel from the County of Forsyth Air Quality
Control Division, Forsyth County, North Carolina.

      The collaborators for the sulfuric acid plant test, all from the Houston, Texas, area, were Mr. Charles L. Owen
fiom Annco Steel; Mr. James S. Corbin, Mr. Robert J. Stahl, Mr. Ronald F. Stockunas and Mr. Wendell La Foe
trum the City of Houston Department of Public Health; Mr. B. L. Bolton from Rollins Environmental Services;
Mr.T. M. Walker and Mr. Klaus R. Gerlach  of Stauffer Chemical Co.; Mr. Mel Remley from Jefferson Chemical Co.;
Mr. William Mooie from the San Jacinto Lung Association; and Mr. Larry P. Stoltz from GAP  Corporation.

     The collaborators for the power plant tests were Mr. Dewey D. Johnson, Mr. Ron Jernigan, Mr. John Leather-
man and Mr. Don Shepherd from the Sfate of North Carolina Department of Natural and Economic Resources;
Mr. William M. Hdsel from the County of Forsyth Air Quality Control Division; Mr. David Da Crema and
Mr. John Yates. from Engineering Science  Inc.; Mr. Alan Luther and Mr. Peter F. Burnette from Environ-
mental Science and Engineering, Inc.; and  Mr. Alfred  Vervaert, Mr. J.W. Brown. Mr. Joseph Peoples, Mr. James
E. Casey, Mr. Jack Siegel and Mr. John Hund from the Environmental  Protection Agency.*

     The tests were conducted under the general supervision of Mr. Nollie F. Swynnerton.  The  collaborators from
the Stale ot North Carolina were provided  through the assistance of Mr. James A.  McColman, Chief, Air Quality
Division, State of North Carolina Department of Natural and Economic Resources. The collaborators for the sulfuric
acid plant test were selected by Mr. Swynnerton from a list of people who had certified at the  most recent State of
Texas certification school in the Houston area.  The list was provided by Mr. Thomas Jay McMickle, Environmental
Health Specialist. Texas State Department  of Health.  The collaborators from EPA, Engineering  Science, and
Environmental Science and Engineering were provided by Mr. Roy Neulicht, Environmental Engineer.
Emissions Standards and Engineering Division, Office of Air Quality Planning and Standards, EPA.  Mr. Dennis
Holzschuh, Pysical Science Technician. Engineering and Enforcement Section. Air Pollution Training Institute,
Control Programs Development Division, Office of Air Quality Standards and Planning. EPA, assisted in the
conduct of the smoke generator test at the National Environmental Research Center.  Mr. William D. Conner,
Research Physicist. Stationary Source Measurements Research Section. Emission Measurements  Research
Branch. Chemistry and Physics Laboratory, National Environmental Research Center. EPA. made the neces-
sary arrangements with Duke Power Company personnel for the  Riverbend Steam Station Tests.
         t llv Kiiijmdcr ot thi\ n.-pm I. I!K- . ilixerv
        :o i!u- .ilmvi- ordered IMIMS> ol ntiM.':ver-.
ode nunilvrs
             d dn not

-------
                 III.  DEFINITIONS AND STATISTICAL TERMINOLOGY


     To facilitate the understanding of this report and the utilization of its findings, this section explains the
statistical terms used in this report and defines certain terms used in presenting the data and results.

A.   Definitions

     1.    Run -A 6-minute period during which the observers determined the opacity of the plume.

     2.    Determination-The arithmetic mean of 25 consecutive opacity readings taken at 15-second
          intervals during a run.

     3.    Meter Average-The average opacity of the plume as determined by the in-stack transmissometer.

     4.    Deviation-The difference between a determination and the corresponding meter average; deviation =
          determination — meter average.

B.   Statistical Terminology
      1.   x = — ^fXi  -the sample mean. This statistic estimates the mean, or center, of a given population.
              "•-1


                1     "
      2.   s2 = - y^ (Xf - ;T)a   —the sample variance.  This statistic estimates the dispersion in a dis-
               » - 1 ,' = j
          tribution around the mean value.
     3.   x = v^7 -the sample standard deviation. The standard deviation is an alternative estimate of
          dispersion.

     4.   Analysis of Variance (ANOVA)— A statistical technique for testing whether different groups or
          factors perform in an equivalent manner, and for obtaining estimates of different variance terms.

     5.   SS— sums of squares. The sum of squared deviations from the mean for a component. The SS are
          used to estimate variability.

     6.   df— degrees of freedom.  The df are an indication of the amount of confidence in the estimate. A
          larger number of df implies a greater degree of confidence.

     7.   MS-the mean square. The MS is the basis for hypothesis testing in the ANOVA  and estimating a
          variance component. MS = SS/df

     8.   EMS-the expected mean square. The EMS determines which mean squares are used to test which
          hypotheses, and to estimate the particular variance components.

     9.   F-Ratio— the ratio of two mean squares. The F-ratio is the test statistic for determining the relative
          performance of two groups.

-------
     All precision components in this report are estimated using an ANOVA procedure. The statistical
model is given separately for each data set, but, in general, consists of an observer term, a run term and an
error term. The MS and EMS from these ANOVA tables are used to determine the significance of the vari-
ous factors, and to estimate the following precision components.

     *     Within-observer. The within-observer, or error, component measures the dispersion that would be
           expected in replicate  determinations by the same observer at the same true opacity.  The within-
           observer variance is denoted by a2.

     •     Observer bias.  The observer bias component measures the dispersion that would be expected in
           replicate determinations due to the use of the method by independent observers. These dif-
           ferences can result from such factors as the training generator on which the observers qualified
           and the relative frequency with which they use the method. The observer bias variance is denoted
              •i-
by a?
           Between-observer. The between-observer component is estimated from the within-observer and
           observer bias terms.  The between-observer term measures the variability associated with the dif-
           ference between results by independent observers. The between-observer term is composed of
           within-observer and observer bias terms, and the between-observer variance, a| is defined as

-------
                              IV.  SMOKE GENERATOR TEST
     The initial test of Method 9 was conducted using two EPA-approved smoke generators to produce plumes,
both white and black, covering the range of opacities that could be expected by observers seeking to determine
compliance with the new source performance standards. These included plumes from just above zero up to
40 percent. On each run, the collaborators recorded their observations to the nearest 5 percent for 25 readings
taken at 15-second intervals.  The observers read on a signal given by test personnel, so that  all determinations
were comparable.  The 25 readings were averaged by SwRI personnel after the test to insure that the deter-
mination was correctly computed.

     To determine to what extent the observers were able to follow changes in the plume opacity, the generated
opacity was subject to change during a run.  During the runs, the opacity was either held constant, increased or
decreased according to a test  plan. In addition, the number of times the plume opacity was changed, the point
at which it was changed and the amount it changed all were varied from run to run so that the observers would
not be able to anticipate these changes.

     The observers were allowed to choose their position for observing the generated plume, consistent with
the constraints of  the method. On some runs, they had a background (e.g., trees, buildings) to read against,
while on others, there was only blue sky.  No record of the background was kept, so any differences in this
regard cannot be determined.

     There were a total of 36 sampling runs, 20 with white smoke and 16 with black, made during the two
day period. One observer participated in only the  first 15 runs then operated the second training generator for
the remainder of the test. This resulted in a total of 303 determinations, 170 with white smoke and 133 with
black. The analyses of the data will concern itself  with two aspects: the accuracy, as measured by the deviation
from the metered  opacity, and the precision, expressed in terms of standard deviations. The results are pre-
sented below separately for the white and black plumes.

A.   White Smoke

     The 170 determinations are shown in Table 5, along with the concurrent meter averages.  To evaluate the
performance of the observers, the determinations will be looked at in two ways. First, they will be compared
to the meter averages to determine how accurately the observers were able to read the opacity; and second, they
will be compared to each other to determine how much variation can be expected both from a single observer,
and between two independent observers when reading the same opacity plume.

     1.    Accuracy

           The deviations from the plume opacity are shown in Table 6. As can be seen from the data, there
is a tendency for the deviation to be greater and negative as the plume opacity increases. Because of this, a
linear regression is used, with opacity as the independent variable and deviation as the dependent variable.
The technique is presented in Appendix Bl.

           For these deviations, the slope of the least-squares fit to these points is significantly different
from zero.  This indicates that there is a linear relationship between the two variables. The equation of the
line derived is

                                        Yj = 3.46 - 0.33*,

where

           Yj    is the predicted deviation
           Xj    is percent opacity

-------
TABLE 5. DETERMINATIONS OF AVERAGE OPACITY OF TRAINING
               GENERATOR, WHITE SMOKE

                    (Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
27
28
29
30
31
32
33
34
35
36
Observer
1
12.8
10.6
8.8
27.0
11.2
5.6
8.8
31.2
22.6
9.4
17.4
8.6
10.0
13.0
18.2
32.2
19.0
9.0
14.8
12.4
2
12.0
9.0
5.8
28.2
9.6
4.6
8.4
22.6
15.4
7.6
17.4
6.4
8.8
14:8
20.8
29.4
18.8
8.8
17.0
10.2
3
10.8
11.2
8.6
23.8
12.0
5.0
7.0
24.6
13.6
8.4
19.2
8.6
12.4
15.8
17.4
30.8
20.8
13.6
20.4
9.8
4
13.2
11.0
9.6
22.6
12.0
6.6
7.2
24 .4
16.6
7.8
19.6
7.4
10.6
19.4
18.4
28.6
19.6
9.6
16.4
10.0
5
10.6
8.2
6.0
13.6
11.4
5.8
7.6
20.2
19.6
10.0
18.8
8.0
11.4
15.4
19.6
27.6
16.6
7.2
17.0
9.4
6
17.4
10.6
8.6
22.2
12.8
5.8
8.2
21.0
17.4
7.8
17.4
6.2
11.4
12.0
12.8
18.2
15.0
7.2
13.0
7.4
7
11.0
8.4
6.6
14.6
11.8
6.0
7.2
21.4
20.6
9.0
16.4
5.2
8.7
13.6
13.8
23.6
18.8
7.8
15.8
9.2
8
10.6
12.8
8.6
25.0
11.4
.5.0
7.6
27.4
17.0
7.2
16.4
5.2
8.2
12.6
10.8
23.2
13.0
7.0
11.0
8.0
9
13.0
14.8
10.0
24.0
11.2
6.6
7.4
27.4
16.0
8.6










Meter
10.0
13.6
9.2
23.8
12.3
4.7
7.6
34.0
25.4
12.4
13.1
8.6
9.0
11.2
14.3
31.3
13.2
16.8
18.7
14.6
     TABLE 6. DEVIATION FROM TRAINING GENERATOR,
                    WHITE SMOKE

                    (Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
27
28
29
30
31
32
33
34
35
36
Observer
1
2.8
-3.0
-0.4
3.2
1.1
0.9
1.2
-2.8
-2.8
-3.0
4.3
0.0
1.0
1.8
3.9
0.9
5.8
-7.8
-3.9
-2.2
2
2.0
-4.6
-3.4
4.4
-2.7
-0.1
0.8
-11.4
-10.0
-4.8
4.3
-2.2
-0.2
3.6
6.5
-1.9
5.6
-8.0
-1.7
-4.4
3
0.8
-2.4
-0.6
0.0
-0.3
0.3
-0.6
-9.4
-11.8
-4.0
6.1
0.0
3.4
4.6
3.1
-0.5
7.6
-3.2
1.7
-4.8
4
3.2
-2.6
-0.4
-1.2
-0.3
1.9
-0.4
-9.6
-8.8
-4.6
6.5
-1.2
1.6
8.2
4.1
-2.7
6.4
-7.2
-2.3
-4.6
5
0.6
-5.4
-3.2
-10.2
-0.9
1.1
0.0
-13.8
-5.8
-2.4
5.7
-0.6
2.4
4.2
5.3
-3.7
3.4
-9.6
-1.7
-5.2
6
7.4
-3.0
-0.6
-1.6
0.5
1.1
0.6
-13.0
-8.0
-4.6
4.3
-2.4
2.4
0.8
-1.5
-13.1
1.8
-9.6
-5.7
-7.2
7
1.0
-5.2
-2.6
-9.2
-0.5
1.3
-0.4
-12.6
-4.8
-3.4
3.3
-3.4
-0.3
2.4
-0.5
-7.7
5.6
-9.0
-2.9
-5.4
8
0.6
-0.8
-0.6
1.2
^0.9
0.3
0.0
-6.6
-8.4
-5.2
3.3
-3.4
-0.8
1.4
-3.5
-8.1
-0.2
-9.8
-7.7
-6.6
9
3.0
1.2
0.8
0.2
-1.1
1.9
-0.2
-6.6
-9.4"
-3.8










Meter
10.0
13.6
9.2
23.8
12.3
4.7
7.6
34.0
25.4
12.4
13.1
8.6
9.0
11.2
14.3
31.3
13.2
16.8
18.7
14.6
                            10

-------
and
           3.34 and -0.33 are the least-squares estimates of the intercept and slope, respectively.

The f-statistic for these data is tc = ^8.59 with 168 degrees of freedom, which is significant at the 5-percent
level. This equation holds over the range of average opacities studied, from 5 to 30 percent.  By substituting
values into this equation, it is possible to predict what the deviation would be at a particular true average
opacity.  This is done for selected opacities in Table 7.
   TABLE?. PREDICTED DEVIATIONS,
        TRAINING GENERATOR,
             WHITE SMOKE
     2.    Precision

           The precision of the determination of the average
opacity of a white plume from a training generator is determined
through use of an Analysis of Variance (ANOVA) technique.
From this, it is possible to estimate both the within-observer and
the observer bias components, and using these, to estimate the
between-observer component.
                                                 The precision estimates are obtained using only the
                                      determinations from the eight observers who completed all 20
                                      runs, which enables an estimate of the within-observer variance
to be made without blocking the data. The details of the ANOVA are given and the estimate obtained in
Appendix B.2.

           The within-observer variance, o2 , is estimated as

                                            a2 = 5.67

with 1 33 df .  This gives an estimated within-observer standard deviation of
Percent
Opacity
5
10
15
20
25
30
Estimated
Deviation
1.81
0.16
-1.49
-3.14
-4.79
-6.44
                                       = 2.38 percent opacity.

           The F-ratio for observers isFc = 4.16. This exceeds the tabled value of 2.75, approximately,
from the F distribution with 7 and 133 df, at the 5-percent significance level. Thus, there is a significant
observer bias variance, a\ , and it is estimated by

                                             CT? =0.90
                                              lj

with 7 df. The estimated observer bias standard deviation, then, is

                                     &L= \AX90

                                        = 0.95 percent opacity.

           The between-observer variance, o\, is estimated from the components above as

                                          a| = 02-f&*

                                             = 5.67 + 0.90

                                             = 6.57.
                                                 11

-------
This gives a between-observer standard deviation of
                                       = 2.56 percent opacity.
           Predicted Results
           By combining the above results, it is possible to predict confidence limits for the average opacity
determined by an observer.  Let k be any true opacity level. Then the average opacity reported by the
observer could be expected to be within

                expected  range:  k + (a + bk) ± (1 .96) a percent opacity

                              :  k + (3.46 - 0.33*) ± (1 .96) (2.38) percent opacity

                              :  (0.67k + 3.46) ± 4.66 percent opacity

at the 95-percent confidence level. The ranges for selected values of k are given in Table 8.
   TABLES. PREDICTED DETER-
      MINATIONS, TRAINING
       GENERATOR-WHITE
             SMOKE
Percent Opacity
5
10
15
20
25
30
35
Expected Range
2.15-11.47
5.50-14.82
8.85-18.17
12.20-21.52
15.55-24.87
18.90-28.22
22.25-31.57
           Similarly, an estimate can be made of the difference
expected between two determinations of the same average plume
opacity made by independent observers. The difference between
determinations has a variance of 2 o|. Thus, the maximum expected
difference would be no more than

         maximum difference:    ± (1.96) \/2 a\,

                            :    ±(2.77)dft

                            :    ± (2.77) (2.56)
                                                                    ± 7.09 percent opacity
                                   at the 95 percent confidence level.
B.    Black Smoke
      The 133 determinations made using a black plume are shown in Table 9 along with the average of the
transmissometer readings taken. These determinations are examined for accuracy and precision in the same
manner as were the white plume determinations.

      1.   Accuracy

           The  deviations from the transmissometer average are shown in Table 10. As before, there is a
tendency for the deviation to be greater as the true value increases.  To obtain the best estimate of the devi-
ation, a linear regression of the deviations on the true opacity is done.

            For these data, the line that best fits these points is

                                        YI = 3.74 -0.34 Xi
                                                 12

-------
                  TABLE 9. DETERMINATIONS OF AVERAGE OPACITY OF TRAINING
                                 GENERATOR, BLACK SMOKE
                                       (Percent Opacity)
Run
No.
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Observer
1
24.4
1.4
20.0
17.6
7.4
9.6
13.6
13.2
12.6
12.0
18.6
29.2
14.8
7.8
15.0
22.0
2
20.4
9.6
20.2
14.4
11.4
5.2
11.0
14.0
10.2
10.8
18.2
17.0
11.4
8.0
12.2
20.2
3
22.0
10.0
22.6
15.6
12.6
5.8
12.0
13.4
16.4
13.8
17.2
25.2
15.8
11.4
15.2
23.2
4
23.2
10.4
24.0
19.6
14.0
4.6
11.0
14.2
16.2
14.0
21.0
29.4
17.4
11.4
20.4
24.2
5
19.0
10.6
22.2
17.2
12.8
8.8
10.4
10.0
11.4
11.6
17.2
29.4
15.0
10.0
15.8
23.0
6
21.4
7.6
20.8
19.6
12.0
5.6
12.0
13.8
14.8
12.4
23.0
31.2
17.4
11.6
16.6
24.4
7
21.6
11.2
23.2
20.0
13.0
4.8
11.2
11.6
11.2
11.8
17.8
30.0
13.4
8.4
15.8
22.0
8
21.8
8.4
21.8
20.2
13.0
4.8
12.4
11.8
15.0
13.6
19.2
28.6
13.0
8.6
15.6
19.8
9
25.4
12.0
23.0
17.0
12.4











Meter
35.0
10.8
26.0
20.0
12.3
16.0
11.0
17.6
17.6
13.6
18.0
28.2
13.8
9.0
14.8
22.2
                TABLE 10. DEVIATION FROM TRAINING GENERATOR, BLACK SMOKE

                                       (Percent Opacity)
Run
No.
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Observer
1
-10.6
-3.4
-6.0
-2.4
-4.9
-6.4
2.6
-4.4
-5.0
-1.6
0.6
1.0
1.0
-1.2
0.2
-0.2
2
-14.6
-1.2
-5.8
-5.6
-0.9
-10.8
0.0
-3.6
-7.4
-2.8
0.2
-11.2
-2.4
-1.0
-2.6
-2.0
3
-13.0
-0.8
-3.4
-4.4
0.3
-10.2
1.0
-4.2
-1.2
0.2
-0.8
-3.0
2.0
2.4
0.4
1.0
4
-11.8
-0.4
-2.0
-0.4
1.7
-11.4
0.0
-3.4
-1.4
0.4
3.0
1.2
3.6
2.4
5.6
2.0
5
-16.0
-0.2
-3.8
-2.8
0.5
-7.2
-0.6
-7.6
-6.2
-2.0
-0.8
1.2
1.2
1.0
1.0
0.8
6
-13.6
-3.2
-5.2
-0.4
-0.3
-10.4
1.0
-3.8
-2.8
-1.2
5.0
3.0
3.6
2.6
1.8
2.2
7
-13.4
0.4
-2.8
0.0
0.7
-11.2
0.2
-6.0
-6.4
-1.8
-0.2
1.8
-0.4
-0.6
1.0
-0.2
8
-13.2
-2.4
-4.2
0.2
0.7
-11.2
1.4
-5.8
-2.6
0.0
1.2
0.4
-0.8
-0.4
0.8
-2.4
9
-9.6
1.2
-3.0
-3.0
0.1











Meter
35.0
10.8
26.0
20.0
12.3
16.0
11.0
17.6
17.6
13.6
18.0
28.2
13.8
9.0
14.8
22.2
where
                                  A
                                 y< — the predicted deviation

                                 Xf — the average opacity,

in the range from 9 to 35 percent average opacity.  This equation is almost identical to that of the white
smoke data. The value of tc is -7.20, with  131 degrees of freedom which is significant at the 5-percent
level.
                                                13

-------
 TABLE 11. PREDICTED DEVIATION
    FROM METERED OPACITY,
     TRAINING GENERATOR-
             BLACK
Average
Opacity
10
15
20
25
30
35
Predicted
Deviation
0.44
-1.21
-2.86
-4.51
-6.16
-7.81
           Using this equation, deviations may be estimated for
plumes within the applicable range. This is done in Table 11.

     2.    Precision

           The precision estimation for the black smoke determina-
tions is similar to that done for the white smoke data.  The results
from the collaborator who did not complete all 16 runs are not in-
cluded.  The remaining 128 determinations of average opacity are
used in an ANOVA to estimate the precision components. The de-
tails are presented in Appendix B.2.

           The within-observer variance, o2, is estimated as

           ft3 = 3.38
with 105 df. This gives an estimated within-observer standard deviation of
                                             = 1.84,

           The F-ratio for observers is Fc - 5.73 which exceeds the critical value, F0.0 5  (7, 1 05) « 2.10 at
the 5-percent level. Thus the observer bias variance, a£, is significantly different from zero, and is estimated
by

                                            al =1.00

with 7df.  Thus, the observer bias standard deviation is estimated by

                                            &L = 1.00.

           The between-observer variance is estimated from the above as
                                            = 3.38 + 1.00

                                            = 4.38.

The estimated between-observer standard deviation, then, is
                                              = 2.09,
      3.    Predicted Results
           Using the results above, it is possible to predict, at the 95-percent confidence level, the range of
opacities that would occur in the use of the method at particular opacity levels. The predicted determination
at fc-percent opacity would be

                                 k + (a + bk) = k + (3.74 - 0.34*)

                                             = 0.66fe +3.74.
                                                   14

-------
The variability in a reading would be measured by the within-observer standard deviation. For the 95-percent

confidence level and 105 df, the factor 1.96 is used. This gives



                           expected range:  (0.66A + 3.74) ± 1.96 a




                                        :  (0.66*+3.74) ±1.96 (1.84)



                                        :  (0.66* + 3.74) ±3.61.




The expected range is given for selected values of k in Table 12.
TABLE 12. PREDICTED DETERMIN-
The maximum difference between two determinations
 ATIONS OF AVERAGE OPACITY,     made by independent collaborators is determined from these data at

     TRAINING GENERATOR-        the 95.perCent confidence level. The factor 1.96 is again used, and
             BLACK                ,,.  .
                                  this gives
Percent Opacity
5
10
15
20
25
30
35
Expected Range
3.43-10.65
6.73-13.95
10.03^17.25
13.33-20.55
16.63-23.85
19.93-27.15
23.23-- 30.45
                                                  maximum difference:   ± 1.96 v2 o\



                                                                       ±(2.77) ab



                                                                       ±(2.77) (2.09)




                                                                       ± S.79 percent opacity.



                                  at the 95-percent confidence level.
                                                 15

-------
                          V.  STAUFFER CHEMICAL COMPANY TEST
      The second test of the method was conducted at the Stauffer Chemical Company sulfuric acid plant.
This test was specifically designed to test the ability of a qualified observer to accurately read the plume
opacity near the compliance limit for that source.  The standard for sulfuric acid plants is 10 percent, mean-
ing that an opacity of 10 percent or greater is in violation.  '

      The observers were chosen both from government enforcement agencies and from the private  sector.
This was done  to investigate whether the affiliation of the individual influenced his determination of opacity.

      There were 14 runs made on the first day of the test by 11 collaborators, and 16 runs on the second with
9 collaborators for a total of 298 determinations. The  average opacity during each run was determined from
a strip chart readout of opacity as determined by an in-stack transmissometer. Because of this, the "true"
opacity coujd only be determined to the nearest percent, and the observers' determinations were rounded to
the nearest percent opacity for comparison.

      The observers made one deviation from the published method on the second day. Since the plume was
light, they suggested that they read to the nearest 1 percent rather than the nearest 5 percent, and this was
permitted.  However, since the method specified that 5-percent increments be used, the individual readings
subsequently were rounded to the nearest 5 percent prior to being averaged to obtain the determinations used.
While there may be a difference between these and a Method 9 determination, it is felt that it would be slight.

      The determinations are shown in Table 13 along with the average meter opacity for that range. These
values are investigated for both their accuracy and  precision, as for the smoke generator test.

A.    Accuracy

      The deviations from the transmissometer average are shown in Table 14 for the sulfuric acid plant data.
The deviations show a negative bias, falling generally 1  to 3 percent below the metered opacity. To  determine
if there is a linear relationship between the deviations and the meter average, a regression line is fit to these
points.

      The equation of the best fit to these points is

                            deviation = —2.36 + (O.OSXmeter average).

From the slope of the line, it can be seen that there is a slightly positive trend to these  deviations compared
to the negative trend of the smoke generator test.

      To determine if the slope is significant, a r-statistic is calculated as described in Appendix B.I. The
value of the statistic is tc = 0.68 with 296 df. This is not significant, which implies that the slope, 0.05,
is not significantly different from zero.  Thus, no linear relationship can be said to exist between deviation
and opacity for these data.  The best estimate of the deviation, then, is the mean deviation of -2.0 percent
opacity. This is valid in the range studied, from 2  to 15  percent opacity. However, since the majority of the
runs were made between 5  and 10 percent average opacity, it is possible that there is a relationship  between
the two variables that would be noticeable if a wider range of opacities was studied.

B.    Precision

      The determinations are submitted to an ANOVA procedure to estimate the precision of the method at
a sulfuric acid plant. The ANOVA was performed using only the nine observers who completed all 30 runs.
The remaining data set is large enough to obtain the desired estimates, and the necessity for blocking is
eliminated by  their exclusion.
                                                  16

-------
                        TABLE 13.  DETERMINATIONS OF AVERAGE OPACITY,
                                      STAUFFER CHEMICAL

                                         (Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Observer
1
7
5
5
5
7
5
5
5
18
7
1
5
5
5
0
0
0
0
9
6
5
5
3
3
4
5
8
6
6
7
2
3
1
0
0
2
3
3
3
10
5
0
2
2
3
0
0
0
0
8
9
9
7
4
5
6
9
9
9
10
10
3
7
7
7
6
6
6
6
6
15
8
3
5
4
5
1
1
1
0
8
7
5
5
4
3
5
7
7
6
8
5
4
6
5
4
3
5
5
5
5
15
7
1
5
5
3
0
1
2
0
10
12
6
5
5
3
6
10
13
16
15
16.
5
5
5
3
1
4
2
5
5
12
10
1
4
3
5
0
3
1
0
12
13
10
5
6
5
7
13
13
17
13
15
6
5
5
S
5
5
S
5
5
14
8
4
5
5
5
0
4
3
1
14
14
10
6
7
5
7 •
13
16
17
16
15
7
4
4
4
4
5
5
5
5
11
9
2
6
5
7
3
4
1
1
7
6
5
5
2
2
2
4
5
4
5
5
8
5
4
4
3
5
3
3
4
9
5
0
3
4
4
0
1
0
0
9
10
6
5
5
5
9
11
8
9
14
14
9
5
4
4
4
6
6
6
6
14
8
5
5
5
5
3
5
3
4
9
8
6
5
5
5
5
6
7
5
7
6
10
7
7
3
3
6
6
6
5
12
9
4
5
5
5
















11
3
2
3
2
4
3
2
2
13
8
1
2
1
3
















Meter
9
8
8
8
8
8
8
8
14
7
5
6
6
6
2
2
5
4
10
10
10
10
7
7
7
9
9
9
9
9
     The precision estimates are taken from the ANOVA table in Appendix B.3. The within-observer com-
ponent is estimated by:

                                            o2 =4.51

which has 232 df associated with it. This gives an estimated within-observer standard deviation of
                                            = 2.12 percent opacity

     The observer term for these data is shown in Appendix B.3 to be significantly different from zero.  The
observer bias variance, o£, is estimated from the ANOVA table as

                                            &l = 0.93

with 8 df.  This gives an estimated observer bias standard deviation of
                                             = 0.96 percent opacity
                                                 17

-------
                 TABLE 14.  DEVIATIONS FROM METER AVERAGE, STAUFFER CHEMICAL

                                          (Percent Opacity)
Run
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Observer
1
-2
-3
3
-3
-1
-3
^3
-3
4
0
-4
_ j
1
-1
_2
-2
-5
-4
-1
-4
-5
-5
-4
-4
-3
_4
_j
-3
—3
-2
2
-6
-7
g
-8
-6
_5
-5
_5
_4
-2
-5
-4
4
-3
-2
-2
-5
~4
-2
— 1
— 1
-3
-3
-2
-1
0
0
0
1
1
3
-2
-1
1
-2
-2
-2
-2
-2
1
1
-2
-1
2
-1
-1
-1
-4
-4
-2
-3
-5
-5
-3
-4
-2
-2
-2
-3
-1
-4
4
-3
__3
4
-5
-3
-3
—3
-3
1
0
-4
-1
i
i
-3
-2
_j
-3
-4
0
2
^4
-5
-2
-4
	 1
1
4
7
6
7
5
-4
-3
5
-7
-4
-6
-3
-3
-2
3
_ j
-2
3
-1
-2
1
-4
-4
2
3
0
-5
	 1
-2
0
4
4
8
4
6
6
-4
-3
3
-3
-3
-3
-3
-3
0
1
_j
-1
i
— i
-1
—2
2
_2
-3
4
4
0
-4
0
-2
0
4
7
8
7
6
7
-5
-4
4
-4
-3
-3
-3
-3
-3
2
-3
0
j
1
-1
2
-4
-3
-3
_4
-5
-5
-5
-5
-5
_5
-4
-5
^t
-4
8
-4
-4
4
— *T
_5
-3
_5
-5
-4
-5
-2
-5
-3
^
— t.
-2
-2
^1
-5
-4
-1
0
-A
-5
-2
-2
2
2
-I
0
5
5
9
-4
-4
4
— t
_4
—2
-2
-2
_2
0
1
0
-1
i
— i
-1
1
3
-2
0
__2
-2
-4
-5
-2
-2
_2
_3
-2
-4
-2
-3
10
-2
-1

-5
-2
-2
-2
-3
-2
2
-1
-1
i
— i
-1
















11
-6
-6

-6
-4
-4
-6
-6
— 1
1
-4
-4
c
— J
-3

















Meter
9
8

8
8
8
8
8
14
7
5
6

6
2
2
5
4
10
10
10
10
7
7
7
9
9
9
9
9
     The between-observer variance, a$ , is the sum of the within-observer and observer bias components.
The estimated value is

                                          afc = tf + a2

                                             = 0.93+4.51

                                             = 5.44.

This results in a between-observer standard deviation of

                                         ab=^/5A4

                                           = 2.33 percent opacity.

C.   Predicted Results

     The above results can be used to make statements concerning the results that would be obtained by an
independent observer. The range of opacities that could be expected is calculated by using the best estimate
of the deviation, along with  the variation that could be expected.
                                                  18

-------
     At k percent opacity, the expected result would be

                                        (k -2)% ±(1.96)5

at the 95-percent confidence level.  The factor 1.96 is used since with 232 df, the distribution of the t statistic
is essentially a standard normal curve.  Substituting in the value of ogives:

                         expected range :   (k - 1)% ± (1.96X2.12)

                                       :   (k-2) ±4.16

                                       :   it-6.16 tofc + 2.16

for plumes of from 2  to 15 percent average opacity.  For selected values, the expected range is calculated and
shown in Table 15.
   TABLE 15. PREDICTED DETERMINATIONS,
           SULFUR1C ACID PLANT
Avg Opacity
5
10
15
Expected Range
0*-7.16
3.84-12.16
8.84-17.16
*Truncated to zero.
         The maximum expected difference between deter-
    minations by two independent observers is calculated using
    the between-observer variance term. As before, the difference
    is

                 maximum difference = (2.306) \/2al
        TABLE 16. OBSERVER MEANS,
           SULFURIC ACID PLANT
Observer
1
2
3
*4
*5
*6
7
*8
9
Mean
5.00
4.40
5.47
6.47
6.60
7.63
4.57
5.40
5.73
•Observer from enforcement
agency.
largest overall means and the sixth largest.
opacity than the private sector people.
    at the 95-percent confidence level.  The factor 2.306 comes
    from Student's t distribution with 8 df, due to the observer
    bias term. Substituting gives

           maximum difference = (3.26X2.33)

                              = 7.60 percent opacity

    D.   Comparison of Enforcement and Private Observers

         The test personnel were chosen from both an enforcement
    agency and from private concerns to allow the determinations
    by each group to be compared. The nine collaborators who
    completed all 30 runs are used, and the mean over the 30 runs
    is computed. The observer means are shown in Table 16.

         The observers from the enforcement agency were num-
    bers 4, 5, 6 and 8. As can be seen, this group gives the three
The indication, then, is that these people tend  to read a higher
      The hypothesis that the two groups read the same average opacity is tested in Appendix B.3.  The
 test is a contrast among the means, and the test statistic follows Student's t distribution. The statistic is
 tc - 5.69 with 232 df. This is a significant value and the conclusion is that the enforcement observers did
 indeed determine higher average opacities overall than the other group.

      The average opacity for all 30 runs, however, was 7.6, so that the enforcement observers were closer
 to the true opacity than those from other agencies.
                                                 19

-------
                         VI.  RIVERBEND STEAM STATION TEST
      The three tests at the Riverbend Steam Station are treated separately in the analysis since there were
different groups of collaborators and varying sky conditions for the three tests.

      The original test plan was designed to allow the evaluation of the method as used for the determination
of minutes of noncompliance with a particular standard/  '  A test run was set as 10 minutes of observation,
and the reported variable was to be how many observations exceeded a certain standard.

      The experimental factors to be studied included the angle of observation and the relative experience
of the observer. Variations to the method to be evaluated included reading in  1 percent rather than 5 percent
increments and averaging the responses of two observers as opposed to a single observer's result.
                                    i

      In the  third testing period, the test personnel were advised that the average opacity, based on 25 read-
ings, was the test variable of interest.  The other factors for the test remained unchanged.  For purposes of
comparison, average opacities were calculated from the first  two data sets using the first 25 observations of
each run.

      Due to the adverse sky and wind conditions during tests 1 and 2, not all of the planned evaluations are
useful. There was an inability to read the low opacity plumes against the type of background that existed,
and as a result, the determinations were generally well below the concurrent meter average. The precision
estimated, however, is independent  of the accuracy of the determination.

      The observers at each test were divided into two groups for the test, a control and an experimental.
The control group observed the plume at all  times from a position consistent with the method as written and
read in increments of 5 percent. The experimental group either read the plume from a more extreme angle,
in increments of 5 percent  or from the same  angle as the control but in increments of 1 percent. Each group
was composed both of observers who had considerable field experience with the method and of observers
who had relatively little such experience.

A.    Test 1

      There were 10 runs made at the first test by 10 observers. The determinations of average opacity
calculated from the observer record sheets are shown in Table 17. These determinations will be evaluated
with respect  to their accuracy and precision.  There are six missing  determinations in the data set.  These
                      TABLE 17.  AVERAGE OPACITY DETERMINATIONS, STEAM
                                         STATION TEST 1
Run

1
2
3
4
5
6
7
8
9
10
Control Group
1
6.4
13.4
11.6
5.4
8.4
14.8
17.6
9.4
1.2
*
2
4.4
8.2
11.0
4.4
6.4
7.6
9.6
3.2
0.2
*
3
6.2
11.2
10.6
6.0
7.8
12.4
12.4
4.0
2.4
*
4
5.2
9.6
13.8
6.8
6.6
14.6
14.2
9.6
5.6
*
5
5.8
12.8
12.6
4.8
11.4
9.0
11.4
1.8
0.6
*
Meter

16.0
23.5
26.0
11.6
15.7
26.1
24.4
14.2
13.0
13.4
Experimental Group
1
5.4
6.6
10.8
5.2
7.6
12.2
7.5
4.4
1.6
2.0
2
5.4
6.2
9.4
5.5
5.8
8.5
9.4
6.0
3.0
3.0
3 •
1.8
3.8
7.8
4.1
5.9
7.5
4.6
1.0
0.2
0.0
4
5.8
6.4
13.0
5.8
5.9
15.0
8.6
5.0
1.0
0.0
5
2.2
4.4
•8.8
6.5
9.5
8.9
7.0
2.2
0.4
*
* Incomplete data sheet.
                                                 20

-------
occur when due to sky and wind conditions and interference from another plume, the observer felt that at
times lie was unable to make a determination of the opacity, and in these instances, a direct comparison
would not be valid.
      1.    Accuracy

           It is apparent that these determinations are generally considerably lower than the meter average.
The deviations from the meter average are shown in Table 18.

               TABLE 18.  DEVIATIONS FROM METERED OPACITY, STEAM STATION TEST 1
Run
1
•>
3
4
5
6
7
8
9
10
Control Group
I
-9.6
-10.1
-14.4
-6.2
-7.3
-11.3
-6.8
-4.8
11.8
*
2
-11.6
-15.3
-15.0
-7.6
-9.3
18.5
-14.8
- 1 1 .0
12.8
*
3
-9.8
-12.3
-15.4
-5.6
-7.9
- 13.7
-12.0
-10.2
-10.6
*
4
-10.8
-13.9
-12.2
-4.8
-9.1
-11.5
-10.2
-4.6
-7.4
*
5
-10.2
-10.7
-13.4
-6.8
-4.3
-17.1
-13.0
-12.4
-12.4
*
Meter
16.0
23.5
26.0
11.6
15.7
26.1
24.4
14.2
13.0
13.4
Experimental Group
1
-10.6
-16.9
-15.2
-6.4
-8.1
-13.9
-16.9
-9.8
-11.4
-11.4
2
-10.6
-7.3
-16.6
-6.1
-9.9
-17.6
-15.0
-8.2
-10.0
-10.4
3
-14.2
-19.7
-18.2
-7.5
-9.8
-18.6
-19.8
-13.2
-12.8
-13.4
4
-10.2
17.1
-13.0
-5.8
9.8
- 11.1
-15.8
-9.2
-12.0
-13.4
5
-13.8
-19.1
-17.2
-5.1
-6.2
-17.2
-17.4
-12.0
-12.6
*
"Incomplete Data Sheet.
           To  assess  the accuracy of the method under these conditions the control group data is used.
These observers had taken the optimum viewing position consistent with the method, and thus are consid-
ered to best represent  the accuracy that could be expected of the method.

           The deviations are fit to a regression model as with the data from the previous tests. The least-
squares fit to these points is given by

                              deviation = -3.01 - 0.40 (meter average).

           The / statistic for this fit is tc = 5.89 with 43 df. This far exceeds the critical value of t 05 (43)
* 2.02, and this equation may be said to be valid in the range up to about 25 percent opacity.
  TABLE 19.  PREDICTED DEVIATION
  FROM METERED OPACITY, STEAM
          STATION TEST 1
                                                Substituting into this equation, it is possible to predict
                                     what the deviation would be at selected opacities in the applicable
                                     range. This is done for the 5 percent increments and the results
                                     shown in Table 19. As can be seen, a plume opacity up to 10 percent
                                     is barely distinguishable from zero under these conditions.

                                           2.   Precision

                                                The precision estimates are obtained using both groups'
                                     determinations from runs where the experimental group was read-
                                     ing in 5-percent increments. The 1-percent increment data were
intended only to evaluate that variation as  a possible improvement to the published method when working
on low opacity plumes, and due to the viewing conditions, no evaluation using these data is performed. The
precision estimation is shown in Appendix  B.4, using runs 1, 2, 3. 7, 8 and 9, omitting run 10 where there
were 6 incomplete determinations.
Percent Opacity
5
10
15
20
25
Predicted Deviation
-5.01
-7.01
-9.01
-11.01
-13.01
                                                  21

-------
      The estimated within -observer variance is

                                              o2 =3.31

 with 45 df. This gives an estimated standard deviation of

                                       & = 1 .82 percent opacity

      The observers are determined to be a significant source of difference between observed values.  The F-ratio
 is 4.50 which exceeds the critical value of 4.08 with 8 and 54 df. Thus, of, is estimated as

                                             (£-1.93

 with 8  df. The estimated observer bias standard deviation is
                                                = 1.39 percent opacity.

      Combining the above estimates, the between-observer component can be estimated. The between-observer
 variance, cr|, is estimated by
                                             = 3.31 -I- 1.93

                                             = 5.24.

 Thus, the estimated between-observer standard deviation is

                                          ab = V5. 24

                                             = 2.29 percent opacity
      3.    Predicted Results

           Using the results of the previous two sections, it is possible to estimate the range of opacities
that could be expected from a qualified observer determining average opacity under these conditions.

           The expected range at any level, k, of true opacity would be given by

                                expected range: k + [a + bk] ± ( 1 .96) 0

at the 95-percent confidence level.  Substituting gives

                          expected range:  k + [-3.01 + 0.40A:] ± 1.96(1.82)

                                        : [0.60* -3.01] ±3.57.

           The range is calculated for selected opacities and shown in Table 20. As expected, there are no
instances where an accurate determination can be expected under these conditions.

           The maximum difference that could be expected between two observers is estimated using the
 between-observer standard deviation. The difference would not be expected to exceed
                                                 22

-------
         TABLE 20. PREDICTED DETER-
          MINATIONS, STEAM STATION
                    TEST 1
Percent Opacity
5
10
15
20
25
Expected Range
0*
0*
2.42
5.42
8.42
3.56
-6.56
9.56
-12.56
-15.56
"Truncated to zero.
TABLE 21. OBSERVER MEANS
    FOR STEAM STATION
          TEST I
Group
Control




Experimental




Observer
1
2
3
4
5
1
2
3
4
5
Mean
Determination
9.93
6.10
7.80
9.67
7.50
6.05
6.57
3.20
6.63
4.17
                               maximum difference = ?o.05

                                                  = (2.306X1.414X2.29)

                                                  = 7.49 percent opacity

at the 95-percent confidence level.

     4.    Experience Effect

           The means of the observers are shown in Table 21 and these are used to determine if there is any
significant difference between the average opacity as determined by an experienced as opposed to an inex-
perienced observer.  The  test statistic is a /'•statistic based upon a contrast among the means.  The details are
shown in Appendix B.4.

           The calculated r-statistic is

                                             tc= 1.39

with 45 df. There can be said to be a difference between the two groups at the 5-percent level only if tc
exceeds the tabled value.  Since /o,05 (45) * 2.016, the difference is not significant at the 5-percent level.
Thus, for these conditions, the relative experience of the observer had no noticeable effect.

B.   Test 2

     There were 18 runs made at the second test period with 10 collaborators participating.  The personnel
breakdown was identical to  the first test, with 4 experienced and 6 relatively inexperienced observers.  One
run, run 10, could not be completed.  During the course of the run, the opacity meter began its  automatic
calibration check. During this time, the meter displays only the calibration opacities, 0 and 95 percent, and
no true value is available  for comparison. Only ten readings were made before the run was halted, and thus
no determinations could  be  calculated.

     The determinations made by the observers are shown in Table 22.  As in the previous test, missing
determinations occur when an observer is unable to make a determination of the plume opacity due to wind
and sky conditions.
                                                 23

-------
                  TABLE 22. AVERAGE OPACITY DETERMINATIONS, STEAM STATION
                                             TEST 2

Run
I
2
3
4
5
6
7
8
9
11
12
13
14
IS
16
17
18
Control Group
1
11.2
13.8
10.0
5.8
20.0
5.0
13.8
5.0
14.4
5.2
5.6
5.2
9.4
17.0
7.8
12.4
7.6
2
5.8
14.4
9.2
5.4
15.0
0.4
11.4
5.4
10.6
4.0
2.2
0.6
7.0
12.8
7.0
8.0
6.0
3
11.6
13.2
9.2
5.6
12.6
4.4
9.4
6.6
10.8
5.0
7.8
5.0
7.6
16.8
10.8
10.0
6.4
4
10.6
.14.8
12.0
6.2
16.2
1.0
14.2
5.2
15.2
7.6
6.4
0.0
11.8
21.8
9.8
12.4
7.6
5
4.2
5.6
5.2
3.4
13.6
0.0
10.4
0.8
10.6
4.4
2.6
0.0
7.4
15.2
3.2
8.2
5.0

MCtcr
14.4
13.2
11.2
5.6
16.4
6.2
11.6
9.5
15.0
7.5
10.8
4.7
10.1
23.6
13.5
13.4
13.6
Experimental Group
1
8.0
13.7
13.0
4.6
20.0
0.4
*
1.8
8.6
4.4
2.7
0.0
6.6
13.6
7.0
8.4
5.0
2
6.1
7.2
5.2
1.4
13.8
1.6
8.0
5.6
6.0
5.0
3.4
0.0
4.8
13.0
6.0
7.6
6.0
3
4.3
9.4
3.4
0.2
11.4
0.0
6.6
0.6
5.4
0.1
0.0
0.0
0.6
9.8
2.3
1.8
2.2
4
11.7
11.3
4.2
3.4
17.8
0.0
*
4.2
9.4
2.6
2.5
0.0
6.4
15.6
5.4
10.0
7.8
5
8.3
8.6
9.0
4.0
19.0
3.0
10.6
3.8
8.8
3.6
1.8
0.0
2.0
12.3
4.0
7.0
6.6
"Incomplete data sheet.
|Run 10 could not be completed.
      1.    Accuracy

           The deviations of the determinations from the meter average are shown in Table 23. As before,
the determinations are generally below the meter due in large part to the viewing conditions. A linear regres-
sion of the deviations is used to assess the accuracy, and as in the first test, only the control group data is
used.

           The best fit to these points is given by

                              deviation  = — 1.02 — 0.19 (meter average)

for the range from 5 to 25 percent average opacity. The t statistic for this model is tc - 2.56 with 83 df,
which is significant at the 5-percent level.

           Using the above equation, it is  possible to predict what deviation would be expected at various
opacity levels.  This is done in Table 24 for selected values in the applicable range.  These are somewhat better
than those from the first test, but still have a strong negative bias at the low range of opacities.

     2.    Precision

           The precision components for  this test will be estimated using runs 3—9,  13, 14, 17 and 18.  The
remaining runs were those where the experimental group was reading in 1-percent increments, and a direct
comparison between the two sets of readings is not proper.

           The precision components are  estimated by an ANOVA procedure comparable to that for the
first test. The details and ANOVA table are shown in Appendix B.5. The within-observer component is
estimated from the error term of the ANOVA.
                                                24

-------
               TABLE 23.  DEVIATIONS FROM METERED OPACITY, STEAM STATION TEST 2
                                          (Percent Opacity)
Run

1
2
3
4
5
6
7
8
9
11
12
13
14
15
16
17
18
Control Group
12345
-3.2 -8.6 -2.8 -3.8 -10.2
0.6 1.2 0.0 1.6 -7.6
1.2 -2.0 -2.0 0.8 -6.0
0.2 0.2 0.0 0.6 -2.2
3.6 1.4 -3.8 -0.2 -2.8
-1.2 -5.8 -1.8 -5.2 -6.2
2.2 -0.2 -2.2 2.6 -1.2
-4.5 -4.1 -2.9 -4.3 -8.7
-0.6 -4.4 -4.2 0.2 -4.4
-2.3 -3.5 -2.5 0.1 -3.1
-5.2 -8.6 -3.0 -4.4 -8.2
0.5 -4.1 0.3 -4.7 -4.7
-0.7 -3.1 -2.5 1.7 -2.7
-6.6 -10.8 -6.8 -1.8 -8.4
-5.7 -6.5 -2.7 -3.7 -10.3
-1.0 -5.4 -3.4 -1.0 -5.2
-6.0 -7.6 -7.2 -6.0 -8.6
Meter

14.4
13.2
11.2
5.6
16.4
6.2
11.6
9.5
15.0
7.5
10.8
4.7
10.1
23.6
13.5
13.4
13.6
Experimental Group
1 2345
-6.4 -8.3 -10.1 -2.7 -6.)
0.5 -6.0 -3.8 -1.9 -4.6
1.8 -6.0 -7.8 -7.0 -2.2
-1.0 -4.2 -5.4 -2.2 -1.6
3.6 -2.6 -5.0 1.4 2.6
-5.8 -4.6 -6.2 -6.2 -3.2
* -3.6 -5.0 * -1.0
-7.7 -3.9 -8.9 -5.3 -5.7
-6.4 -9.0 -9.6 -5.6 -6.2
-3.1 -2.5 -7.4 -4.9 -3.9
-8.1 -7.4 -10.8 -8.3 -9.0
-4.7 -4.7 -4.7 -4.7 -4.7
-3.5 -5.3 -9.5 -3.7 -8.1
-10.0 -10.6 -13.8 -8.0 -11.3
-6.5 -7.5 -11.2 -8.1 9.5
-5.0 -5.8 -11.6 -3.4 -6.4
-8.6 -7.6 -11.4 -5.8 -7.0
*lncomplete Data Sheet.
  TABLE 24. PREDICTED DEVIATION
  FROM METERED OPACITY, STEAM
          STATION TEST 2
Percent Opacity
5
10
15
20
25
Predicted Deviation
-1.97
-2.92
-3.87
-4.82
-5.77
           The within-observer variance, a2, is estimated by

                          62 = 3.37

with 88 df. This gives an estimated standard deviation of
                                                                  = 1.84 percent opacity

                                                 The observer-within-group term is used to determine
                                      if there is a significant observer bias component. The F-ratio is
                                      Fc = 8.37, which is significant at the 5-percent level. This is
equivalent to saying that the observer bias component, a|, is greater than zero. The observer bias variance
is estimated from the observer mean square as

                                            &l = 2.26

with 8 df. The observer bias standard deviation is estimated by

                                            »L =
                                              =  1.50 percent opacity
           The between-observer component is estimated from the above.  The between-observer variance,
a?, is estimated as
                                            = 2.26 + 3.37
                                            = 5.63.

           This gives an estimated standard deviation of
                                            6b =V5.63

                                              = 2.37 percent opacity
                                                 25

-------
     3.    Predicted Results

           The accuracy and precision statements above can be combined to predict the performance that
could be expected when using Method 9 under these conditions.

           The range of opacities that could be expected from a qualified observer at fc-percent opacity, at
the 95-percent confidence level, is given by

                                expected range:  k + [a + bk] ± 1.96 &

                                             :  fc +[-1.02-0.19*]+ 1.96(1.84)

                                             :  [0.81*- 1.02] ±3.61
  TABLE 25. PREDICTED DETER-
   MINATIONS, STEAM STATION
             TEST 2
Percent Opacity
5
10
15
20
25
Expected Range
0*-6.64
3.47-10.69
7.52-14.74
11.57-18.79
15.62-22.84
*Truncated to zero.
           Values for this expected range are shown in Table 25 for
the range of opacities studied. As can be seen, the observer would be
expected to be biased on the low side above 10 percent opacity.

           The between-observer standard deviation can be used to
estimate the difference that  could be expected between two observers.
The maximum expected difference is calculated as

             maximum difference = fo.osW V2 Of,

at the 95-percent confidence level. The 8 df comes from the observer
bias term. Substituting gives
                              expected difference = (2.306X1.414X2.37)

                                                = 7.73 percent opacity
     4.    Experience Effect
           The relative experience of an observer is investigated using the data from the ANOVA in Appendix
B.5.  The means of the observers over the 11 runs are calculated, and these are shown in Table 26.
 TABLE 26. OBSERVER MEANS FOR STEAM
             STATION TEST 2
Group
Control




Experimental




Observer
1
2
3
4
5
1
2
3
4
5
Mean
Determination
9.48
6.76
7.82
8.76
5.42
6.84
5.20
2.56
6.32
6.32
                   The 1% variation runs are excluded to avoid the
        possibility of this variation influencing the results. Run 7 is
        also excluded, due to the two missing observations.

                   The details of the test for significance are presented
        in Appendix B.5.  The value of the test statistic is tc = 3.19.
        From a Student's t distribution with 88 df, the  critical value
        is approximately 1.99, and the hypothesis of equality  between
        experienced and non-experienced observers is rejected  since
        tc exceeds  1.99.

        C.    TestS

              There were 8 collaborators for  the third test.  Of these,
        4 were from state and county agencies, 2 were from EPA
offices and 2 were from a private contractor. The observers were divided into two equal groups as in the two
previous tests.  There were  10 observation runs on the first day and 14 on the second for a total of 192 indi-
vidual determinations. These are shown in Table 27, along with the meter average concurrently obtained.
                                                26

-------
                 TABLE 27.  AVERAGE OPACITY DETERMINATIONS. STEAM STATION
                                             TEST 3
Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Control Group
1
5.0
13.6
5.0
9.0
10.4
6.6
11.0
5.8
7.6
6.8
25.0
40.8
11.2
17.0
12.8
26.0
13.4
29.2
23.8
23.0
33.8
17.4
23.8
12.6
2
5.2
13.6
5.0
8.4
6.8
6.6
10.2
6.0
5.0
6.0
23.0
39.0
9.8
15.2
10.8
25.6
12.4
25.2
23.0
19.4
34.6
12.6
19.8
15.0
3
3.0
15.4
4.0
8.6
8.8
5.4
8.4
5.8
5.4
5.4
36.2
33.0
5.6
6.4
7.8
19.0
11.2
24.0
18.4
16.6
30.0
10.0
18.0
9.4
4
5.0
5.4
5.0
5.2
5.2
5.0
5.6
5.2
5.0
5.0
16.8
39.6
6.2
10.2
10.8
25.2
7.6
25.0
21.2
12.4
33.2
8.0
19.2
6.8
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
Experimental Group
1
4.2
12.2
4.2
7.8
9.2
5.6
5.4
3.2
3.0
3.8
15.3
25.2
10.2
18.0
14.4
29.8
16.6
30.0
21.8
22.8
34.6
14.0
21.2
15.0
2
4.6
8.6
3.0
7.0
6.4
6.0
5.6
2.0
1.6
2.0
13.5
28.7
9.0
14.2
16.6
26.6
16.4
30.6
25.6
21.6
40.2
13.4
25.2
9.2
3
2.1
6.1
2.4
1.8
3.8
4.0
4.0
2.0
3.3
2.3
16.5
25.7
8.6
12.2
9.6
24.2
10.0
22.0
15.8
13.5
31.6
11.8
22.2
8.8
4
3.1
10.8
5.0
6.8
7.8
4.8
6.0
1.6
4.5
5.3
10.2
18.1
7.4
8.8
12.8
21.6
12.8
27.2
21.6
18.8
31.8
12.2
24.8
12.8
     To evaluate the observer performance at this test, the accuracy and precision will be estimated as for
the first two sites.  In addition, the relative performance of the tested variations to the method will be evaluated.

     1.    Accuracy
           The evaluation of the accuracy of the method is performed using the deviations from the meter
obtained from the control group data as shown in Table 28. These deviations were all from determinations
made within the constraints of the method, and thus offer the most applicable information.
               TABLE 28. DEVIATIONS FROM METERED OPACITY, STEAM STATION TEST 3
Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Control Group
1
1.9
6.1
0.8
1.8
3.7
1.6
5.6
2.4
3.4
2.0
3.8
4.8
0.9
2.8
0.6
-3.6
-1.2
0.9
- 10.8
-4.2
-3.1
2.2
-4.0
-1.8
2
2.1
6.1
0.8
1.2
0.1
1.6
4.8
2.6
0.8
1.2
1.8
3.0
-0.5
1.0
-1.4
-4.0
-2.2
-3.1
-11.6
-7.8
-2.3
-2.6
-8.0
0.6
3
-0.1
7.9
-0.2
1.4
2.1
0.4
3.0
2.4
1.2
0.6
15.0
-3.0
-4.7
-7.8
-4A
-10.6
-3.4
-4.3
-16.2
-10.6
-6.9
-5.2
-9.8
-5.0
4
1.9
-2.1
0.8
-2.0
-1.5
0.0
0.2
1.8
0.8
0.2
-4. 4
3.6
-*.\
-^.0
-1.4
-4.4
-7.0
-3.3
-13.4
-14.8
-3.7
-7.2
-8.6
-7.6
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
Experimental Giou
1
1.1
4.7
0.0
0.6
2.5
0.6
0.0
-0.2
-1.2
-1.0
-5.9
-10.8
-0.1
3.8
2.2
0.2
2.0
1.7
-12.8
-4.4
-2.3
-1.2
-6.6
0.6
2
1.5
1.1
-1.2
-0.2
-0.3
1.0
0.2
-1.4
-2.6
-2.8
-7.7
-7.3
-1.3
0.0
4.4
-3.0
1.8
2.3
-9.0
-5.6
3.3
-1.8
-2.6
-5.2
3
-1.0
-1.4
-1.8
-5.4
-2.9
-1.0
-1.4
-1.4
-0.9
-2.5
-4.7
-10.3
-1.7
-2.0
-2.6
-5.4
-4.6
-6.3
-18.8
-13.7
-5.3
-3.4
-5.6
-5.6

4
0.0
3.3
0.8
-0.4
1.1
-0.2
0.6
-1.8
0.3
0.5
-11.0
-17.9
-2.9
-5.4
0.6
-8.0
-1.8
-1.1
-13.0
-8.4
-5.1
-3.0
3.0
-1.6
                                                 27

-------
           As was the case with the smoke generator test, there appears to be a relationship between the
magnitude and direction of the deviation and the meter average. To test this, a linear regression model is fit
in the manner of Appendix B.I.  The least-squares line obtained is

                                deviation = 2.27 - (0.24)(meter average).

The t statistic for this test is tc = 6.10 with 94 df, which is significant at the 5-percent level.  This equation
is valid in the range from 5 to 35 percent average opacity.

           By substituting into the equation, it is possible to predict what the deviation would be at
different levels of true  average  opacity.  This is done in Table 29.   These predicted  deviations are
comparable  to those  obtained  in the smoke generator test.  The observers are  fairly  accurate in the low
opacity ranges, but acquire a low bias as the opacity increases.

           For comparison, the deviations from the meter are calculated for the experimental group as
well.  The three cases, 45°, 90°, and 1 percent are treated separately and regression lines calculated for
                                             each case. These are:
    TABLE 29.  PREDICTED DEVIATION
        FROM METERED OPACITY,
          STEAM STATION TEST 3
Percent Opacity
5
10
15
20
25
30
35
Predicted Deviation
1.07
-0.13
1.33
-2.53
-3.73
-4.93
-6.13
            deviation = -0.68 - 0.03 (opacity)

            deviation = 1.34-0.19 (opacity)

            deviation = 1.76 - 0.39 (opacity)

respectively, for the three cases. The first line does not
have a significant slope, while the second two do.  However,
it is easy to see that the three slopes are different, and
thus that the observers' determinations differed when the man-
ner and position of observation differed.
           It is interesting to note that the most accurate readings were made when the group was at an
approximate  45° angle to the sun. T-he mean deviation was —1.18 over the range from 5—32 percent average
opacity.

      2.   Precision

           The model used for estimating the precision of the method is similar to that used in the two previous
tests.  The variance components are estimated from an ANOVA of all determinations made by  both  groups
on the 45° and 90°  runs.  The ANOVA is presented in detail in  Appendix B.6.  Precision estimation using the
1 percent variation runs is contained in a separate evaluation.

           The estimated within-observer variance is

                                              o2 = 3.57

with 90 df.  The estimated within-observer standard deviation, then, is

                                              §=V/3.57

                                               = 1.89 percent opacity

           The F-ratio for observers/groups isFc = 14.99.  From a table of the F distribution, the critical
value at the 5-percent significance level is FQ.OS (6,90) « 2.30. Thus, the observer/group term is significant,
                                                 28

-------
which is equivalent to saying that an observer bias variance term,
estimated by
                                                              , does exist. The observer bias variance is
with 6 df. The estimated observer standard deviation, then, is
                                              = 1 .77 percent opacity.

           The between-observer variance is estimated from the above two terms as
                                              = 3.12 + 3.57

                                              = 6.69.

This gives a between-observer standard deviation of

                                    ab = V6 .69

                                       = 2.59 percent opacity.

      3.    Predicted Results

           The above results can be combined to give estimates of the type of average opacities that would
be expected from field use of the method. The expected range of opacities that would be reported by an
independent observer are obtained by using the predicted deviation and the within-observer standard deviation.

           The range  of opacities, at k percent opacity, is defined as

                     expected range: [k + predicted deviation]  ±(1.96)a

at the 95-percent confidence level.  The normal variate 1.96 is used since there are 69 df for the within-observer
component. Substituting gives

                   expected range: k% + 2.27 - (0.24)* ± (1 .96X1 .89)

                                 (0.76)Jt + 2.27 ± (3.70) percent opacity
   TABLE 30. PREDICTED DETERMINATIONS,
           STEAM STATION TEST 3
Percent Opacity
5
10
15
20
25
30
35
Expected Range
2
6
37-9.77
17-13.57
9.97-17.37
13
17
21
25
77-21.17
57-24.97
37-28.77
1^-32.57 '
                                              Values for the expected range are given in Table 30 for 5-per-
                                              cent increments in the applicable range. Similar to the pre-
                                              vious results, these ranges contain the true opacities at the
                                              low end, but acquire a negative bias beginning at the 25-per-
                                              cent level.

                                                         The between-observer component is used to
                                              estimate the difference that would be expected between
                                              two  observers making independent observations of the plume
                                              opacity. The maximum difference that would be expected
                                                 29

-------
                             maximum difference = (Q 05 (6) -v/lSj,

                                                 = (2.447)(1.414)(2.59)

                                                 = 8.96 percent opacity

at the 95-percent confidence level.

           The t value has 6 df from the observer bias variance term. This maximum difference is slightly
larger than the numbers previously obtained.

     4.    Experience Effect

           The observer means shown in Table 31, are used to test the hypothesis that the experienced
observers read the same average opacity as the inexperienced. The test is a contrast among the means, and
the details are given in Appendix B. 6.
   TABLE 31.  OBSERVER MEANS FOR
        STEAM STATION TEST 3
Group
Control



Experimental



Observer
1
2
3
4
1
2
3
4
Mean
Determination
15.31
14.00
11.40
11.46
14.95
14.46
11.19
12.76
           The test statistic is based on Student's t distribution,
and compares the difference between the four experienced ob-
servers, 1, 2, 5, and 6, and the four inexperienced, 3, 4, 7, and
8. The value  of the statistic is tc = 8.89 with 69 df. The tabled
value is ro.os(69) ** 2.00, and thus the hypothesis is clearly re-
jected.  The conclusion, then, is that the experience observers
read different, and in this case higher, average opacities than the
inexperienced. By inspection of the deviations, it appears that
this difference occurs mainly in the high opacity range ( > 25%)
where the inexperienced observers read much lower than the
experienced.  As it was previously, though, the experienced
observers compare more favorably to the meter average.

     5.   1'Percent Increment*
                                                    On runs 1,2,9, 10, 11, 12, 19 and 20, the experi-
                                         mental group read from the same viewing position as the con-
trol, but recorded their observations to the nearest 1 percent opacity rather than the nearest 5 percent. These
runs are considered separately from the others in terms of their accuracy and precision.

           a.    Accuracy

                As the regression line in Section VI.C.l above indicates, there was a negative  bias in the deter-
minations.  Using that  line, predicted deviations are calculated for the applicable  range, and these are calculated
and presented in Table 32.
   TABLE 32. PREDICTED DEVIATIONS,
      1-PERCENT INCREMENT DATA
Opacity
5
10
15
20
25
30
35
Expected Deviation
—0.19
-2.14
^.09
-6.04
-7.99
-9.94
-11.84
                 In addition, the runs are broken down into
single readings, and the opacity from the meter is used to group
the observations.  All observations made at the same opacity
level are treated as a single sample, and the accuracy of each is
calculated. The results are summarized in Table 33.

                 The meter values ranged from 3 to 62 percent
opacity.  For each, a mean is calculated and a standard deviation.
A 95-percent confidence interval is calculated using the expression
                                                             CIo.95
                                                30

-------
             TABLE 33.  1-PERCENT DATA SUMMARY
Opacity
3
4
5
6
7
8
9
10
11
13
17
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
44
45
46
50
58
62

.X
3.3
3.2
4.6
9.4
6.0
9.5
9.8
10.4
11.0
14.3
13.3
13.5
13.9
13.3
15.1
14.9
20.0
18.3
19.1
17.3
18.6
20.0
20.3
20.3
21.4
22.2
20.7
20.4
22.8
23.5
27.0
21.3
22.8
32.3
31.0
30.5
32.0
36.3
36.5
,s
2.0
2.0
3.1
3.7
3.3
2.9
5.0
2.7
2.3
4.0
2.2
5.0
3.8
3.4
4.5
2.7
1.6
4.0
6.2
4.2
4.6
4.2
4.0
6.6
3.9
6.7
6.8
6.1
3.6
4.4
8.8
6.1
1.7
5.4
4.7
4.9
5.4
4.8
6.6
n
108
100
112
20
28
4
8
8
8
4
4
4
24
44
16
12
4
16
32
20
32
12
16
8
16
20
32
36
8
4
4
8
4
4
4
4
4
4
4
t
1.96
1.96
1.96
2.093
2.052
3.182
2.365
2.365
2.365
3.182
3.182
3.182
2.069
2.021
2.131
2.201
3.182
2.131
2.042
2.093
2.042
2.201
2.131
2.365
2.131
2.093
2.042
2.031
2.365
3.182
3.182
2.365
3.182
3.182
3.182
3.182
3.182
3.182
3.182
CI0.95
UL
3.677
3.592
5.174
11.132
7.280
14.114
13.981
12.658
12.923
20.664
16.800
21.455
15.505
14.336
17.497
16.616
22.546
20.431
21.338
19.266
20.260
22.669
22.431
25.819
23.478
25.336
23.155
22.465
25.810
30.500
41.001
26.401
25.505
40.891
38.478
38.296
40.591
43.937
47.001
LL
2.923
2.808
4.026
7.668
4.720
4.886
5.619
8.142
9.077
7.936
9.800
5.545
12.295
12.264
12.703
13.184
17.454
16.169
16.862
15.334
16.940
17.331
18.169
14.781
19.322
19.064
18.245
18.335
19.790
16.500
12.999
16.199
20.095
23.709
23.522
22.704
23.409
28.663
25.999
Max
10
9
16
17
14
13
18
15
15
19
15
19
23
26
24
20
22
28
28
25
25
25
26
28
25
30
34
36
28
28
39
28
25
38
36
37
38
40
44
Min
0
0
0
4
1
6
5
8
8
10
10
7
8
8
8
12
18
12
5
11
9
12
12
8
16
7
5
5
19
18
21
12
21
25
25
25
25
30
30
where CQ.OS is the tabled value,
taken either from the standard
normal or from Student's / distribu-
tion, and s is the calculated standard
deviation. The observations are
accurate when the "true value" lies
within the interval.  As can be seen
from the table, the 1 -percent data
are generally accurate up to 20 per-
cent opacity, but beyond 20 percent
the accuracy standard is met only
once. Also included in the tables
are the maximum and minimum  ob-
servations at each opacity level.

      b.     Precision

            The precision of the
1 -percent variation is investigated
using an ANOVA, as before.  The
details are contained in Appendix
B.7. The  estimated within-observer
variance is

            ft2  = 8.29

with 21 df.  This gives an estimated
within-observer standard deviation
of
                                                                         = 2.88 percent opacity.

                                                                             This value is somewhat
                                                                  larger than that obtained from the
                                                                  5-percent variation. To test whether
                                                                  or not these two variance terms are
equivalent, an F-ratio is calculated comparing this estimate to the variance estimate from the 5-percent in-
crement data. The statistic is Fc = 2.32 with 21 and 90 df. From the tables, the critical value for this test
is FQ 05 (2 1 ,90) « 1.75.  Since the calculated F exceeds the tabled F, the two variances may be said to be
different.

           The F-ratio for observers is Fc = 1 .84 with 3 df. This is not significant at the 5-percent level,
which indicates that there was no observer effect in these runs.  Thus, a£ is taken to be zero, and
                                             0\ = 8.29,
with 21 df as above.
      c.    Predicted Results

           Using the results obtained above, the range of opacities that would be expected from the use
of this variation can be predicted. As before, the range is calculated as
                                                 31

-------
                              expected range:   A: + predicted deviation ± tQQ5 (21) a

                                           :   *+ [1.76-0.39*] ±(2.080)(2.88)

                                           :   [0.71 k+  1.76] ±(5.99)

Values of this range are listed in Table 34 for selected opacity levels. The data indicate that there is fairly
good agreement at the lower opacities, but that a negative bias appears above 25 percent.  This is a com-
parable result to the 5-percent increment data.

           The maximum difference that would be expected between observations by two independent
observers is estimated using the between-observer standard deviation.  As before, the equation is
                        maximum difference = fu.05

                                           = (2.080)v/f6"38

                                           = (2.080X4.07)

                                           = 8.47 percent opacity

           As expected, this value is greater than that for the 5-percent increment data.  The apparent
conclusion is that there was more variability in the 1-percent data than the 5 percent. A contributing factor
to this was undoubtedly that  the observers had not been trained to read to the nearest percent, but it is
impossible to determine how  much effect this actually had.  From a second ANOVA shown in Appendix
B.7, the two groups of observers cannot be shown to be reading different average opacities on these runs.
  TABLE 34. PREDICTED DETER-
  MINATIONS, 1-PERCENT DATA
                                       TABLE 35. OPACITY DETERMINATIONS
                                             USING PAIRED OBSERVERS
Opacity
5
10
15
20
25
30
35
40
Expected Range
0*-11.30
2.87-14.85
6.42-18.40
9.97-21.95
13.52-25.50
17.07-29.05
20.62-32.60
21.17-36.15
*Truncated to zero.
       6.
Two Observers
            A second variation to be
 studied involved taking the average
 of two observers results as opposed  to
 a single observer's average. The 6
 possible pairings among the control
 group are made, and the determinations
 are shown in Table 35 with the devia-
 tions shown in Table 36.
Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Observer Pair
(1,2)
5.1
13.6
5.0
8.7
8.6
6.6
10.6
5.9
6.3
6.4
24.0
39.9
10.5
16.1
11.8
25.8
12.9
27.2
23.4
21.2
34.2
15.0
21.8
13.8
(1,3)
4.0
14.5
4.5
8.8
9.6
6.0
9.7
5.8
6.5
6.1
30.6
36.9
8.4
11.7
10.3
22.5
12.3
26.6
21.1
19.8
31.9
13.7
20.9
11.0
(1,4)
5.0
9.5
5.0
7.1
7.8
5.8
8.3
5.5
6.3
5.9
20.9
40.2
8.7
13.6
11.8
25.6
10.5
27.1
22.5
17.7
33.5
12.7
21.5
9.7
(2,3)
4.1
14.5
4.5
8.5
7.8
6.0
9.3
5.9
5.2
5.7
29.6
36.0
7.7
10.8
9.3
22.3
11.8
24.6
20.7
18.0
32.3
11.3
18.9
12.2
(2,4)
5.1
9.5
5.0
6.8
6.0
5.8
7.9
5.6
5.0
5.5
19.9
39.3
8.0
12.7
10.8
25.4
10.0
25.1
22.1
15.9
33.9
10.3
19.5
10.9
(3,4)
4.0
10.4
4.5
6.9
7.0
5.2
7.0
5.5
5.2
5.2
26.5
36.3
5.9
8.3
9.3
22.1
9.4
24.5
19.8
14.5
31.6
9.0
18.6
8.1
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
                                               32

-------
           There are 15 deviations of
absolute magnitude greater than 7.5 per-
cent out of the 144 pairings, and also
15 out of the original 96 determinations.
Thus, there was a decrease in the fre-
quency of the large deviation. The per-
centages are 0.10 as opposed to 0.16.
In Appendix B.8, it is shown that this
is a significant decrease in frequency.

           The precision that would be
expected from this variation is esti-
mated by taking one-half of the between-
observer variance term from the previous
analysis.  The variance between two
observations, then, would be

         o£ =  1/2 (6.69)

            = 3.35

and a standard deviation of

    dh = 1.83 percent opacity.
                                           TABLE 36. DEVIATIONS FROM OPACITY, PAIRED OBSERVERS
Run
1
2
3
4
5
6
7
8
9
10
It
12
13
14
15
16
17
18
19
20
21
22
23
24
Observer Pail
(1,2)
2.0
6.1
0.8
1.5
1.9
1.6
5.2
2.5
2.1
1.6
2.8
3.9
0.2
1.9
-0.4
-3.8
-1.7
--1.1
-11.2
-6.0
-2.7
-0.2
-6.0
-0.6
(1,3)
0.9
7.0
0.3
1.6
2.9
1.0
4.3
2.4
2.3
1.3
9.4
0.9
-1.9
-2.5
-1.9
-7.1
-2.3
-1.7
-13.5
-7.4
-5.0
-1.5
-6.9
-3.4
(1,4)
1.9
2.0
0.8
-0.1
1.1
0.8
2.9
2.1
2.1
1.1
-0.3
4.2
-1.6
-0.6
-0.4
-4.0
-4.1
-1.2
-12.1
-9.5
-3.4
-2.5
-6.3
-4.7
(2,3)
1.0
7.0
0.3
1.3
1.1
1.0
3.9
2.5
1.0
0.9
8.4
0.0
-2.6
-3.4
-2.9
-7.3
-2.8
-3.7
-13.9
-9.2
-4.6
-3.9
-8.9
-2.2
(2,4)
2.0
2.0
0.8
-0.4
-0.7
0.8
2.5
2.2
0.8
0.7
-1.3
3.3
-2.3
-1.5
-1.4
-4.2
-4.6
-3.2
-12.5
-11.3
-3.0
-4.9
-8.3
-3.5
(3,4)
0.9
2.9
0.3
0.3
0.3
0.2
1.6
2.1
1.0
0.4
5.3
0.3
-4.4
-5.9
-2.9
-7.5
-5.2
-3.8
-14.8
-12.7
-5.3
-6.2
-9.2
-6.3
Meter
3.1
7.5
4.2
7.2
6.7
5.0
5.4
3.4
4.2
4.8
21.2
36.0
10.3
14.2
12.2
29.6
14.6
28.3
34.6
27.2
36.9
15.2
27.8
14.4
           The 95-percent confidence limit on the difference between two determinations made this way
 would be

                             max imum difference:  t$ _ Q 5 (6) \/2aft

                                                :  (2.447X1-414X1.83)

                                                :  6.33 percent opacity.

 This is a 29-percent reduction from the single observer determination result.
                                                 33

-------
                                VII. COMPOSITE ESTIMATION
      As a final evaluation, it is investigated whether the precision and accuracy from all the tests may be
combined to produce statements that are applicable to any site. The accuracy statements are derived from
the training generator and Steam Station Test 3 data, where good background was available. The sulfuric
acid test data is not used since it applies to a much narrower range of opacities.

A.   Accuracy

      The slopes of the regression lines for the training generator and steam station test are tested for equal-
ity.  A test statistic is calculated for comparing the three estimated slopes using their standard deviations and
an uncertainty factor. The details are given in Appendix B.9.

      The slopes can be said to be estimating the same true value if they differ from each other by no more
than 0.12.  As can be seen, the difference between the largest and the smallest slope estimate is 0.10, and
thus the three values are insignificantly different from one another.

      Because of this, a composite line is calculated using all the points from the three sets of data. The
equation thus calculated is

                             deviation = 3.13 — (0.31) (percent opacity)

for the range from 5 to 35 percent average  opacity. The expected deviation calculated using this equation is
shown in Table  37 for values in that range.

TABLE 37. PREDICTED DEVIATION    B.   Precision
    FROM METERED OPACITY,
      COMPOSITE ESTIMATE
Opacity
5
10
15
20
25
30
35
Expected Deviation
1.58
0.03
-1.52
-3.07
-4.62
-6.17
-7.72
                                         The six estimates of the within-observer variance, aj, are com-
                                    pared using Bartlett's tesr3'. The test statistic is a chi-square, and is
                                    calculated to be 13.77 with 5 df, and the significance level for this
                                    value is approximately 0.02.  This is sufficient to accept the hypoth-
                                    esis that all are estimating the same true variance.

                                         The combined estimate is obtained by "pooling" the variances,
                                    as described in Appendix B.9. The pooled variance is
with 693 df.  This gives an estimated within-observer standard deviation of
                                       = 2.05 percent opacity.

      Similarly, the 6 values of o/,are tested for equality using Bartlett's test. The test statistic is 4.72 with
5 df, which is not significant.  Thus, for these components as well, the hypothesis of equality is accepted.
The pooled estimate of the observer bias variance is

                                             al = 1.66

with 44 df. This gives an estimated observer bias standard deviation of

                                     aL = 1.29 percent opacity.
                                                  34

-------
      Combining the above estimates, the between-observer variance is estimated as



                                           = 4.22 + 1.66

                                           = 5.88.

This gives a between-observer standard deviation of
                                       = 2.42 percent opacity.
C.    Predicted Results
     Using the results of the previous two sections, the expected range and maximum expected difference
can be calculated for the composite values. The expected range would be

                           expected range: k + [3.13 -0.3 Ik] ± 1.96 d

                                         : (0.69fc + 3.13)±1.96(2,05)

                                         : (0.69* + 3.13) ±4.02

for any true opacity k, at the 95-percent confidence level. The expected range is presented for selected
values of k in Table 38. As before, the results show accuracy up  to about 25 percent opacity, then acquire
a low-side bias.
 TABLE 38. PREDICTED DETERMIN-
  ATIONS OF AVERAGE OPACITY,
      COMPOSITE ESTIMATE
Percent Opacity
5
10
15
20
25
30
35
Expected Range
2.56-10.60
6.01-14.05
9.46-17.50
12.91-20.95
16.36-24.40
19.81-27.85
23.26-31.30
     The maximum difference that could be expected between two
independent observers is calculated using the between-observer standard
deviation. At the 95 percent confidence level, the difference could not
be expected to exceed

           maximum difference:   fo.os (44) \/2 a\

                             :   (2.021) (1.414) (2.42)

                             :   6.92 percent opacity.

The 44 df are due to the observer bias term.
                                                  35

-------

-------
APPENDIX A. EPA METHOD 9 FOR VISUAL DETERMINATION
                   OF OPACITY
                        37

-------

-------
A.1.    Federal Register, Vol 36, No. 247-Thursday, December 23, 1971
                                    RULES  AND REGULATIONS
                                                         24895
           Horn, Jerome J-, Maintenance. Calibration,
         and  Operation of Isokinettc  Source Sam-
         pling Equipment. Environmental  Protection
         Agency, Air Pollution.  Control Office Publi-
         cation No. APTD-O576.
           Shell Development Co. Analytical Depart-
         ment, Determination of Sulfur Dioxide and
         Sulfur Trioxide in Stack Oases,  Emeryville
         Method Series, 4516/5»a.

         MITTHOD  9	VISUAL  DETERMINATION  OF  THE
           OPACITY  OF EMISSIONS  FROM  STATTONARIT
           SOURCES

           1.  Principle and applicability.
           1.1  Principle. The relative opacity of an
         emission  from a stationary source is de-
         termined visually by  a  qualified observer.
           1.2  Applicability.  This method is  appli-
         cable for the determination of the relative
         opacity of  visible emissions from stationary
         sources only when  specified by teat proce-
         dures for determining  compliance with the
         New Source Performance Standards.
           2.  Procedure.
           2.1  The qualified observer stands at ap-
         proximately two stack heights, but not more
         than  a quarter ol a mile from the base of
         the stack with the sun to his back. Prom a
         vantage point perpendicular to the plume,
         the observer studies the point of greatest
         opacity in  the plume.  The data required In
Figure 9-1 is recorded every 15 to 30 seconds
to the nearest 5'." opacity. A minimum of 25
readings is taken.
  3. Qualifications.
  3.1  To certify as an observer, a candidate
must  complete  a smokereadlng  course con-
ducted by EPA, or equivalent;  in order to
certify the candidate  must  assign  opacity
readings  in  5% Increments to  25 different
black plumes and 25 different white plumes,
with  an  error not to exceed 15 percent on
any one reading and an average error not to
exceed 7.5  percent in  each category.  The
smoke generator used  to qualify  the ob-
servers must be equipped with  a calibrated
smoke indicator or light transmission meter
located  in  the source  stack, if the  smoke
generator is to  determine  the actual opacity
of the emissions. All qualified observers must
pass this test  every 6  months  in order to
remain certified.
  4. Calculation*.
  4.1  Determine the average opacity.
  6. References.
  Air  Pollution Control District Rule* and
Regulations, Loa Angeles  County Air Pollu-
tion Control District, Chapter 2, Schedule 6,
Regulation 4. Prohibition, Rule SO, 17 p.
  Kudluk. Rudolf. Rlngelmann Smoke Chart,
VJS. Department of Interior, Bureau of Mines,
Information Circular No. 8333, May 1967.

































•==^11:










1
1
1
1
1
t
7
*
»
29
21
22
23
24
2S
26
27
3»
2*


.0































IS































30































4§































Utt
1 30
31
32
31
34
35
36
31
39
39
40
41
«2
43
44
4«
47
4»
49
M
51
62
S3
6*
S&
&«
57
&|
59


0































IS































30












— 1

	 1
















4S































O**»rv»l>a« 4*1* "v





n.i.
Tim* , _



m;~t Al.mft^» iu-ii , , , ,.„_
Vl^>p*W., . 	 »












Sum of no*. f«CO*rf*4
* Tt>t*l no. t ***'»• ft


... •

                                      Figure 9-1. Field data.

                                 [FR Doc.71-13624 Filed 12-22-71:8:45 am)
                     FEDEHAL REGISTER, VOL 36, NO.  247—THURSDAY,  DECEMBER  23, 1971
                                                        39

-------
A.2.     Federal Register, Vol 39, No. 177-Wednesday, September 11, 1974
                                                 PROPOSED RULES
                                                                                                                32857
    (e)  For the purpose of reports pursu-
  ant to § 60.7 adju.i»d

  where:

   OF=dally conversion factor (mg frs/dscm
         per ppm). and
  %Oj = percent oxygen by volume (expressed
         as percent), dry basis.

      *       •       •       t       •

  Subpart L—Standards  of Performance for
         Secondary Lead Smelters

  §60.122   [Amended]

    16. Section  60.122 is amended by de-
  leting paragraph (c).

  Subpart M—Standards of Performance for
    Secondary Brass and Bronze Ingot Pro-
    duction Plants

  §60.132   r Amended)

    17. Section  60.132 is amended by de-
 leting paragraph  (c).

 Subpart O—Standards of Performance for
         Sewage Treatment Plants

 §60.152   [Amended]

   18. Paragraph   
-------
32858

In that portion of the plume where condensed
water vapor is not present. The observer shall
not look continuously at the plume, but in-
stead shall observe the plume momentarily at
15-second intervals.
  2.3.1 Attached steam plumes. When con-
densed  water vapor  is present  within  the
ulume as It emerges from the emission outlet,
opacity observations shall be made beyond
the point in  the plume at which condensed
water vapor is no longer visible. The observer
shall record  the approximate distance from
the  emission  outlet to the  point  at  which
the observations are made.
   2.3.2 Detached steam plume.  When water
vapor  in the  plume condenses at a distinct
distance from the emission outlet, the opacity
of emissions may be evaluated at the emission
outlet prior  to the  condensation  of  water
vapor and the formation of the steam plume.
   2.4 Recording observations. Opacity obser-
vations shall be recorded to the  nearest  5
percent at 15-second  intervals on  an  obser-
vational  record  sheet (Figure  »-2).  Each
momentary observation recorded shall repre-
sent the  opacity  of emissions  for that  15-
second time period.
   2.5 Data reduction. Determine the opacity
standard applicable  to the  affected facility
being observed and compute minutes of non-
                                                       PROPOSED RULES

                                           erator. Plumes within each set of 26 black
                                           and 25 white runs shall be presented In ran-
                                           dom order. The candidate assigns an opacity
                                           value to each plume and records his observa-
                                           tion on a suitable form. At the completion of
                                           each run of 50  readings, the score of  the
                                           trainee Is determined. If a trainee  falls to
                                           qualify, the complete run of 60 readings must
                                           be repeated In any retest. The smoke test may
                                           be administered  as part of a smoke school
                                           or training program,  and may be preceded by
                                           training or familiarization runs of the smoke
                                           generator during which candidates are shown
                                           black and white plumes of known opacity.
                                              3.3  Smoke  generator  specifications.  Any
                                           smoke  generator  used for the purposes of
                                           paragraph 3.2 shall be equipped with a smoke
                                           meter Installed to measure opacity across the
                                           diameter of the smoke generator stack. The
                                           smoke  meter output shall  display  in-stack
                                           opacity  based  :ipon  a pathlength  equal to
                                           the stack exit diameter, on  a full 0 to  100
                                           percent chart recorder scale. The smoke meter
                                           optical  design and performance  shall meet
                                            the specifications  shown In  Table  9-1.  The
                                            smoke meter shall  be calibrated as prescribed
                                            In paragraph  33.1 prior to  the  conduct of
                                            each smoke reading test. At the completion of
                                            each test, the zero  and span drift  shall be
                                            ch :cked and If the drift exceeds ±1 percent
                                             soclated  electronic circuitry  Including  the
                                             chart recorder or output meter, or every 6
                                             months, whichever occurs first.

                                                 TABIJE 9-1 BMOKM lOTTEX DESIGN AND
                                                    FXBrOKlOANCX SPECIFICATIONS
                                             Parameter:
                                               a. Light source	
                                               b. Spectral response
                                                    of photocell.
                                               c. Angle of view	

                                               d. Angle  of  projec-
                                                    tion.
                                               e. Calibration error-

                                                              span
   Specification
Incandescent  lamp
  operated at nomi-
  nal rated voltage.
Photopic   (Daylight
  spectral  response
  of the human eye:
  reference 4-3) .
16
comoiiance using the appropriate procedure  opacity, the condition shall be corrected prior
as follows-                                  **> conducting any subsequent test runs. The
  2 5 1 For all opacity standards e»ept aver-  smoke meter shall be demonstrated, at the
jure ooacitv  mark on the observation record  time  of  Installation, to meet the  speclncft-
sheet  (Fleure  9-2)  all  observation* which  tlona listed. In Table »-l. This demonstration
exceed  the  opacity level specified by  the  shall  be repeated following any subsequent
standard  Compute the aggregate number of  repair or replacement of the photocell or as-
OL,(tuu«uu.  vi*."I**."—      DO  D                	J_i	,        .- • -  ,.4w»il+w  in»*1,iH4«f» +Vi&
minutes during which  the opacity of emis-
sions was observed to exceed the applicable
standard.  Exclude from this computation the
number of minutes of  operation exempted
by the applicable  opacity standard or  by
§60.11(c)  (If any). Record  the minutes of
noncompllance on Figure 9-1.
   2.5.2  For  average opacity standards, sum
on the observation record sheet (Figure 9-3)
all observations for any specific time period
(use time periods specified by the standard
when  applicable),  and  compute  the  arith-
metic  mean average bv dividing by the total
number of observations lui  ihe time period
evaluated. Exclude  from  tills  computation
any minutes  of  opefivtion exempted by the
applicable opacity standard or  S«O.ll(c)  (If
any). Record the miuutes of each time period
of noncompllance on Figure 9-1.
   3. Qualifications and teattttf.
   3.1  Certification requirements.  To receive
certification as a qualified observer, a candi-
date must  be tested and demonstrate the
ability to assign opacity reading* In 5-percent
increments to 25 different black plumes and
25 different white plumes, with sn error not
to «xo»ed 15-percent opacity on any one read-
Ing and an average error not to exceed 7.5-
percent opacity in each category. Candidate*
shan  be  tested according to the procedures
described In paragraph 8-2. Smoke generators
used  pursuant  to  paragraph  3.2  shall  be
equipped with a smoke meter which meet*
 the requirements of paragraph 33.
   The certification shall be valid for a period
 of 6 months, at which time the qualification
 procedure must be repeated by any observer
 In order to retain certification.
   8.2  Certification procedure. The (moke test
 consists of showing th« candidate a complete
 run of 60 plumes—26 black plumes and 25
 white plumes—generated by  a smoke gen-
15* maximum total
  angle.
±8*  opacity maxl-
±1*   opacity,
  minutes.
                 30
                                               t. Zero  and
                                                    drtrt.
                                               f. Response ttme.
                                                3.3.1 Calibration. The smoke meter is cali-
                                              brated after allowing t. minimum of SO min-
                                              utes warmup by alternately producing stara-
                                              lated  opacity of 0 percent and 100 percent.
                                              When stable response at  0  percent or  100
                                              percent is noted, the smoke meter  !•  ad-
                                              justed to produce an output of 0 percent or
                                              100 percent, as appropriate. This calibration
                                              shall be repeated until stable 0 percent and
                                              1OO percent readings an produced without
                                              adjustment.  Simulated  0  percent and  100
                                              percent opacity values may be produced by
                                              alternately switching the power to the light
                                              source on and off while the smoke generator
                                              Is not producing smoke.
                                                8.8.2 Smoke meter evaluation-  The smoke
                                              meter design and perfonnanoe are  to  be
                                              evaluated tut follows:
  S.3.2.1  Light source.  Verify  from  manu-
facturer's data and from  voltage measure-
ments made  at the lamp,  as Installed, that
the lamp Is operated within ±B percent of
the nominal rated voltage.
  8.3.2.2Spectral response of photocell. Verify
from manufacturer's data that the photocell
has a photoplc response;  I.e., the  spectral
sensitivity of the cell  shall closely approxi-
mate  the standard spectral-luminosity curve
for photoplc vision which Is referenced in
 (b) of Table 9-1.
  3.3.2.3  Angle of view. Check construction
geometry to  ensure that the total angle of
view  of  the  smoke plume, as seen  by the
photocell, does not exceed 15°. The total angle
of view may be calculated from: 0 — 2  tan-1
d/2L,  where  0 —total angle of view,  d —the
sum of the photocell diameter 4- the diameter
of the limiting aperture: and L = the distance
from  the photocell to  the limiting aperture.
The limiting aperture Is  the point In the
path  between the  photocell and the smoke
plume where  the angle  of view  is  most
restricted. In smoke generator smoke meters
 this Is normally an orifice plate.
   3.3.2.4 Angle of projection. Check construc-
 tion geometry to ensure that the total angle
 of  projection  of the  lamp on   the  smoke
 plume does not exceed 16*. The  total  angle
 of  projection may be  calculated  from:  »=2
tan-1 d/2L, when »=total range of projection;
 d—the sum of the length of the lamp fila-
 ment + the diameter of  the limiting  aper-
 ture;  and L=the distance from the lamp to
 the limiting  aperture.
   3.1.2.5  Calibration  error. Using  neutral-
 density  filters of known opacity, check the
 error  between the actual response and the
 theoretical  linear response  of   the  smoke
 meter. This check Is accomplished  by first
 calibrating the  smoke meter according  to
 3.3.1  and then  Inserting a series  of three
 neutral-density filters  of nominal opacity of
 2O. SO, and  75 percent In the smoke meter
 patblength.  Filters  calibrated  within  ±2
 percent shall be used. Gsre should be taken
 when Inserting  the filters to prevent stray
 light from affecting the meter. Make a total
 of five noneonseeutlye readings for each filter.
 The  maximum  error  on any  one  reading
 Bhsfl be 8 percent opacity.
   3.3.2 8 Zero and span drift. Determine the
 zero and span drift by calibrating and oper-
 ating the smfAft generator in a normal man-
 ner  over a 1-hour   period.  The   drift  is
 measured by checking the  eero and  span
 at the end  of this period.
    3.3.2.7 Besponse time.  Determine the re-
 sponse  time by producing a series of five
 simulated O percent and 100 percent opacity
 values  and observing Uw time  required  to
 reach stable response. Opacity  values of 0
 percent and 100 percent may be  simulated by
 alternately switching  the power to the light
 source off and on while the smoke generator
 is not operating.
    4. References.
    4.1 Air Pollution Control District Ruiei and
 Regulations. Los Angeles County Air  Pollu-
 tion Control District. Regulations  IV, Pro-
 hibitions. Bule SO.
    4.2 Welsburd. Kelvin I, Field Operations
 and  Enforcement Manual for Atr, US. En-
 vironmental  Protection  Agency,  Besetrch
 Triangle Park. N.C, APTD-UOO. August 1972,
 pp. 4.1-4.36.
    43 Condon, K. U, and  Odtahaw, E., Hand-
  book of Phytia*. McGraw-Hill Co, New Tort,
  N.T, 1868, Table 8.1. p. »-62.
                                 FEOCtAl REGISTf*,  VOL S9,  NO. 177—WEDNeSOAY, SETTEMBM 11,  1974
                                                                   41

-------
                                     PROPOSED  RULES
                                                                                                             32859
                                          Figure 9-1
                             UCOU) or vaatL DBTERHINATIOII OF OPACITY
                        CCMftRT
                        tOCATiaT
                                            HOTKS OF CBSntVATION
                                            OKBtm
   MM acmrr	
   COBOL WVICB
                                                                        cnnnciTioi D
                                                                OBSBRVZ2 APFILIATIOH
                                                                FODfT OF EMISSIONS
                                                                BEICBT OF DISCHARGE FOINT
     Record th« folloving Infonutlon prior to and opoa ecu
                                                                 FAtloot at each source.
                                                                 ECordiogs should b« mada
  CDSOITQI LOCATIOH
    BlltBncc to Discharg*
    Direction ft cm DUch.rjs
    Itollht of Obiervatloa Poin

  BiCCOLOtM) DBSCUFnCM

  •KiTBB connnons
    VlDd Direction
    Vlnd 3p«e
-------
32860
                                                       PROPOSED  RULES
                    FIGURE  9-2  OBSERVATION RECORD
                               (Cont.)
                      PAGt
OF
  COMPANY 	
  LOCATION
  TEST NUMBElT
  DATE
OBSERVER 	
TYPE FACILITY
POINT OF EMISSlW
Hr.






























Min.
30
31
32
33
3 it
35
36
37
38
39
1*0
1»1
U2
"«3
>|l»
i»5
*9
50
51
52
53
5>t
55
56
57
58
59
Seconds
0






























Ib






























30






























45






























STEAM PLUME
(check if applicable)
Attached

.



























-
Detached






























COMMENTS





























-
  20. Appendix B is added as follows:

  APPENDIX B—PZHFOBMANCI SPECIFICATIONS

PEKFOBMANCE SPECIFICATION  1	PERFORMANCE
  SPECIFICATIONS AND SPECIFICATION TEST FBO-
  CKDUUS FOB TaANSMISSOMETES SYSTEMS FOB
  CONTINUOI7S MEASUREMENT OF THE  OPACITY
  OF STACK EFFLUENTS

  1.  Principle and Applicability.
  1.1  Principle. TransmlaBometry Is a direct
measurement of attenuation of visible radia-
tion  (opacity)  by particular matter In a
stack effluent. Light from a lamp Is projected
across the stack of a pollution  source to a
light sensor. The light Is attenuated due  to
absorption  and  scatter by the particulate
matter In the effluent. The percentage of light
attenuated is  denned as the opacity of the
emission. Transparent  stack emissions that
do not attenuate light  will have a traremit-
tance of 100 or an opacity of 0. Opaque stack
emissions that attenuate all of the light will
hare a transmlttance of 0 or an opacity  of
100 percent.
  : .2  Applicability. This method is applicable
to the instrument systems specified in the
subparts  for   continuously  monitoring the
opacity of emissions. Specifications for con«
t.r.uous measurement of visible emissions are
given in terms of  design, performance, and
 installation parameters. Performance specifi-
cs: ;o as and  test  procedures are  given  for
transmissometer systems to test their capa-
 bility before approving the systems Installed
 by an affected facility.
   2. Apparatus.
   2.1 Neutral Density Filters. Filters  with
 neutral spectral  characteristics and known
 optical  densities to visible light. One  each
 low, mid, and high range filters, 5- or 6-lnch
 square or 6-lnch diameter, with nominal op-
 tical densities of 0.1, 0.2, and 0.3 (20, 37, and
 SO percent opacity) are  required.  Calibrated
 niters with accuracies certified by the manu-
 facturer to within 3 percent shall be used. It
 is recommended  that filter  calibrations be
 checked with   a  well-collimated photopic
 transmissometer of known linearity prior to
 use.
   2.2 Chart Recorder.  Analog chart  recorder
 with Input voltage range and  performance
 characteristics compatible with ttie measure-
 ment system output.
   2.3 Opacity Measurement  System. An In-
 stack transmissometer (folded or single path)
 with the optical  design  specifications desig-
 nated below, associated- control units and ap-
 paratus to keep optical surfaces clean.
   3. Definition*.
   3.1 Measurement System. The total equip-
 ment required for the continuous determi-
 nation  of  a  pollutant  concentration  In  a
 source effluent. The system consists  of  three
 major  subsystems:
   3.1.1  Sampling Interface—That  portion of
 the  measurement system that performs one
 or more of the following operations:  delinea-
tion, acquisition, transportation, and condi-
tioning of a sample of the source effluent, or
protection of the analyzer from the effluent.
  3.1.2 Analyzer—That  portion of  the sys-
tem which senses the pollutant and generates
a signal  output  that is a  function  of  the
pollutant concentration.
  3.1.3 Data Recorder—That portion of  the
measurement system that processes the ana-
lyzer output and provides a permanent rec-
ord of the output signal in terms of pollutant
concentration.
  3.2  Span. The  value of opacity at which
the measurement system is set to  produce
the maximum data display  output.  For  the
purpose of this method,  the  span shall be set
at an opacity of 50 percent  for  a pathlength
equal to the stack exit diameter of the source.
  3.3  Calibration Error.  The difference  be-
tween  the opacity reading indicated  by  the
measurement system and the known values
of a series of test standards. For this method
the test standards are a series  of calibrated
neutral density niters.
  3.4 Zero Drift. The change in measurement
system output  over a stated period of time
of normal  continuous operation when  the
pollutant concentration  at  the time of  the
measurements is zero.
  3.5 Calibration  Drift. The  change in meas-
urement  system output over a  stated period
of  time  of  normal  continuous  operation
when the pollutant concentration at the time
of the  measurements is the  same known  up-
scale value.
  3.6  System  Response.  The  time  interval
from a step change In opacity In  the stack
at the input to the measurement system to
the time at which 95  percent of  the cor-
responding final value is reached as displayed
on  the measurement system  data  presen-
tation  device.
  3.7 Operational Test Period.  A minimum
period of time over which a measurement
system is expected to  operate within cer-
tain performance specifications without un-
scheduled maintenance, repair, or  adjust-
ment.
  3.8  Transmittance. The  fraction  of  in-
cident light that is transmitted through an
optical medium of Interest.
  3.9 Opacity. The fraction  of incident light
that Is attenuated by  an  optical  medium
of Interest  Opacity  (O) and transmittance
(T) are related as follows:

                 O=1-T

  3.10  Optical Density. A logarithmic meas-
ure of the amount of light that Is  attenu-
ated by an optical medium of  Interest Op-
tical density  (D) is  related to the trans-
mittance and opacity as follows:

  D=  -log,8T
  D=  -log,. (1-0)
  3.11  Mean Spectral Response The wave-
length which  bisects the total area under
the curve obtained pursuant to paragraph
9.2.1.
  3.12  Angle of View. The maximum  (total)
angle of radiation that Is seen by the photo-
detector  assembly of an optical  tranamls-
someter.
  3.13  Angle of  Projection. The maximum
(total) angle of radiation that is projected
by  the lamp assembly of an optical trans-
missometer.
  3.1*  Pathlength. The  depth of effluent In
the light beam between the receiver and the
transmitter of  the  single-pass  transmis-
someter. or the depth of effluent between the
transceiver  and  reflector of a double-pass
transmissometer.
  4. Installation Specif cation.
  4.1   Location. The  transmissometer must
be located  across a section  of duct or stack
that will provide a particulate matter flow
through  the optical  volume of  the trans-
                               FEDHAL REGISTER,  VOL  39,  NO. 177—WEDNESDAY, SEPTEMBEt 11, 1974
                                                                43

-------

-------
APPENDIX B. STATISTICAL METHODS
                 45

-------
                         APPENDIX B.  STATISTICAL METHODS
B.1   Linear Regression and Test for Significance

      In order to obtain the best estimate of the average deviation from the metered opacity, the relationship
between the deviation and the meter average is investigated.  To investigate this, the method of least-squares
is used to give the equation of the best straight line that fits the data obtained.  The line is defined by its
slope and intercept, estimated by:
and

                                            a = y  - bx

where

      Xi   -    the Ith value of the independent variable

      Yj   -    the /th value of the dependent variable

      x , y -    the sample means of X and Y , respectively

      n    -    the number of paired observations.

      To determine if the line is a good estimate  of the relationship between the two variables, the slope of
 the line is tested. A linear relationship exists if the slope is non-zero.  To test this, a statistic is used that is
 based on Student's t distribution. The statistic is
 where

      tc    -     the calculated value

 and

      sb    -     the standard deviation of the slope, b. This statistic has n - 2 df associated with it for
                  n pairs of observations.

       The standard deviation of the slope, sb , is calculated by first calculating the standard deviation around
 the regression line, sy.x . The equation for the variance, S*y.x is:

                                            H-2
                                                    46

-------
The variance of the slope, b, is then given by
                                        4-
                                                iS y i
     Taking the ratio of the slope to its standard deviation, gives the desired statistic. To determine if this
is significant, the value is compared to a Student's t distribution with n - 2 df.

B.2 Precision Estimation for Training Generator Test

     1.    White Smoke

           An analysis of variance was performed on the average opacities determined by the 8 collaborators
who completed all 20 runs.  The model is a random effects model represented by
where

     y,-y   -    the determination by observer; in run/

     p     —    overall mean

     Pi    -    the effect of run i

     7,    -    the effect of collaborator;



           -    the random error associated with Yn.
and
This is an additive or no-interaction model. This means that the collaborator and run effects are assumed to
be independent of each other, or that an observer reads consistently low or high with respect to the other
observers regardless of the opacity level.
 TABLE Bl.  ANALYSIS OF VARIANCE FROM TRAINING
          GENERATOR TEST, WHITE SMOKE
Source
Runs
Observers
Error
Total
df
19
7
133
159
SS
5545.06
165.04
754 .51
6464.61
MS
_*
23.58
5.67
F
_*
4.16f
EMS
_*
CT2 + 20 a}
„' •*•
*Not of interest.
tSignificant. FC > FQ 05(7,133) * 2.09.
                                                                The ANOVA results are shown in
                                                     Table Bl. The expected mean squares indicate
                                                     the proper F-ratios and the means of obtaining
                                                     estimates of the desired precision components.
                                                     The EMS of the error term is a1, the within-
                                                     observer variance. Thus, the error mean square
                                                     is the estimated within-observer variance, and
                                                     this gives

                                                                       a2 = 5.67

                                                     with 133 df. This gives an estimated within-
                                                     observer standard deviation of
                                      = 2.38 percent opacity.
                                                  47

-------
           The significance of the observer term is determined by comparing the observer mean square to the
 error mean square. The F-ratio is Fc = 4.16 which exceeds the critical value, F0.0s (7, 133)^2.09. Thus
 there are differences among the observers, and an observer bias variance can be estimated .

           The EMS of the observer term is a2  + 20 a£, so that the observer bias variance estimate is

                                         -2  =MSL-&2
                                         °L      20

                                             23.58-5.67
                                                  20

                                            _ 17.91
                                               20

                                            = 0.90

with 7 df. The estimated observer bias standard deviation is
                                       = 0.95 percent opacity.
      2.    Black Smoke
           The ANOVA table for the black smoke determinations is in Table B2. The model is identical to
that for the white smoke.
 TABLE B2.  ANALYSIS OF VARIANCE FROM TRAINING
          GENERATOR TEST, BLACK SMOKE
Source
Runs
Observers
Error
Total
df
15
7
105
127
SS
3963.43
135.58
354.81
4453.82
MS
_*
19.37
3.38
F
_*
5.73t
EMS
_*
a7 + 16 a)
a* L
*Not of interest.
tSignificant. F005(7,105) » 2.12.
           The expected value of the error
mean square is a2, the within-observer variance.
Thus, the estimated value is

                 &2=3.38,

the error mean square. There are 105 df associ-
ated with this estimate. The within-observer
standard deviation is
                                                                 = 1.84 percent opacity.

           The ratio of the observer mean square to the error mean square tests for differences among the
observers. The calculated F-statistic is Fc - 5.73, which exceeds the critical value of 2.13, approximately,
with 7 and 105 df.  Thus, there can be said to be differences among the observers.

           The observer bias variance is estimated as

                                            _MSL -a2
                                          L      16
                                                 48

-------
as indicated by the EMS for observers. This gives

                                             19.37-3.38
                                        51 =
                                                  16

                                             15.99
                                               16

                                            = 1.00

with 7 df. Thus the observer bias standard deviation is

                                     aL  = 1.00 percent opacity.

B.3  Precision Estimation for Sulf uric Acid Plant and Tests of Hypotheses

      The precision estimates for use of the method at a sulfuric acid plant are obtained from an analysis of
variance. The model for the data is a random effects model,

                                       10 = M + It + Pj + «(/

where

      Y--   -    the observation by observer / in run /

      p    —    the overall mean

      ~fj   -    the effect due to collaborator i

      p-   -    the effect due to run ;'

 and

      e«    -    the random error associated with YIJ.

      The model assumes that there is no interaction between the run and the collaborator. This is equivalent
 to saying that an observer's tendency to read higher or lower than other observers is independent of the true
 opacity level. The error term, then, is taken from the interaction between the observers and the runs.

       The ANOVA is summarized in Table B3.  There are 29 df from the  30 runs and 8 df for the observer
 term This leaves a remainder of 232 for error.  The expected mean square (EMS) column gives the basis for
 forming F-ratios to test for significance  and for estimating the individual variance components.

       The EMS of the error term is o2 , the within-observer variance term. Thus, the estimated within-observer
 variance is

                                             a2 =4.51,

  the error mean square. This gives an estimated standard deviation  of
                                              = 2.12 percent opacity
                                                    49

-------
     TABLE B3.  ANALYSIS OF VARIANCE FROM
           SULFURIC ACID PLANT TEST
Source
Runs
Observers
Error
Total
df
29
8
232
269
SS
2636.68
259.06
1046.56
3942.30
MS
90.92
32.38
4.51

F
__*
7.18J


EMS
_*
oa + 10o}
a2

*Not of interest.
f Significant. FO.OS (8,232) » 2.00.
     Solving the EMS for a^ gives
EMSobs
                   The EMS of the observer term is a2 + 30a£.
             The factor 30 results from the fact that each
             observer mean is the average over the 30 runs.
             The ratio of the observer mean square to the error
             mean square provides the proper ratio for deter-
             mining if there  is  a significant observer term
             The value of Fc is 7.18, which exceeds the tabled
             value for the F distribution at the  95-percent
             confidence level.  The tabled value is.Fo.05 (8,232)
             =* 2.00 and, thus, there is a significant observer
             effect.
                                                     • -a"
                                                30
so that an estimate is given as
                                         _ MSobserver - a2
                                       L=      30      ~

                                        _ 32.38-4.51
                                               30

                                        _27.87
                                           30

                                        = 0.93

with 8 df. The estimated observer bias standard deviation, then, is
                                       = 0.96 percent opacity.
     Combining these two estimates gives
                                           = 0.93+4.51

                                           = 5.44
and
                                           = 2.33 percent opacity.

     The test for significance of a\ is a test for the equality among the observers of the mean opacity
determined. A significant value of 0£ is equivalent to saying that not all the means are equal.  The next
problem is to investigate where the differences lie.
                                                 50

-------
     For this test, the only comparison of interest concerns the difference between the observers from the
enforcement agency and those from the private sector.  The enforcement personnel were observers 4, 5, 6 and
The means for the 9 collaborators who participated in all 30 runs are shown in Table 16.

     To determine if there is a significant difference between the two groups, a contrast among the means is
tested. A contrast is of the form
where

     Hi    -    the jth true mean

     Cj    -    the coefficient of the ith mean

and
 The null hypothesis to be tested is that there is no difference between the two groups of observers.  The
 contrast that tests this hypothesis is written as
                                              Mi) -

 where 5 and 4 are chosen so that the sum of the coefficients is 0.

      The test statistic follows Student's t distribution and is defined by
 where

       &    -    the standard deviation for error

      jc /   -    the /th sample mean

 and

       n    -    the number of determinations used to obtain* £-

 The degrees of freedom for this statistic are the df associated with the estimated variance.

       For this test,

                              jX, = 5(6.47 + .. . + 5.40) - 4(5.00 +... + 5.73)

                                 = 29.54.

                                         o=2.12, from Table B3
                                                      51

-------
                                    2c? = 4(52) + 5(42) = 180

                                               n = 30

 so that
                                                 29.54
 A t value at the 0.05 level with 232 df is essentially a standard normal value or 1.96. Since tc exceeds
 1.96, we reject the null hypothesis, and conclude that the observers from the enforcement agency read
 a higher opacity  than did the observers from the private sector.

 B.4  Analysis of Variance and Tests of Hypotheses from Riverbend Steam Station Test 1

      The determinations from the first test are used to obtain precision estimates for within- and between-
 observer and observer bias components. The determinations used are taken from runs 1, 2, 3, 7, 8 and 9.
 Runs 4, 5, and 6 are eliminated since the experimental group was reading in 1-percent increments. Run 10
 was eliminated since  there were no determinations from 6 of the 10 observers, including the entire control
 group.  This was due  to the fact that sky conditions and interference from another plume at times prevented
 the observer from making a reading.

      The model for  the AMOVA is represented by
a completely random model, where

      Yjjk  -    the kth observation by collaborator /in group i

      p.    —    the overall mean

      (3,-    —    the effect due to group /

      tj/i  —    the effect due to observer/, nested within group/'

      pk   -    the effect due to run k

and

      ei/k  ~    the random error associated with Yjj/f

The observers are treated as a nested factor since the observers differ from one group to another.


                                                 52

-------
        TABLE B4. ANALYSIS OF VARIANCE FROM
                STEAM STATION TEST 1
Source
Groups
Runs
Observers/Group
Error
Total
df
1
5
8
45
59
SS
124.42
662.03
119.05
149.10
1054.60
MS
124.42
-t
14.88
3.31

f
8.36*
-t
4.50J


EMS
-t
-t
a2 + 6o*L


*Significant. FQ.QS d.8) = 5.32.
•(•Not of interest.
^Significant. F0.Q5 (8,45) « 4.08.
                  The ANOVA table is shown in Table B4.
            The expected mean squares (EMS) determine
            the proper F-ratio for each factor. The group
            mean square is divided by the observer mean
            square to test for a difference between groups.
            The F-ratio is Fc = 8.36 with 1 and 8 degrees
            of freedom. The critical value from the
            F-distribution isFo.os (1,8) = 5.32, so that
            the group term  is significant.  The  means
            for the two groups  are 8.20 and 5.32
            for the control  and  experimental,
            respectively, so  that  the control group
            was reading significantly higher opacities
            during the test.  Since the determinations
 that  the control group was  able to judge the opacity
were generally biased low, however, this indicates
more accurately than the experimental.

     The EMS of the error  term is a1 , the within-observer component.  This gives an estimated within-
observer variance of

                                               a2 = 3.31

with 45  df.  The estimated within-laboratory standard deviation,  then  is
                                            0 =
                                              = 1.82 percent opacity

      To determine if there is a significant observer effect, the  observer mean square is compared to
 the error mean  square.  The  ratio is Fc = 4.50.  The critical value for this test is F0-05 (8,45) « 4.08,
 and the observer bias term, Q\ , Js determined to be greater than zero.

      The observer bias is estimated from the observer group mean square. The EMS is a2  + oj,, so the
 estimated variance is

                                            MSobserver ~ *
                                         _ 14.88-3.31
                                                6

                                         = 1.93

 with 8 df.  The df are calculated as (5-1) df for each group.

      Since there are differences among the observers, the experience levels of the observers are important
 considerations.  The test is based on the means of the  collaborators, and a contrast among the means is
 tested.  The hypothesis is
HO •
= 0
                                                                  0
 where c/ is the coefficient of the ith mean, chosen so that
                                                    53

-------
     Since there are 4 experienced observers and 6 inexperienced observers, the c/ are chosen as
                   HO- 6(Mi  + ^2  +M6 +M7)-4(M3 + M4
is the hypothesis tested.  The test statistic is based on Student's t distribution, and given by
                                             o
                                                  «
where n is the number of determinations comprising each mean, in this case 6. Substituting gives



                                                ,)- 4 (* 3 + ...+*io)
                                     /62 + . . . + 6* + 42 + . . . + 42
                                L82y_ - . -


                                         6(28.65) - 4 (38.97)
                                                  144 + 96
                                            1.82
                                                      6


                                         171.90-155.88

                                            1.82 v/40


                                         16.02
                                       =	=1.39
                                         11.50


This statistic has 45 df associated with it. These df came from the estimated variance, a, used in the calculation.
A f-value with 45 df must exceed 2.016, approximately, to be considered significant at the 0.05 level. Thus,

no difference is determined between the experienced and the inexperienced observer for these data.


B.5  Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 2


      The deviations shown in Table 23 are submitted to an ANOVA procedure to estimate the precision of

the method. The model selected is a completely random model represented by


                                    *ijk = M + ft + Ijli + Pk + etfk

where


      Yijk  -    the observed value in run k by observer / in group i


      p.    —    the overall mean


      (3,-    -    the effect due to group i


      "fj/j  -     the effect due to collaborator / in group i


      pk   -    the effect due to run k


 and

      eijk  -    the random error associated  with y,yjt
                                                  54

-------
      The observer term is nested within groups, since a different set of observers comprised each group. The
run term is crossed with both the group and the observer terms. The error term is estimated from the inter-
action of the run and group, and of the run and observer, with the interactions assumed to be zero. The
resulting ANOVA table is in Table B5.

      The EMS column tells which mean squares provide the proper denominator for the F-ratio, and how
the resulting variance components may be estimated.

      The within-observer variance is estimated from the error term. Since the EMS is a2, the estimated variance
 is
                                             o2 = 3.37,

 the mean square for error. There are 88 df for this estimate. The standard deviation is estimated as
                                              = 1.84 percent opacity.
       TABLE B5. ANALYSIS OF VARIANCE FROM STEAM
                      STATION TEST 2
Source
Groups
Runs
Observers/Groups
Error
Total
df
1
10
8
88
107
SS
156.77
1866.06
225.71
296.21
2544.75
MS
156.77
186.61
28.21
3.37

F
5.56*
-t
8.37*


EMS
-t
-t
a2 + lla£


*Significant. FQ.QS (9,8) = 5.32.
t Not of interest.
^Significant. F0.05 (8,88) ~ 2.10.
The observer bias term is estimated from
the observer term in the ANOVA. The
denominator for the observer F-ratio is
the error term, and the test statistic is
Fc = 8.37 with 8 and 88 df. The tabled
F value at the 5-percent level is approxi-
mately 2.10 and the observer term is
determined to be significant. The expected
value of the mean square is a2 4 11 a\.
Thus, we estimate the variance by
                    -a2
                   11
where MS/, is the observer/group mean square. Substituting gives

                                          .   28.21-3.37
                                           _ 24.84
                                               11

                                           = 2.26

with 8 df. The degrees of freedom are calculated asn-l=5-l=4 from each group. The estimated
observer bias standard deviation is
                                                = 1 .50 percent opacity.

      The F-ratio for groups is the ratio of the group mean square to the observer/group mean square. The
statistic is Fc = 5.56. To determine if this value is significant, Fc is compared to a tabled F value at the 5-per-
cent significance level. The tabled value is Frj.osO .8) = 5.32, so that a significant difference can be said
to exist between the two groups in the determination of average opacity. This is an indication that the angle
of view is an important consideration in the visual estimation of opacity.
                                                  55

-------
      Since it has been determined that there are differences among the observers, the hypothesis that the
experienced observers differ from the inexperienced can be tested. This is done by use of a contrast among
the means. A contrast is represented by 2 c,-/i,-, where the Cj are coefficients applied to the means, constructed
so that 2c, = 0.  For this test, the null hypothesis is stated as

                   H0. 6(^1 +JU

where

      M,-    -     the mean for observer i.

      Observers 1,2,6 and 7 are experienced, and 3,4, 5, 8, 9 and 10 are relatively inexperienced. The test
statistic is given by
                                                  n

where

      x j   -    the sample mean for observer /

      a    —    the estimate of the within-observer standard deviation

and

      n    —    the number of determinations that comprise Xj.

      For these data, the statistic is:
                                      _ (6) (29.60) +(-4) (3 7.20)
                                    C
                                      _ 177.60-148.80
                                           1.84-V/24

                                      _ 28.80
                                        9.02

                                      = 3.19

There are 88 df for the statistic taken from the estimated variance. By comparing this statistic to a tabled t
with 88 df, tc = 3.19 exceeds /o.OsC88) *= 1 •" and thus a significant difference does exist. The conclusion
is that the experienced observers did determine a higher average opacity than the  inexperienced.

B.6 Precision Estimation and Tests of Hypotheses for Riverbend Steam Station Test 3

     The determinations of average opacity made by the collaborators as shown  in Table 27 are  submitted
to an ANOVA procedure to estimate the  precision components. The model for these data is a completely
random  model,
                                                56

-------
                                  tjk =H+Pt+ tjli + Pk + Pfiik + etjk
where
      Yjjk  -    the observation on run k by observer / in group i

      p.    -    the overall mean

      fa    -    the effect of the ith group

      yj/j  -    the effect of the /th collaborator in group

      pk   -    the effect due to run k

      p0ik  -    the interaction between run k and group i
and
      £ijk  -
                 the random error associated with
      The two groups are considered together with respect to precision under the assumption that a more
direct sun angle would not affect the precision of the individual observer.  To test this further, separate esti-
mates of the error variance are made using the run by observer interaction within each group. The estimated
values of a2 are
                                            6* = 3.49

                                            4 = 4.09

for the control and experimental groups, respectively. To test whether the experimental estimate is signifi-
cantly larger, an F-ratio is used.  The calcualted statistic is

                                                 4.09
                                             F  = ,. ,-. -
                                              °  3.49
 with 45 and 45 df. The critical value is F0.0s (45,45) « 1.69, and the two values may be said to be estimat-
 ing the same true value.
      The ANOVA is summarized in Table B6 for the third test. The evaluation is made using the 16 runs
 where the experimental group read in 5-percent increments. The EMS column provides the basis for testing
 for significant effects and estimating the variance components.

                    TABLE B6.  ANALYSIS OF VARIANCE FROM STEAM STATION TEST 3
Source
Groups
Runs
Group X Run
Observer/Group
Error
Total
df
1
15
15
6
90
127
SS
2.88
9440.54
138.39
321.05
313.73
10216.59
MS
2.88
629.37
9.23
53.51
3.57
698.56
F
0.31*
_•}-
-t
14.99 J


EMS
oj + 16a£ + 32ajjG + 64ac
-t
aj + 16o£ + 32
-------
      The error term comes from the observer-run interaction.  The assumption made is that this effect
is equal to zero, which is equivalent to saying that if a particular observer reads higher opacities than the
others on one run, he would not read lower on other runs. The EMS of the error term is a2 , which gives
an estimated within-observer variance of

                                               a2  =3.57

with 90 df.  This results in a within-observer standard deviation estimate of

                                               & = \/3~37

                                                = 1.89.

      The EMS for observers/group is a2 + 16a£ , since each observer mean is the average over the 16 runs.
The F-ratio is the ratio of the observer mean square to the error mean square.  This gives Fc = 14.99 with
6 and 90 degrees of freedom. The critical value is Fo.os(6,90) « 2.30. Thus, there is a significant observer
effect at the 5-percent level.

      To estimate the observer variance component, the EMS is again used.
                                                    16

where MS^ is the observer mean square, and a2 is the error mean square. Substituting gives

                                           ,   53.51-3.57
                                         a? = -
                                          L        16

                                            _ 49.94
                                                16

                                            = 3.12.

There are 6 df for laboratories calculated as (4-1) for each gioup. The estimated observer bias standard
deviation is
                                               = 1 .77 percent opacity

      To test whether there is a significant group effect, an F-ratio is constructed for the group term.  The
denominator is the group by run interaction, and the statistic is Fc = 0.31 with 1 and 15 df. The critical
value is F0-OS (1 , 15) = 4.54, and no group effect is found. This indicates that the angle of observation had
essentially no effect in this test.

      The means of the observers over the  16 runs are used to determine if there was a significant difference
in this test between the experineced and the inexperienced observers.  A contrast among the means is used
for the test. The hypothesis is of the form
                                 HO '•  2
-------
and
                                  to is the mean of collaborator i.

For this test, the q are chosen to be 1 for the experienced and -1 for the inexperienced observer means.
The test statistic is given by
where
                        xf is the sample mean for collaborator i
                        a is the estimated within-observer standard deviation

and
                        n is the number of determinations used to obtain AT,-.

     Substituting from Table 31 gives
                            (15.31 + ... + 14.46) -(11.40 + ... + 12.76)
                         te =	—
                                              1.89  /—

                            (58.72)-(46.81)
                              (1.89) (0.71)

                            11.91
                           ~ 1.34
                          = 8.89.
     There are 90 df associated with this statistic, and this gives a critical value of f0.os (90) « 2.00. Thus,
there was a significant difference between the  experienced and inexperienced observers.
B.7  Precision Estimation for Riverbend Steam Station Test 3,1-Percent Increment Runs
      The determinations made in increments of 1 percent are submitted to an ANOVA procedure according
to the completely random model
 where
      YIJ   -    the observation in run j by observer /

      H     —    the overall mean

      Pi    -    the effect due to  run i

      f     —    the effect due to  observer/
                                                  59

-------
and
     fc'/
                 the random error associated with Y,
               i]
      The model assumes that there is no interaction between run and observer.  This assumption is equivalent
to saying thai the observers will be consistently high or low with respect to each other, regardless of the density
of the plume.  By making this assumption, blocking the runs  in order to make an estimate of the within-ohserver
variance is not necessary.
   TABLE B7. ANALYSIS OF VARIANCE FROM
    STI: AM STATION TEST 3, \','i VARIATION
Source
Runs
Observers
Error
Total
df
7
3
21
31
SS
2105.62
45.75
174.07
2325.44
MS
__ *
15.25
8.29

F
_. *
1.84t


EMS
_ *



*Not of interest.
tNot signit
cant. F0()5 (3.21)= 3.07.
                    There were 8 runs by 4 the observers in the control
               group for a total of 32 determinations. The ANOVA
               is summarized for these data in Table B7. The EMS of
               the error term is a2 , the within-observer variance.  Thus,
               the estimated value of o2 is

                                     &2 =8.29

               with 21 df. This gives a within-observer standard
               deviation of
      The EMS of the observer term is a2-I- Sa2. .  Thus, the ratio of the observer term to the error term tests
for significance. The resultant f-ratio is Fc - 1.84. The critical  value for this test is F0 05 (3.21) = 3.07.
and thus, the observer term is not significant. This implies that the between-observer term, a2h. is identical
to o2. and gives
                                            ah = a = 2.88 percent opacity
as before.
      The within-observer term for these runs is higher than that for the previous runs. An F-test is con-
ducted to determine if a significant difference exists between the true variances of the two sets. The F-ratio
is

                                                  8.29
                                                  3.57

                                                = 2.32

with 21 and 90 df. The critical value for this test is F0 05 (21. 90) ~ 1.75. Thus, the hypothesis of equality
is rejected, and we can say that there is more variability reading in 1  percent increments.

      To determine if there was any difference between the performance of the two groups, a second ANOVA
is done in which the control group data is included.  The model is
ijk
                   Pk
                                                                  eljh
where
                 the determination by observer/' of group / in run k

                 the overall mean
                                                   60

-------
     Pk
the effect due to group i

the effect due to observer/' in group i

the effect due to run k

the run by group interaction
and
     e,y£   -    the random error associated with y^

As before, the run by observer interaction is assumed to be zero and used to estimate the error term.

                     TABLE B8.  ANALYSIS OF VARIANCE FROM STEAM STATION
                                        TEST 3,1% RUNS
Source
Groups
Runs
Group X Run
Observer/Group
Error
Total
df
1
7
7
6
42
63
SS
268.96
5549.59
408.16
127.69
476.61
6831.01
MS
268.96
792.80
58.31
21.28
11.35

F
4.61*
-t
-t
-t


EMS
o2 + 8<7/ + Ifi^fcfj + 32<7 G
-t
a2 + Sal + 16er)jG
a' +8o£
o1

*Not significant. Frj.05 d>7) = 5-59-
tNot of interest.
      The ANOVA is summarized in Table B8. The only term of interest is the group term.  The F-ratio is
 formed by dividing the group mean square by the group by run mean square.  The calculated value is Fc = 4.61
 with 1 and 7 df. The critical value is Frj.osO ,7) =5.59, and so no group effect can be determined.

 B.8. Significance of Reduction of Frequency of Large Deviations

      To determine if a significant decrease is noted in the frequency of large deviations, a confidence interval
 is set up around the observed proportion. The confidence interval is constructed using
                                                       -P)
 where
      P
 and
  the sample proportion

  the number of items in the sample
       1.96  -    the standard normal deviate at the 95-percent confidence level.
       The proportion of large deviations (absolute value greater than 7.5) in the sample is obtained by taking
  the number of large deviations, and dividing by the size of the sample. This gives
                                              __ J5_
                                              P  144

                                                = 0.10.

                                                  61

-------
 Substituting this into the above formula gives
                                                         /(o.ioxi - o.io)
                                      CI095: 0.10 ± 1.96V-	—	
                                                               144
                                            : 0.10 ±1.96(0.03)

 or from 0.05 to 0.15

      The frequency of large deviations in the original data set is 15/96 or 0.16. Thus, since this falls outside
 the above confidence interval, the two may be said to represent different true proportions of occurrence. Thus,
 the frequency of large deviations was reduced by averaging two observers' result.

 B.9. Multiple Comparison Tests and Composite Estimation

      The slopes of the three regression lines obtained as representative of observer deviation from plume
 opacity are compared for equality using  the least significant difference (LSD) test. The test statistic provides
 a number which must be exceeded in order for two sample values to be said to represent different true values.

      The standard deviations of the slopes is calculated as in Appendix B.I for each regression line. For the
 white training generator smoke, black training generator smoke and steam station data, these values are 0.04,
 0.05, and 0.04, respectively. Since these are close  to each other, it is reasonable to assume that there is a
 common variance, s^, for all three slopes, and to estimate it by pooling the three estimates. A pooled variance
 is  of the form
                                             1= 1

 where

      dfi   -    the number of degrees of freedom in the i'th sample

      s]    -    the variance of the jth sample

      k    —    the number of samples.

      Pooling the variance gives an estimated value of s* = 0.0019 with 393 df.  The df are obtained by sum-
 ming the df for the individual variance estimates.

      The LSD at the 5-percent significance level is given by
                                                = 1.96V0.0038

                                                = 0.12.

All three slopes can be said to be equivalent if they differ from each other by no more than 0.12. The dif-
ference between the greatest and least values is 0.34 - 0.24 = 0.10, so that all three slopes can be said to be
estimating the same true value.
                                                  62

-------
                                    LIST OF REFERENCES


1     Environmental Protection Agency, "Standards of Performance for New Stationary Sources," Federal Register.
     Vol. 36, No. 247, December 23, 1971, pp. 24876-24893.

2    Environmental Protection Agency, "Stationary Sources, Proposed Emission Monitoring and Perform
     Testing Requirements," Federal Register, Vol. 39, No. 177, September 11, 1974, pp. 32852-32874.

3.    Dixon, W.J. and Massey, F.J., Jr., Introduction to Statistical Analysis, 3rd Edition, McGraw-Hill, New York,
     1969.
                                                    63

-------

-------
  REPORT NO.
       EPA-650/4-75-009
J.TTTL E~ANo SUBTITLE
                                             TECHNICAL REPOBT DATA
                                    (Please read Instructions on the reverse before completing)
                                      2.
                                                                            3 RECIPIENT
     -Evaluation and Collaborative Study of Method for Visual Determination
     of Opacity of Emissions from Stationary Sources."
5. REPORT DATE
    January  1975 (date of approval)
6. PERFORMING ORGANIZATION CODE
~7. AUTHOR(S)
     Henry F. Hamil,Richard E.Thomas and Nollie F.Swynnerton
8. PERFORMING ORGANIZATION REPORT

    SwRl #01-3462-006
9TER~FORM!NG"ORT-:ANIZATION NAME AND ADDRESS

     Southwest Research Institute
     8500 Culebra Road
     San Antonio. Texas 78284
 12. SPONSORING AGENCY NAME AND ADDRESS

     U.S. Environmental Protection Agency
     Office of Research and Development
     Washington. D.C. 20460
10. PROGRAM ELEMENT NO.
    1HA327	
11. CONTRACT/GRANT NO""
    68-02-0626
 13. TYPE OF REPORT AND PERIOD C.OVE.K
    Task Order
 14. SPONSORING AGENCY CODE
 15. SUPPLEMENTARY NOTES
 16. ABSTRACT"
           This study presents the results of statistical analyses of determinations of the average opacity of emissions. The
     determinations are based on 25 readings made by qualified observers at three types of sources. The tests were con-
     ducted 1) using training smoke generators, 2) at a sulfuric acid plant and 3) at a  coal-fired power plant. Observers
     used in the tests came from both enforcement agencies and private companies, and varied in the amount of field
     experience in the use of the method.

           For each test, the accuracy and precision of the determinations is investigated. Accuracy is measured by the
     deviation of the determination from a concurrent meter average, used as true opacity for these tests.  The precision
     of the method is measured as standard deviations for wi thin-observer, observer bias and between-observer terms.
     For each test, the expected range of determinations by a single observer and the maximum difference expected be-
     tween two observers are calculated.

           The results of all the tests are used to estimate the accuracy and precision of the method independent of the
     nature of the site or color of the plume.  These estimates show a reasonable degree of accuracy and precision for
     the method when used in the probable range of opacities that would occur in field use, 0 to 35 percent opacity.
17 KEY WORDS AND DOCUMENT ANALYSIS
a DESCRIPTORS
Emission
Field tests
Statistical analysis
Plume
Opacity
13. DISTRIBUTION STATEMENT
Unlimited
b.lDENTIFIERS/OPEN ENDED TERMS
EPA Method 9
Stationary Sources
Visual method
Smoke generator
Collaborative testing
19. SECURITY CLASS (This Report)
Unclassified
20. SECURITY CLASS (This page/
Unclassified
_;
c. COSATI Field/Group |
I
13B j
14B
70
22. PRICE
  EPA Form 2220-1 (9-73)

-------

-------
                                      ERRATA

                                  EPA-650/4-75-009
           EVALUATION AND COLLABORATIVE STUDY OF METHOD FOR
               VISUAL DETERMINATION OF OPACITY OF EMISSIONS

                                          by

                                     Henry F. Hamil
                                   Richard E. Thomas
                                   Nollie F. Swynnerton

       The following correction is applicable to the above cited report which was issued under Project No.
01-3462-006, dated January, 1975.

                                        Page 32

       Paragraph 2 is deleted in its entirety and the following paragraph is substituted:

       This value is comparable to that for the 5-percent increment data. From a second ANOVA shown
in Append* B.7, the two groups of observers cannot be shown to be reading different average opacmes on
these runs.

-------

-------