-------
For  conditions as assumed in Section IV, page  76 of  the Management Manual,
a study  of Figure 21, page  81, gives the following results.  These conditions
may  or may not apply to your operation.  They  are included here to call
attention to a methodology.  Local  costs must  be used for conditions  to
apply to your operation.

Decision Rule - Accept the  Lot as Good If No Defects Are Found  (i.e.,  d=0)

Most Cost Effective Auditing Level  - In Figure 21, page 81, the two solid
lines are applicable to this decision rule, i.e., d=0.  The cost curve has a
minimum  at n=7 or an auditing level of 7 checks out  of 100 sampling periods.
From the probability curve  it is seen that at  this auditing level there is
a probability of 0.47 of accepting  a lot as good when the lot (for N=100)
actually has 10 defects with an associated average cost of 234 dollars per
lot.

Auditing Level for Low Probability  of Accepting Bad  Data - Increasing the
auditing level to n=20, using the same curves  in Figure 21 as in above,
shows a  probability of 0.09 of accepting a lot as good when the lot actually
has  10 defects.  The average cost associated with this level of auditing
is approximately 430 dollars per lot.

Decision Rule - Accept the Lot as Good If No More than One (1) Defect  is
Found (i.e., d <. 1) .

Most Cost Effective Auditing Level  - From the  two dashed curves in
Figure 21 it can be seen that the cost curve has a minimum at n=15.   At
this level of auditing there is a probability  of 0.51 of accepting a  lot
of data  as good when it has 10 defects.  The average cost per lot is
approximately 340 dollars.

Auditing Level for Low Probability  of Accepting Bad  Data - For an auditing
level of n=20 the probability of accepting a lot with 10 percent defects
is about 0.36 as read from  the d _<  1 probability curve.  The average  cost
per  lot  is approximately 375 dollars.

It must  be realized that the shape  of a cost curve is determined by the
assumed  costs of performing the audit and of reporting bad data.  These
costs must be determined for individual monitoring situations in order to
select optimum auditing levels.

MONITORING STRATEGIES AND COST

Selecting the optimum monitoring strategy in terms of cost and data quality
requires a knowledge of the present data quality, major error components,
cost of  implementing available control procedures, and potential increase
in system precision and accuracy.
                                   64

-------
A methodology for comparing strategies to obtain the desired precision of
the data is illustrated in Section IV, page 86.  Table 6, page 62, lists
control procedures with estimated costs of implementation and expected
results in terms of which error component(s) are affected by the control.

Three system configurations identified as best strategies in Figure 23,
page 88, of the Management Manual are summarized here.   All calculations
were made for a true ozone concentration of 0.1 ppm.  The relationship of
the monitoring strategies will remain the same for all concentration
levels.  However, the variance in the data is a function of concentration
and should be computed for the concentration range of interest.

Reference Method

Description of Method - This refers to a sampling system as illustrated  in
Figure 2, page 6, of the Operations Manual.  Routine operating procedures
as given in the Operations Manual are to be followed with special checks
performed to identify problem areas when performance standards are not
being met.  An auditing level of n=7 out of a lot size of N=100 is
recommended for this strategy.  This strategy is identified as AO in
Table 14 and Figure 23 in the Management Manual.

Costs - Taken as reference or zero cost.

Data Quality - Combining the assumptions made concerning the measurement
of control samples and data processing errors, the data quality as read
from the graph in Figure 23 shows a standard deviation of 0.0066 ppm.
This means that a true ozone concentration of 0.1 ppm would be measured
within the limits of 0.0802 ppm and 0.1198 ppm approximately 99.7 percent
of the time.  To express it another way, one could say that true concen-
trations close to 0.1 ppm would be measured within + 20 percent of the
true value approximately'99.7 percent of the time.

Reference Method Plus Actions Al and A2 (A1+A2)

Description of Method - Identical with the above method except an average
calibration curve is constructed from 5 replicates of the calibration
curve and a 15 second time constant is used on the photomultiplier output
signal to integrate out spikes of less than 15 seconds duration.  This
reduces the chance of large data reduction errors which can be a highly
subjective process in the measurement method.

Costs - Estimated average cost per lot in excess of the costs of the
reference method above is 150 dollars.

Data Quality - From Table 14 and Figure 23 the data quality would be
described by a standard deviation of 0.0051 ppm.  This means that concen-
trations near 0.1 ppm will be measured within about + 15 percent of  the
true value 99.7 percent of the time.
                                    65

-------
Reference Method Plus Actions Al, A2, A3, and A4 (A6)

Description of Method - Identical to the immediate above method with the
addition of operating the analyzer in a shelter where the temperature is
controlled to within jf 3°C of a set value and utilizing a constant voltage
regulator in the power line to the analyzer.

Costs - From Figure 23 it is seen that the average cost per lot of data in
excess of the C9st of the .reference method is about 240 dollars.

Data Quality - The combination of all the actions as shown in Figure 23
has a standard deviation of about O.OOAO ppm (for a true concentration of
0.1 ppm).  This means that ozone concentrations near 0.1 ppm will be
measured within + 12 percent of the true value approximately 99.7 percent
of the time.

For these assumed values and conditions the data quality as measured by •
the standard deviation has improved by 40 percent (i.e., the standard
deviation has decreased, by 40 percent) in going from Action AO to Action A6
at an average additional cost of 240 dollars per lot of 100 samples.
                                   66

-------
                          MANAGEMENT MANUAL
GENERAL

The objectives of a data quality assurance program for the Chemilumines-
cent Method of measuring the concentration of ozone (ppm 0_) in the
ambient air were given in Section I.  In this part of the document,
procedures will be given to assist the manager in making decisions
pertaining to data quality based on the checking and auditing procedures
described in Sections II and III.  These procedures can be employed to:

     (1)  determine the extent of independent auditing to be
          performed,

     (2)  detect when the data quality is inadequate,

     (3)  assess overall data quality,

     (4)  relate costs of data quality assurance procedures
          to a measure of data quality, and to

     (5)  select from the options available to the manager
          the alternative(s) which will enable him to meet
          the data quality goals by the most cost-effective
          means.

The determination of the extent of auditing is considered in the section
entitled AUDITING SCHEMES.  Objectives 2 and 3 are discussed in the
section entitled DATA QUALITY ASSESSMENT, page 82.  Finally, Objectives
4 and 5 above are described in the section entitled DATA QUALITY VS COST
OF IMPLEMENTING ACTIONS, page 86.  The cost data are assumed and a
methodology provided.  When better cost data become available, improve-
ments can be made in the management decisions.

If the current reference system is providing data quality consistent with
that required by the user, there will be no need to alter the physical
system or to increase the auditing level.  In fact several detailed
procedures could be bypassed if continuing satisfactory data quality is
implied by the audit.  However, if the data quality is not adequate, e.g.,
either a large bias and/or imprecision exists in the reported data, then
(1) increased auditing should be employed, (2) the assignable cause
determined, and (3) the system deficiency corrected.  The correction can
take the form of a change in the operating procedure, e.g., use an
average of five calibrations with the KI method to construct an average
calibration curve; or it may be a change in equipment such as the instal-
lation of an improved temperature control system.  An increase in the
auditing level will increase the confidence in the reported measure of
precision/bias and aid in identifying the assignable cause(s) of the
large deviations.  The level of auditing will be considered in the
following subsection.
                                    67

-------
The audit procedure and the reported results can serve a two fold
purpose.  They can be used to  (1) screen the data, by lots of say N = 50
or 100, to detect when the data quality may be inadequate, and  (2)
calculate the bias and precision of the audited measurement and hence
estimate the bias/precision of the final reported concentration of ozone
in the ambient air.  In order to perform (1), suggested standards are
provided for use in comparing the audited results with the reported
values and a defect is defined in terms of the standards.  This approach
requires only the reporting of the number of defects in the n auditing
checks.  In the second method above, it is required to report the
measures of bias/precision in the audits.  These values are then used
in assessing the overall data quality.  Approach (1) is suggested as a
beginning step even though it will not make maximum use of the data
collected in the auditing program.  The simplicity of the approach and the
explicit definition of a defect will aid in its implementation.  After
experience has been gained in using the auditing scheme and in reporting
and calculating the results, it is recommended that approach (2) be
implemented.

It is important that the audit procedure be independent of previously
reported results and be a true check of the system under normal operating
procedures.  Independence can be achieved, for example, by providing a
control sample of unknown concentration (i.e., randomly setting the
generator sleeve position) to the operator and requesting that he measure
and report the concentration of the sample.  To insure that the check is
made under normal operating procedures, it is required that the audit be
performed without any special check of the system prior to the audit
other than that usually performed during each sampling period.

AUDITING SCHEMES

Auditing a measurement process costs time and money.  On the other hand,
reporting poor quality data can also be very costly.  For example, the
reported data might be used to determine a relationship between health
damage and concentrations of certain pollutants.  If poor quality data
are reported, it is possible that invalid inferences or standards derived
from the data will cost many dollars.  These implications may be unknown
to the manager until some report is provided to him referencing his data;
hence the importance of reporting the precision and bias with the data.

     Considering the cost of reporting poor quality data, it is desirable
to perform the necessary audits to assess the data quality and to invali-
date unsatisfactory data with high probability.  On the other hand, if
the data quality is satisfactory, an auditing scheme will only increase
the data measurement and processing cost.  An appropriate tradeoff or
balance of these costs must be sought.  These costs are discussed under
Cost Relationships.

Now consider the implication of an auditing scheme to determine or judge
the quality of the reported data in terms of an acceptance sampling
scheme.  Let the data be assembled into homogeneous lots of N = 50 or
100 sampling periods.  Suppose that n periods are sampled in the
                                    68

-------
manner suggested in Section III.  That is, the N = 50 or 100 sampling
periods are subdivided into equal time intervals (as nearly equal as
possible); then one day is selected at random during each interval.
Figure 18 gives a diagram of the data flow, sampling, and decision making
process for an auditing level of n = 7.

Statistics of Various Auditing Schemes

Suppose that the lot size is N = 100 periods (days), that n = 7 periods
are selected at random, and that there are 5% defectives in the 100, or
5 defectives.  The probability that the sample of 7 contains 0, 1, ..., 5
defectives is given by the following.
                    fr, A c  **   \
                   p(0 defectives)
                                         /100\
and for d defectives                   /5\  / /5 \

                   p(d defectives)  =     >JQQ\—  , d £ 5 .
                                    -      V7J

The values are tabulated' below for d = 0, 1,  ..., 6 and for the two
data quality levels.
Table 7. P(d defectives)
d
0
1
2
3
5
6
Data
D=5% Defectives
0.6903
0.2715
0.0362
0.0020
0.00004
= 0
Quality
D=15% Defectives
0.3083
0.4098
0.2152
0.0576
0.0084
ss 0
Figure 19A gives the probabilities of d = 0 and d <_ 1 defectives as a
function of sample size.  The probability is given  for lot size N = 100,
D = 5 and 15% defectives, for sample sizes  (auditing levels) from 1 to
25.  For example, if n = 10 measurements are audited and D = 5% defectives,
 (2) (?)
             /  5!  \  /  95!  \
	  \Ql5l)  W!88!/      95-94---S9
 /100\     ~     / 100! \          100-99-"94
 \ 7  )           \7!93!/
                                    69

-------
Data Flow
Lot 1
N = 100
Days


Lot 2
- 1UU
Days


                                            Sample
                                             n = 7
                                        Periods  (days)
       Observe
    d =.0 defects
Observe
= 1 defect
                          Calculate Costs of
                             Accepting and
                           Rejecting the Lot
    Accept Data If
 1.  Cost 'Comparison
    Favors This Action
 2.  Data Quality Is
    Acceptable
                    Reject Data
                     Otherwise
       Figure 18:   Data Flow Diagram for Auditing Scheme
                                 70

-------
                                                         d  s 1,  D - 5X
                           10          15
                          Sample Size (n)
20
                                                         d  -  0, D  -  5%
                                                         d  S  1, D  -  15X
                                                        d - 0, D - 15%
25
Figure 19A:  Probability of d Defectives in the Sample If
       the Lot (N » 100) Contains D% Defectives
                            71

-------
 I
 C/3
 01
 01
 
-------
the probability of d = 0 defectives is 0.58.  Figure 19B gives the
probabilities for lot size N = 50, for D = 6, 10, and 20% defectives,
and for d = 0 and d <_ 1.  These curves will be used in calculating the
cost relationships.

Selecting the Auditing Level

One consideration in determining an auditing level n used in assessing
the data quality is to calculate the value of n which for a prescribed
level of confidence will imply that the percent of defectives in the
lot is less than 10 percent, say, if zero defectives are observed in
the sample.*  Figures 20A and 20B give the percentage of good measure-
ments in the lot sampled for several levels of confidence, 50, 60, 80,
90, and 95%.  The curves in 20A assume that 0 defectives are observed
in the sample, and those in 20B that 1 defective is observed in the sample.
The solid curves on the figures are based on a lot size of N = 100; two
dashed curves are shown in Figure 20A for N = 50; the differences between
the corresponding curves are small for the range of sample sizes
considered.

For example, for zero defectives in a sample of 7 from a lot of N = 100,
one is 50% confident that there are less than 10% defective mesaurements
among the 100 reported values.  For zero defectives in a sample of 15
from N = 100, one is 90% confident that there are less than 10% defec-
tive measurements.  Several such values were obtained from Figure 20A
and placed in Table 8 below for convenient reference.
             Table 8.  REQUIRED AUDITING LEVELS n FOR LOT
                SIZE N = 100 ASSUMING ZERO DEFECTIVES               	

     Confidence Level            D = 10%            15%            20%
50%
60%
80%
90%
95%
7
9
15
20
K 25
<5
6
10
15
18
<5
<5
8
11
13
 Obviously, the definition of defective need not always be  the same and
must be clearly stated each time.  The definitions employed herein are
based on results of collaborative test programs.
                                    73
-------
   100
    80
00

4J



S
g   60

0)
"g   AO
a
-------
  100
   80
3  60
to
•o
o

S

u-i
o  40



8
u
   20
             50%
             60%
            1  80%
90%

I 1 I  I


95%
                             10           15


                               Sample  Size (n)
                                        20
25
Figure 20B:  Percentage of Good Measurements Vs.  Sample Size



   for 1 Defective Observed and Indicated  Confidence Level



                       Lot Size = 100
                               75
-------
Cost Relationships

The auditing scheme can be translated into costs using the costs of
auditing, rejecting good data, and accepting poor quality data.  These
costs may be very different in different geographic locations.  There-
fore, purely for purposes of illustrating a method, the cost of auditing
is assumed to be directly proportional to the, auditing level.  For n = 7
it is assumed to be $155 per lot of 100.  The cost of rejecting good
quality data is assumed to be $600 for a lot of N = 100.  The cost of
reporting poor quality data is taken to be $800.  To repeat, these costs
given in Table 9 are assumed for the purpose of illustrating a methodology
of relating auditing costs to data quality.  Meaningful results can only
be obtained by using correct local information.
                   Table 9.  COSTS VS. DATA QUALITY
                                           Data Quality
     Reject Lot of
     Data
                                "Good"
                                    "Bad"
         D £ 10%
    Incorrect Decision

Lose cost of performing
audit plus cost of reject-
ing good quality data.
(-$600 - $155)
       D > 10%
   Correct Decision

Lose cost of performing
audit, save cost of not
permitting poor quality
data to be reported.
($400 - $155)
     Accept Lot of
     Data
     Correct Decision

Lose cost of performing
audit.
(-$155)
  Incorrect Decision

Lose cost of performing
audit plus cost of de-
claring poor quality
data valid.
(-$800 - $155)
Suppose that 50 percent of the lots have more than 10 percent defectives
and 50 percent have less than 10 percent defectives.  (The percentage of
defective lots can be varied as will be described in the final report
under the contract.)  For simplicity of calculation, it is further assumed
that the good lots have exactly 5 percent defectives and the poor quality
lots have 15 percent defectives.
 Cost of performing audit varies with the sample size; it is assumed to
be $155 for n = 7 audits per N « 100 lot size.
                                    76
-------
Suppose that n = 7 measurements out of a lot of N = 100 have been audited
and none found to be defective.  Furthermore, consider the two possible
decisions of rejecting the lot and accepting the lot and the relative
costs of each.  These results are given in Tables 10A and 10B.
Table 10A.  COSTS IF 0 DEFECTIVES ARE OBSERVED AND THE LOT IS REJECTED

Reject Lot

D - 5%
D = 15%
Correct
Decision
	
P2 = 0.31
C2 = 400 - 155
Incorrect
Decision
Px = 0.69
GI = -600 - 155
	
Net Value ($)
p^ - -$521
P2C2 = $76
                                           Cost = p1C1 + p2C2»-$445
Table 10B.  COSTS IF 0 DEFECTIVES ARE OBSERVED AND THE LOT IS ACCEPTED

Accept Lot
D = 5%
D = 15%
Correct
Decision
Pl » 0.69
C3 = -155
	
Incorrect
Decision
	
P2 = 0.31
C4 = -800 - 155
Net Value ($)
PlC3 = -$107
P2C4 = -$296
                                           Cost
-$403
The value P,(P2) in the above table is the probability that the lot is
5% (15%) detective given that 0 defectives have been observed.  For
example,
                                    77
-------
                  /probability that the lot is 5% defective"}
          	'_	\   and 0 defectives are observed        /
   Pl  ~   /lot is 5% defective andV~~/lot is 15% defective and\
           \ 0 defectives observed /      \ 0 defectives observed  /

       	0.5(0.69)	
          0.5(0.69)  +  0.5(0.31)
                   /probability that the lot is 15% defective)
       _  	   •    \   and .0 defectives are observed         /
   P2  ~    /lot. is 5% defective and\~/lot is 15% defective and\
            \ 0 defectives observed /      \ 0 defectives observed  /

       	0-5(0.31)	  =  0.31 .
          0.5(0.31)  +  0.5(0.69)


It was" assumed that the probability that the lot is 5% defective is 0.5.
The probability of observing zero defectives, given the lot quality is
5% or 15%, can be read from the graph of Figure 19A.

A similar table can be constructed for 1, 2, ..., defectives and the net
costs determined.  The net costs are tabulated in Table 11 for 1, 2, and
3 defectives.  The resulting costs indicate that the decision preferred
from a purely monetary viewpoint is to accept the lot if 0 defectives are
observed and to reject it otherwise.  The decision cannot be made on this
basis alone.  The details of the audit scheme also affect the confidence'
which can.be placed in the data qualification; consideration must be
given to that aspect as well as to cost.
                     Table 11.  COSTS IN DOLLARS


Reject Lot
Accept Lot

0
-445
-403
d = number
1
-155
-635
of defectives
2
+101
-839

3
+207
-928
Cost Versus Audit Level

After, the decision criteria have been selected, an average cost can be
calculated.  Based on the results of Table 11, the decision criterion
is to accept the lot if d = 0 defectives are observed and to reject the
lot if d = 1 or more defectives are observed.  All the assumptions of the
previous section are retained.  The auditing level is later varied to obtain
the data in Figure 21.
                                    78
-------
One example calculation is given below and summarized in Table 12.  The
four cells of Table 12 consider all the possible situations which can
occur, i.e., the lots may be bad or good and the decision can be to
either accept or reject the lot based on the rule indicated by Table 11.
The costs are exactly as indicated in Tables 10A and 10B.  The probabili-
ties are computed as follows.
            (prob. that the lot is 5% defective and 1 or more
            defects are obtained in the sample)

            (prob. that the lot is 5% defective)(prob. 1 or
            more defectives are obtained in the sample given
            the lot is 5% defective)
            0.5 (0.31)
0.155
Similarly q2> q_, and q, in Table 12 are obtained as indicated below.
q2
«3
= 0.5 (0.69)
= 0.5 (0.69)
= 0.5 (0.31)
= 0.345
= 0.345
= 0.155
The sum of all the q's must be unity as all possibilities are considered.
The value of 0.5 in each equation is the assumed proportion of good lots
(or poor quality lots).  The values of 0.31 and 0.69 are the conditional
probabilities that given the quality of the lot, either d = 0 or d = 1 or
more defectives are observed in the sample.  Further details of the
computation are given in the final report of this contract.

               Table 12.  OVERALL AVERAGE COSTS FOR ONE
	ACCEPTANCE - REJECTION SCHEME	

Decision
Reject any lot of
data if 1 or more
defects are found.
Accept any lot of
data if 0 defects
are found .
Good Lots
D = 5%
q-L = 0.155
Cj_ = -$755
q3 = 0.345
C3 = -$155
Bad Lots
D = 15%
q2 = 0.345
C2 = $245
q^ = 0.155
C4 = -$955


qlCl + q2C2 = "$ 32

q3C3 + q4C4 " "$2°2

                                                 Average Cost =  -$234
                                    79
-------
In order to interpret the concept of average cost, consider a large
number of data lots coming through the system; a decision will be made
on each lot in accordance with the above, and a resulting cost of the
decision will be determined.  For a given lot, the cost may be any one
of the four costs, and the proportion of lots with each cost is given
by the q's.  Hence the overall average cost is given by the sum of the
product of q's by the corresponding C's.

In order to relate the average cost as given in Table 12 to
the costs given in Table 11, it is necessary to weight the costs in
Table 11 by the relative frequency of occurrence of each observed
number of defectives, i.e., prob(d).  This calculation is made below.

  No. of       Decision     Costs ($) from
Defectives       Rule          Table 11         Prob(d)     Cost * Prob(d)

  d = 0         Accept            - 403          0.50            -$201.5

      1         Reject            - 155          0.34     .       -   52.7

      2         Reject             101          0.1255             12.6

      3         Reject             207          0.030               6.2

      4         Reject             244          0.0042              1.0

                                       Totals  0.9997          -$234.4


Thus the value -$234 is the average cost of Table 12 and the weighted
average of the costs of Table 11.  The weights, Prob(d), are obtained
as follows:


  Prob(d=0)  =  Prob(lot is good and d=0 defectives are observed)

             +  Prob(lot is poor quality and d=0 defectives are observed)

             = 0.5(0.69) + 0.5(0.31)  =  0.50  .

This is the proportion of all lots which will have exactly 0 defectives
under the assumptions stated.  For d = 1, 2, 3, and 4, the values of the
probabilities in parentheses above can be read from Table 7.

Based on the stated assumptions, the average cost was determined for
several auditing levels as indicated in Table 12.  These costs are given
in Figure  21.  One observes from this figure that n = 7 is cost effective
given that one accepts it only if zero defectives are observed.  (See
curve for d = 0.)

If the lots are accepted if either 0 or 1 defectives are observed, then
referring to the curve d £ 1, the best sampling level is n = 15.  The
curve of probability of d = 0 (d £ 1) defectives in a sample of n from a
lot of N = 100 measurements, given that there are 10% defectives in the
lot, is also given on the same figure.
                                    80
-------
                                                              J-S
                                                              o
                                                               n


                                                              Q

                                                              M-J
                                                              •H


                                                               C

                                                               HI
                                                               N
                                                               O.

                                                               n)
                                                               0
                                                               (U
                                                               14-1
                                                               0)
                                                               Q
                                                               00
                                                               C
                                                               •H

                                                               t-i
                                                               OJ
                                                               Ul
             5             10           15


             Audit Level (Sample Size)
                                                               n)
                                                               .o
                                                               o
                                                               M
                                                               rx,
Figure  21:   Average Cost Vs.  Audit Level

                 (Lot  Size N =  100)
                       81
-------
Another alternative is to accept all data without performing an audit.
Assuming that one-half (50%) of the lots contain more than 10% defectives,
the average cost on a per lot basis would be 0.5(-$800) = -$400.  This,
however, would preclude qualification of the data.  Regardless of cost,
It would be an unacceptable alternative.

DATA QUALITY ASSESSMENT

In this section, approach 2 is considered; that is, the precisions and
biases of the individual measurements and operational procedures are
estimated.  These results are then used to make an overall assessment of
Hie data quality.

Assessment of Individual Measurements

Assume for convenience that an auditing period consists of N = 100 days
(or sampling periods).  Subdivide the auditing period into n equal or
nearly equal periods.  Make one audit during each period and compute  the
deviations (differences) between the audit values and the stated values
(or previously determined values as measured by the operator) as indicated
in the Supervision Manual.  For example, if seven audits (n = 7) are  to be
performed over 100 sampling periods (N = 100), the 100 periods can be
subdivided into  7 intervals  (6 with 14 periods and 1 with 16 periods).
Select one day at random within each interval and perform the suggested
audits.  The operator should not be aware of when the checks are to be
per Formed .

[n order to assess the data quality using measures of bias/precision,  the
checks are to be combined for the selected auditing period, and the mean
difference or bias and the  standard deviation of  the differences are  to
be computed as indicated below.

The formulas for average bias and the estimated standard deviations are
the standard ones given in  statistical  texts  (e.g., see Ref. 6).  The
level of sampling or  auditing, n, will be considered as a parameter to
be selected by the manager  to assess the quality  of data as required.
      (1)  Control  Sample
                       Bias   =   d
                                 1

where
           d  .   =  difference  in generator output and analyzer
             J     response in ppm  (see page  51).
                                    82
-------
           Standard Deviation  =  - 8.  =


where

          d^  =  the average bias, and

          s..  =•  the estimated, standard deviation of the .KI  (or
                 chemiluminescent) method corrected for the
                 average bias d...

     (2)  Data Processing Check
                                       n


                      Bias  = ^d"0  =  ~^	   ,
                               ' 2        n
where
         d_.  =  deviation of the concentration of ozone as read
                 from the strip chart, calibration table, and
                 recorded by the operator and by one .performing
                 the audit,
           Standard Deviation  •  s_  =   v       ,
                                   /      ?     n—i


Individual checks on the standard deviations of the two audits  can be
made by.computing the ratio of the estimated standard deviation,  s., to
the corresponding suggested performance standard,  o , given  in  Table 13.
If this ratio exceeds values given.in Table 13 for any one of the audits,
it would indicate that the source of trouble may be assigned to that
particular aspect of the measurement process.  Critical values  of this
ratio are given in Figure 22 as a function of sample size and two levels
of confidence.  Having assessed the general problem area, one then needs
to perform the appropriate quality control checks  to determine  the specific
causes of the large deviations.
 The factor 2 is inserted in the denominator  to account  for  the  fact
that the variance of the difference of  two measurements,  each with  the
same variance, is twice the variance of an individual measurement.  The
assumption that the two measurements have the same variances is  reasonable
on the basis of in-house studies.  An analysis of these  results  is  given
in the final report of this contract.
                                    83
-------
1.60
1.10
     0    5     10    15     20    25     30    35    40




                  Sample  Size (n)









Figure 22:  Critical Values of Ratio s /a± Vs. n
                         84
-------
                 Table 13.   CRITICAL VALUES OF s.
Level of
Confidence Statistic
90% s±/a1
95% SI/OJL
Audit Level
n-5 n=10 n=15 n-20
1.40 1.29 1.23 1.20
1.54 1.37 1.30 1.26
s. =• estimated standard deviation
0. = hypothesized or suggested standard deviation.
n=25
1.18
1.23


Audit
Control Sample
Data Processing Check
Overall Standard Deviation
Suggested Performance Standard
a = (0. 0033+0. 025*ppm 0,)
02 - 0.003 ppm 03
*
a = 0.0066 ppm 0_
 a  is calculated for 0.10 ppm 0,.
Overall Assessment of Data Quality

The values d_, d_, s.,, and a. above measure the bias and variation
of the reported aata for the two audits.  These measures can now be
combined to obtain an overall bias estimate T and standard deviation
o , as follows:
These estimates can then be used in reporting the overall bias and
precision as suggested by the following.  The true concentration of ozone
should fall in the following interval where ppm 0. is the measured
concentration,

                         ppm 03 + T ± 2oT  ,
                                   85
-------
 approximately  95 percent  of  the  time, or within  the  interval
                           ppm
approximately  99.7 percent  of  the  time.  When  computed  from audit data,
the value  2&T  (or 3$T)  is actually dependent on the number of audits

conducted.   If n is  large,  say about 25 or  larger, the  value 2  (or  3) is
appropriate.

In reporting the data quality, the bias, overall standard deviation, and
auditing level should be reported  in an ideal  situation (see the section
entitled DATA  PRESENTATION  for further discussion).  More restricted
information  following approach 1 is suggested  in the Supervision Manual
as a minimal reporting  procedure.

If the overall reported precisions/biases of the data meet or satisfy
the requirements of  the user of the data, then a reduced auditing level
may be employed; on  the other  hand, if the  data quality is not  adequate,
assignable causes of large  deviations should be determined and  appropriate
action taken to correct the deficiencies.   This determination may require
increased  checking or auditing of  the measurement process as well as the
performance  of certain  quality control checks, e.g., monitoring of
temperature  variations  over the 24-hour sampling period.


DATA QUALITY VERSUS COST OF IMPLEMENTING ACTIONS

The discussion and methodology given in a previous section were concerned
with the auditing scheme (i.e., level of audit or sample size,  costs
associated with the data quality,  etc.).  Increasing the level  of audit
of the measurement process does not by itself change the quality of the
data, but it does increase the information  about the quality of the
reported data.   Hence,  fewer good  lots will be rejected and more poor
quality data will be rejected.  If the results of the audit imply that
certain process measurement variables or operational procedures are major
contributors to the total error or variation in the reported concentration
of ozone, then alternative strategies for reducing these variations need
to be investigated.  This section  illustrates a methodology for comparing
the strategies to obtain the desired precision of the data.  In practice
it would be necessary to experiment with one or more strategies, to
determine the  potential increase in precision, and to relate the
precisions to  the relative costs as indicated herein.   Several  strategies
are considered, but only a few of  the least costly ones would be acceptable,
as illustrated in Figure 23.  The  assumed values of the standard deviations
and biases for each type audit are not based on actual  data, except for the
reference.method.  In this case values were taken from  Environmental
Protection Agency in-house studies.  These  values are probably  smaller than
those experienced in the field.
                                    86
-------
Several alternative actions or strategies can be taken to increase the
precision of the reported data.  For example, if the temperature variations
are large, the measurement methods may vary and, cause variation in
ppm 0-.  Under these conditions additional control equipment for temperature
variation can reduce the variation of the measured responses by calculated
amounts and thus reduce the error of the reported concentrations.  In this
manner, the cost of the added controls can be related to the data quality as
measured by the estimated bias/precision of the reported results.

In order to determine a cost efficient procedure, it is necessary to
estimate the variance for each source of error (or variation) for each
strategy and then to select the strategy or combination of strategies
which yields the desired precision with minimum cost.  These calculations
are summarized in Table 14 with assumed costs of equipment and control
procedures.

Suppose that it is desired to make a statement that the true 0- concentration
is within 0.012 of the measured concentration (for simplicity of discussion all
calculations were made at a true concentration of 0.1 ppm) with approximately 95
percent confidence.   Minimal cost control equipment  and  checking  procedures  are
to be employed to attain this desired precision.
Examining the graph in Figure 23 of cost versus precision, one observes
that A2 is the least costly strategy that meets the required goal of
2oT = 0.012 or a  - 0.006 ppm 0- in the reported concentration.  Similarly
the combination of Al and A2 meets the requirement that 3o_ = 0.0153 or
a» = 0.0051 ppm 0,.  The assumed values of the standard deviations of the
measured concentrations of ozone for the alternative courses of action are
given in Table 14.  The costs for the various alternatives are given in
Table 6 of Section III and in Table 14.


Suppose that it is desired that o  be less than 0.005 and that the cost
of reporting poor quality data increases rapidly for a  greater than 0.005.
This assumption is illustrated by the cost curve given by the solid line
in Figure 23.   For any alternative strategy, the cost of reporting poor
quality data is given by the ordinate of this curve corresponding to the
strategy.

DATA PRESENTATION

A reported value whose precision and accuracy  (bias) are  unknown is of
little, if any, worth.  The actual error of a reported value—that is,
the magnitude and sign of its deviation from the true value—is  usually
unknown.  Limits to this error, however, can usually be inferred, with
some risk of being incorrect, from the precision of the measurement
process by which the reported value was obtained and from reasonable
limits to the possible bias of the measurement process.   The bias, or
systematic error, of a measurement process  is the magnitude and  direction
of its tendency to measure something other  than what was  intended; its
precision refers to the closeness or dispersion of  successive independent
measurements generated by repeated applications of  the process under
specified conditions, and its accuracy is determined by the closeness to
the true value characteristic of such measurements.

                                   87
-------
250 i-
                               CT  (xio-4)
                                                       COST OF REPORTING
                                                       POOR QUALITY DATA
                                                          70
    Figure- 23.   Costs Vs. Precision for Alternative Strategies
                                 88
-------
    .Table 14.   ASSUMED STANDARD DEVIATIONS FOR ALTERNATIVE STRATEGIES
1. Control d
Sample
2. Data d
Processing
°2

AO
0
0
0.003
Al
0
0.87a1
0
0.003
A2
0
0
0.001
A3
0
0
0.003
A4
0
0.930.,^
0
0.003


A1+A2
0
0.87^
0
0.001
it
A6
0
0.67^
0
0.001
      =  (0.0033 +0.025 * ppm
       **
            ppm
  ***
    *

     Bias=T ppm 0-




  Added Cost

  ($)/100 Samples
0.0066
0
0
0.0058
0
150
0.0059
0
0
0.0061
0
75
0.0062
0
15
. . .
. . .
• • •
0.0051
0
150
0.0040
0
240
***
  A6



 * 2

  °T



Bias
=  Al + A2 + A3 + A4



    22           I 2
=  a  + o2 '  0   =  \a  ppm 0_, the a  is calculated at 0.10 ppm  0.^.




= T «• d. + d  , the biases are assumed to be zero  for all  strategies.
                                      89
-------
Precision and accuracy are inherent characteristics of the measurement
process employed and not of the particular end result obtained.  From
experience with a particular measurement process and knowledge of its
sensitivity to uncontrolled factors, one can often place reasonable
bounds on its likely systematic error (bias).  This has been done in the
model for the measured concentration as indicated in Table 14.  It is
also necessary to know how well the particular value in hand is likely
to agree with other values that the same measurement process might have
provided in this instance or might yield on measurements of the same
magnitude on another occasion.  Such information is provided by the
estimated standard deviation of the reported value, which measures (or
is an index of) the characteristic disagreement of repeated determinations
of the same quantity by the same method and thus serves to indicate the
precision (strictly, the imprecision) of the reported value.

A reported result should be qualified by a quasi-absolute type of statement
that places bounds on its systematic error and a separate statement of its
standard deviation, or of an upper bound thereto, whenever a reliable
determination of such value is available.  Otherwise, a computed value of
the standard deviation should be given together with a statement of the
number of degrees of freedom on which it is based.

As an example, consider strategy AO in Table 14.  Here, the assumed standard
deviation and bias for true ozone concentration of 0.10 ppm 0_  are OT = 0.0066
and T = 0, respectively.  The results would be reported as the measured
concentration, ppm 0-, plus the bias and with the following 2a limits, along
with the audit level and lot size N; e.g.,
                   ppm 0_ + 0.0066, n = 7, N
100
For concentration other than 0.10 ppm, the overall standard deviation  is
obtained by
                                      _,_  2
                                      + a2

where a. is given in Table 14, computed at the desired  concentration.


PERSONNEL REQUIREMENTS

Personnel requirements as described here are in terms of the chemilumines-
cent method only.  It is realized that these requirements may be only a
minor factor in the overall requirements from a systems point of view where
several measurement methods are of concern simultaneously.

Training and Experience

Director - The director or one of the.professional-level employees should
have a basic understanding of statistics as used in quality control.  He
should be able to perform calculations, such as the mean and standard
                                    90
-------
deviation, required to define data quality.  The importance of and require-
ments for performing independent and random checks as part of the auditing
process must be understood.  Three references which treat the above-
mentioned topics are listed below:

     Probability and Statistics for Engineers, Irvin Miller and
     John E. Freund, published by Prentice-Hall, Inc., Englewood,
     N. J., 1965.

     Introductory Engineering Statistics, Irwin Guttman and
     S. S. Wilks, published by John Wiley and Sons, Inc., New York,
     N. Y., 1965.

     The Analysis of Management Decisions, William T. Morris,
     published by Richard D. Irwin, Inc., Homewood, Illinois, 1964.

Operator - There are or can be two levels of operation involved in the
cherailuminescent method.

First, an operator or technician who is involved in the .preliminary or
initial setup and checkout or is responsible for troubleshooting and
repairing the analyzer should have technical training in electronics
and/or instrumentation as obtained in a technical or service school or
several years of on-the-job experience.  For a specific analyzer it
would be desirable to have the technician checked out by a manufacturer's
representative or at least to have him participate, with the representative,
in the initial installation and startup.  The manufacturer's instruction
book should be available for study or reference by the technician.

Routine operations involve the use of external controls only and require
no high-level skills.  A high school graduate with proper supervision
and on-the-job training can become effective at this level in a very
short time.

An effective on-the-job training program could be as follows:

     (1)  Observe experienced operator perform the different tasks
          in the measurement process.

     (2)  Study the operational manual of this document and use it
          as a guide for performing the operations.

     (3)  Perform operations under the direct supervision of an
          experienced operator.

     (A)  Perform operations independently but with a high level
          of quality control checks utilizing the technique
          described in the section on Operator Proficiency
          Evaluation Procedures below to encourage high quality
          work.

Another alternative would be to have the operator attend an appropriate
basic training course sponsored by EPA.

                                   91
-------
OPERATOR PROFICIENCY EVALUATION PROCEDURES

One technique which may be useful for early training and qualification
of operators is a system of- rating the operators as indicated below.

Various types of violations (e.g., invalid sample resulting from operator
carelessness, failure to maintain records, use of improper equipment, or
calculation error) would be assigned a number of demerits depending upon
the relative consequences of the violation.  These demerits could then be
summed over a fixed period of time of one week, month, etc., and a
continuous record maintained.  The mean and standard deviation of the
number of demerits per week can be determined for each operator and a
quality control chart provided for maintaining a record of proficiency
of each operator and whether any changes in this level have occurred.  In
comparing operators, it is necessary to assign demerits on a per unit work
load basis in order that the inferences drawn from the chart be consistent.
It -CA not necewotf/ on dei-cAoMe ^ofi the. op&uvLon. to be owoAe o& thu,
0($ evaluation.  The. AupeAv^Aon. should o6e  a mean* o
whe.n and what kind ofi  needed.

A sample QC chart is given in Figure 24 below.  This chart assumes that
the mean and standard deviation of the number of demerits per week,
are 5 and 1, respectively.  After several operators have been evaluated
for a few weeks, the limits can be checked to determine if they are both
reasonable and effective in helping to improve and/or maintain the quality
of the air quality measurement.

The limits should be based on the operators whose proficiency is average or
slightly better than average.  Deviations outside the QC limits, either
above or below, should be considered in evaluating the operators.  Identify-
ing those operators whose proficiency may have improved is just as important
as knowing those operators whose proficiency may have decreased.

The above procedure may be extended to an entire monitoring network  (system).
With appropriate definitions of work load, a continuous record may be
maintained of demerits assigned to the system.  This procedure might serve
as an incentive for teamwork, making suggestions for improved operation
procedures, etc.
                    12345   6  7  8  9  10 11 12 13
                            TIME INTERVALS (WEEKS)
     Figure 24:  Sample QC Chart  for Evaluating Operator Proficiency

                                    92
-------
                              REFERENCES

1.   National Bureau of Standards Technical Note 585,  Microchemical Analysis
    Section; Summary of Activities, July 1970 to June 1971, John K. Taylor
    Editor, January 1970, 77 pages.

2.   Herbert C. McKee, and Ralph S.  Childers, Provisional Report of
    Collaborative Study of Reference Method for Measurement of Photochemical
    Oxidants Corrected for Interference Due to Nitrogen Oxides and Sulfur
    Dioxide, Contract CPA 70-40.

3.   J. A. Hodgeson, R. K. Stevens,  and B.  E. Martin,  A Stable Ozone Source
    Applicable as a Secondary Standard for Calibration of Atmospheric
    Monitors, Air Quality Instruments, Vol. 1, Instrument Society of
    America, Pittsburg, 1972, pp. 149-158.

4.   J. A. Hodgeson, B. E. Martin, and R. E. Baumgardner, Comparison of
    Chemiluminescent Methods for Measurements of Atmospheric Ozone,
    Eastern Analytical Symposium, paper number 70, New York, N. Y.,
    November 1970.

5.   Gary J. Warren and Gordon Babcock, Portable Ethylene Chemiluminescence
    Ozone Monitor, Review of Scientific Instruments,  Vol. 41, 1970,
    pages 280-282.

6.   John Mandel, The Statistical Analysis of Experimental Data, Interscience
    Publishers, Division of John Wiley & Sons, New York, N. Y., 1964.
                                   93
-------
                         APPENDIX

           REFERENCE METHOD FOR THE MEASUREMENT
         OF PHOTOCHEMICAL OXIDANTS CORRECTED FOR
           INTERFERENCES DUE TO NITROGEN  OXIDES
                    AND SULFUR DIOXIDE
Reproduced from Appendix D, "National Primary  and  Secondary
Ambient Air Quality Standards," F'idsJuoJL Reg-ci-te/L,  Vol 36,
No. 84, Part II, Friday, April 30,  1971.
                              94
-------
                                                  RULES  AND REGULATIONS
APPENDIX  D—REFERENCE .METHOD  rom fan
  MEASUREMENT or PHOTOCH«KICAL OimiMrm
  CORRECTED  FOB  IHTERrEBXNCia  DOT   TO
  NITROGEN  OXIDES AND SULFUB Dioxms

  1. Principle and  Applicability.
  1.1   Ambient air  and  ethylene are  de-
livered  simultaneously to  a mixing zone
where  the ozone In the air reacts with tha
ethylene to  emit light which  Is detected by
a photomultlpller tube. The resulting photo-
current Is  amplified and Is either read di-
rectly or displayed on a recorder.
  1.3   The method Is applicable to the con-
tinuous measurement  of  ozone In ambient
air.
  2. Range and Sensitivity.
  2.1   The range Is 9.8 #g. O/m.' to greater
than  1960  sg.  CVm.1  (0.006 p.pjn. O,  to
greater than 1 p.p.m. O,).
  2.2  The sensitivity la 8.8 «. Oi/m.«  (0.006
p.p.m. O>).
  3.  Interferences.
  3.1  Other oxidizing and reducing species
normally found In ambient air do not  Inter-
fere.
  4. Precision ant Accuracy.
  4.1  The average deviation from the mean
of repeated single measurements does not ex-
ceed 5 percent of the mean of the measure-
ments.
  4.3  The method Is accurate within  ±7
percent.
  6.  Apparatus.
  5.1  Detector Cett. Figure Dl Is a drawing
of a typical detector cell showing flow  path*
of gaae*, the mi^niy acne,  and placement of
ttoe photomultlpller  tub*.  Ottaer now  path*
In whloh the air and (thylene streams meet
 :il it  wint new the photormilUplifr tulw :ire
 moo Allowable.
  5.2  Air Flowmetcr.  A device capable of
 controlling air flows between 0-1.5 1/niin.
  5.3  Ethylene Floiameter. A device capable
 of controlling ethylene flows  between  0-50
 ml./mln. At any flow In this range, the device
 should  be capable of maintaining constant
 flow rate  within -^3  ml./mln.
  5.4  Air  Inlet  Filter.  A  Teflon   niter
 capable of removing all particles greater than
 5 microns  in diameter.
  5.5  Photomultiplicr  Tube.  A  high  gain
 low  dark  current  (not more  than  1 x 10-*
 ampere)  photomultiplier  tube  having  its
 maximum gain at about 430  nm. The  fol-
 lowing  tubes  are  satisfactory:  RCA  4507.
 RCA  8575,  EMI 9750.  EMI  9524.  and  EMI
 9536.
  5.6  High  Voltage  Power Supply. Capable
 of delivering  up to 2,000 volts of  regulated
 power.
  5.7  Direct Current Amplifier. Capable of.
 full scale amplification of currents from  10-'*
 to 10-7 ampere; an electrometer is commonly
 used.
  5.8  Recorder. Capable of full scale display
 of voltages from the DC amplifier. These volt-
 ages commonly are in the 1 millivolt to 1-volt
 range.
  5.9  Ozone Source  and  Dilution System.
 The ozone source consists of a quartz tube
 into which ozone-free air Is introduced and
 then  irradiated with  a  very stable  low pres-
 sure mercury lamp. The level of irradiation is
 controlled  by an adjustable aluminum sleeve
 which fits around the  lamp. Ozone concen-
 trations are varied  by  adjustment of  this
 sleeve. At a fixed level of Irradiation, ozone Is
 produced at  a  constant rate. By carefully
 controlling the flow of air through the quartz
 tube, atmospheres are generated which con-
 tain constant concentrations of ozone.  The
 levels of ozone  In the  test atmospheres are
 determined by  the neutral  buffered potas-
 sium Iodide method  (see section 8). This
 ozone source and dilution system  Is shown
 schematically in Figures D2 and D3, and has
 been  described by Hodgesorr,  Stevens,  and
 Martin.
  5.10  Apparatus for Calibration
  5.10.1  Absorber. All-glass  Implngers as
 shown In Figure D4  are recommended.  The
 Implngers may be purchased from most  ma-
 jor glassware  suppliers. Two  absorbers In
 series are needed to Insure complete collec-
 tion of the sample.
  5.10.2  Air  Pump.  Capable of drawing 1
 liter/minute  through  the  absorbers.  The
 pump should be equipped with a needle valve
 on the inlet side to regulate flow.
  5.10.3  Thermometer.  With  an  accuracy
 of ±2« C.
  5.10.4  Barometer. Accurate to the nearest
 nun. Hg.
  5.10.5  Flaiometer. Calibrated metering de-
 vice for measuring flow up to 1 liter/minute
-within  ±2  percent.   (For  measuring  flow
 through Lmplngera.)
  5.10.8  riowmeter.  For measuring airflow
 past the"lamp; must be capable of measuring
 flows  from 2 to 15 liters/minute within ±6
 percent.
  5.10.7  Trap. Containing glass wool to pro-
 tect needle valve.
  6.10.8  Volumetric  Flasks.  25,  100,  500.
 1,000  ml.
  8.10.9  Buret. 50 ml.
  6.10.10  Pipets. 0.5, 1, 2, 3, 4, 10, 25, and
 60 ml. volumetric.
  5.10.11  Erlenmeyer Flasks. 300 rnL
  5.10.12  Spectrophatomettr.  Capable   of
measuring absorbance at 362 nm.  Matched
 1-cm. cells should be used.
  6. Reagents.
  6.1  Ethylene. C. P.  grade (minimum).
  6.2' Cylinder Air. Dry grad*.
                               FEDERAL REGISTER, VOL. 36, NO. 221—THURSDAY,  NOVEMBER 25, 1971

                                                                95
-------
                                                   RULES AND REGULATIONS
                                                                                22393
   e.a  Activated Charcoal Trap. For filtering
 cylinder air.
   8.4  Purified Water. Used for «11 reagent*.
 To distilled or delonlzed water In an all-glass
 distillation apparatus, add • crystal of potas-
 «lum permanganate and a crystal  of barium
 hydroxide, and redistill.
   6.6  Absorbing Reagent.  Dissolve 13.6 g.
 potassium dlhydrogen phosphate  (KTT.PO,),
 14.3 g. anhydrous dlsodlum hydrogen phos-
 phate (Na^PO.)  or 36.8 g. dodecahydrata
 salt (Na]HPO.HH,O). and 10.0 g.  potassium
 Iodide (KI) In purified water and dilute to
 1,000 ml. The pR  should be  6.8 ±0.2. The
 solution Is stable for several weeks. If stored
 In a glass-stoppered amber  bottle In a cool,
 dark place.
   6.6  Standard  Araenfoiw  Oxide Solution
 (0.05 N). Use primary standard grade arse-
 nlous oxide (As,O,).  Dry 1  hour at 106' O.
 Immediately before using. Accurately weigh,
 to the nearest 0.1 mg.. 3.4 g. anenlous oxide
' from a small glass-stoppered weighing bottle.
 Dissolve in 26 ml. 1 N sodium hydroxide In a
 flask or beaker on a steam bath. Add 28 ml.
 1 N sulrurtc  add.  Oool,  transfer quantita-
 tively  to a 1,000-ml.  volumetric  flask,  and
 dilute to volume. NOTE:  Solution roust be
 neutral to litmus, not alkaline.
       Normality  ABjOs= •
                        wt As.0.  (g.)
                            40.46
    6.7  Starch Indicator Solution  (0.2 per-
 cent) . Triturate 0.4 g. soluble starch and ap-
 proximately 2 mg. mercuric Iodide (preserva-
 tive) with a little water. Add the paste slowly
 to 200 ml. of boiling water. Continue boiling
 until the solution Is clear, allow to cool, and
 transfer to a glass-stoppered bottle.
    6.8  Standard  Iodine Solution (0.05 N).
    6.8.1   Preparation.  Dissolve  6.0 g. potas-
 sium Iodide (KI) and 3.2 g. resubllmed iodine
  (I,) In 10 ml. purified water. When the Iodine
 dissolves, transfer the solution to a 600-ml.
 glass-stoppered  volumetric flask. Dilate to
 mark  with purified  water  and mix thor-
 oughly. Keep solution In a dark brown glass-
 stoppered bottle away  from light,  and re-
 standardize as necessary.
    6.8.2   Standardization. Plpet accurately 20
 ml. standard arsenlous  oxide solution Into a
 300-ml.  Ertenmeyer  flask. Acidify slightly
 with 1:10 sulfurie acid, neutralize with solid
 sodium bicarbonate, and add about 3 g. ei-
 oeas. Titrate with the standard Iodine solu-
 tion using 6 ™i  starch  solution as Indicator.
 Batorato the  solution  with carbon dioxide
 near the end point by  adding  1  ml. of 1:10
 sulfurie acid. Continue the  tltratlon to the
 first appearance  of a blue odor which per-
 sists for 80 second*.
                ml. AB.O.X Normality A*O>
   Normality It= -
                          ml.
    6.9  Diluted Standard Iodine. Immediately
  before use, plpet 1 ml. standard iodine solu-
  tion  into  a  100-ml. volumetric  flack anil
  dilute to volume with absorMnf reagent.
  7. Procedure.
  7.1  Instruments can be constructed from
the components given here or may be pur-
chased.  If commercial Instruments are used,
follow the specific Instructions given In  the
manufacturer's  manual.  Calibrate  the  In-
strument as directed In section 8. Introduce
samples Into the system under the same con-
ditions  of pressure and flow rate  as are used
In calibration. By proper adjustments of zero
and span controls, direct  reading of ozone
concentration Is possible.
  8. Calibration.
  8.1  KI Calibration Curve. Prepare a curve
of absorbance of various Iodine  solutions
against   calculated  ozone equivalents   as
follows:
  8.1.1  Into a series of  26  ml.  volumetric
flasks, plpet 0.6. 1. 2, 3, and 4 ml. of diluted
standard  Iodine solution  (6.9). Dilute each
to the  mark  with  absorbing reagent. Mix
thoroughly, and  immediately read  the  ab-
sorbance of each at 362 run.  against unex-
posed absorbing reagent as the reference.
  8.1.2  Calculate the concentration of  the
solutions as total MB. Oi as follows:
         Total »g.O.= (N)(96)(V,)
   N=Nonnallty T, (see 6.8.2), meq./ml.
  V,=Volume of diluted  standard I, added,
        ml. (0.6,1,2,3,4).
Plot absorbance versus total  #g. O,.
  8.2  instrument Calibration.
  8.2.1  Generation o) Test Atmospheres. As-
semble  the apparatus as shown In Figure D3.
The ozone concentration  produced by  the
generator can be  varied by changing the  po-
sition of the adjustable sleeve. For calibra-
tion of ambient  air analyzers,  the ozone
source should be  capable of producing ozone
concentrations In the  range 100 to  1,000
Pg./m.'.  (0.06  to 0.6 p.p.m.) at a flow rate of
at least 6 liters per minute. At all times  the
airflow  through the generator must be great-
er than the total flow required by the sam-
pling  systems.
  8.2.2  Sampling and Analyses of Test  At-
mospheres. Assemble the  KI sampling train
as shown  In  Figure D4.  Use ground-glass
connections upstream from  the Unplnger.
Butt-to-butt connections with Tygon tubing
may be used. The manifold distributing  tbe
test  atmospheres  must be sampled slmul- ,
taneously by  the KI sampling train and  the
Instrument to be calibrated.  Check assem-
bled systems for leaks. Record the Instru-
ment response   In  nanoamperes  at  each
concentration (usually six). Establish  these
concentrations by analysis, using tbe neu-
tral buffered  potassium  Iodide  method  M
follows:
  8.3.3.1  Blank.  With ozone lamp off. flush
ttte system for  several mlnutee  to remove
residual oBone. Prpet 10  ml.  absorbing  re-
agent mto each absorber.  Draw air from  the
ozone-generating system through the sam-
pling  train at 0.3 to 1 liter/minute tor 10
minutec. T****»*ii1lfltflly transfer the  exposed
solution to a clean 1-enx oelL Determine the
abeorbance at M2  nm. against  onexposed
•beortrtng reagent M the  reference. It  the
system  blank gives an absorbanoe, continue
flushing the  o*ana generation system  unUl
»M* absortoaooe Is obtained.
  8.2.2.2  Test Atmospheres. With the ozone
lamp operating, equilibrate the system for
about  10 minutes. Plpet 10 ml.  of absorbing
reagent Into each  absorber  and  collect sam-
ples for 10  minutes In  the  concentration
range  desired for calibration.  Immediately
transfer the solutions from the two absorb'
ers to  clean 1-cxn. cells. Determine the ab-
sorbance of each  at 362 nm.  against unex-
posed absorbing reagent as the reference. Add
the absorbances of the  two solutions to ob-
tain total absorbance. Read total Mg O, from
the calibration curve (see 8.1). Calculate to-
tal volume  of air  sampled  corrected to ref-
erence conditions of 26* C.  and  760 mm. Hg.
as follows:
                  P     298
        Va=Vx - X - X10-"
                760  t+273
  Vs = Volume  of air  at  reference condi-
         tions, m.'
  V   = Volume of  air  at  sampling condi-
         tions, liters.
  P   = Barometric  pressure  at  sampling
         conditions, *""*  Hg.
  t   = Temperature at sampling conditions,
         •C.
  10-'= Conversion of liters to m.1
  Calculate ozone  concentration in p.p.m. as
follows:
                  Mg. O.
         p.p.m. O*= — — X5.10X10-*
                     VE
  8.2.3  Imtrument  Calibration Curve.  In-
strument response from the photomultlpller
tube Is ordinarily in current or  voltage. Plot
tbe  current, or  voltage  If  appropriate,
(y-axls)  for the  test  atmospheres  against
ozone  concentration  as determined  by  the
neutral buffered  potassium Iodide  method,
In p.p.m. (xisjds) .
  9. Calculations.
  9.1   If a recorder  Is used which has been
properly zeroed and  spanned,  ozone concen-
trations can be read directly.
  9.2   If the DC amplifier Is read  directly,
the reading must be  converted to  ozone
concentrations using the Instrument calibra-
tion curve (8.2.3) .
  9.3   Conversion  between  p.p.m.  and Mg./
m.* values for ozone  can be  made as follows:
        p.pjn. Oi=
                         X B.10X 10-'
  10. Bibliography.
  Hodgeson, J. A., Martin, B. E., and Baum-
gardner, R. E., "Comparison of cnemlluml-
neaoent  Methods  for  Measurement of  At-
mospheric  Ozone", Preprint,  Eastern  Ana-
lytical Symposium, New York. N.T, October,
1970.
  Hodgeson, J. A., Stevens, R. K., and Martin,
B. B., "A Stable Ozone Source  Applicable as
a Secondary Standard  for Calibration of At-
moaptierta  Monitors-,  Preprint No. 71-MO.
Instrument Society of America,  International
Oonferonoe and Bxblblt, Chicago, m., Octo-
ber. 197L
  Nederbragt,  O. W, Van der Horst. A., and
Van  Duljo, J, Katun MA. 97 (1008).  .
  Warren, Ck. J. and Baboock, O, Rex. Sal.
ftMtr.«/,380(1970).
       Mo.
                                 ROHM UOISTEI, VOL  M, NO. M«—THUMOAY, NOVEMMt JS,  1971
                                                                96
-------
22394
RULES AND REGULATIONS
                SAMPLE AIR IN
                 _-l	
 EXHAUST
 6mm
                                     ETHYLENE IN
                                  6 mm
                         - 10 mm

                          6 mm
           2 mm
            V
                         2 mm
                                    PYREX CONSTRUCTION
                                          1^-inO.D.
          PHOTOMULTIPLIER TUBE
                                      • EPOXY SEALED OPTICALLY FLAT
                                       PYREX WINDOW ON END
                   Figure D1. Detector eel I.
                                                  .97
-------
              RULES  AND REGULATIONS
                                                                          22395
             6-In. PEN-RAY
                LAMP
  ALUMINUM
BOX ENCLOSURE
                                   Figure D2, Ozone source.
                      FLOW METER
                     (0-10 llteiVmln)
                   6 liters/rain
                                                                  OZONE
                                                                  60URCE
                        FLOW
                    [CONTROLLER
                                         ¥9Tfg5|    I
                                                       SAMPLE
                                                       LJtLJ
   MANIFOLD
          JL_J
                             FIgifl»D3.
                                      •mlfoldvttem.
FEOEUU. KOIim. VOL 36, Ma tit—THU1SDAT, NOVEMKt M. 1971
                       98
-------
 BIBLIOGRAPHIC DATA
 SHEET
J. Report No.
EPA-R4-73-028c
                                                                       3. Recipient's Accession No.
4. Title and Subtitle
  GUIDELINES  FOR DEVELOPMENT OF A QUALITY ASSURANCE PROGRAM
  Reference Method for  Measurement of Photochemical Oxidants
                                                 5. Report Date
                                                  June  1973
                                                                       6.
7. Author(s)
  Franklin  Smith and A.  Carl Nelson, Jr.
                                                 8. Performing Organization Kept.
                                                   No.
9. Performing Organization Name and Address
  Research Triangle Institute
  Research Triangle Park, North Carolina  27709
                                                 10. Project/Task/Work Unit No.
                                                 11. Contract/Grant No.
                                                  EPA Durham 68-02-0598
12. Sponsoring Organization Name and Address
  Environmental Protection Agency
  National Environmental Research  Center
  Research Triangle Park, North Carolina  27711
                                                 13. Type of Report & Period
                                                    Covered Interim con-
                                                  tract  report-field docv
               ment
                                                 14.
15. Supplementary Notes
16. Abstracts
       Guidelines for  the quality  control of  the Federal  reference method for photo-
  chemical  oxidants are  presented.   These include:
            1.    Good operating practices
            2.    Directions on how  to assess data and qualify data
            3.    Directions on how  to identify trouble and improve data quality
            4.    Directions to permit design of auditing activities
            5.    Procedures for selecting action options and relating  them to costs
  This document is not a research  report.  It is for use  by operating personnel.
17. Key Words and Document Analysis.  17o. Descriptors
 Quality Assurance
 Quality Control
 Air Pollution
 Quantitative  Analysis
 Gas Analysis
 Ozone
17b. Identifiers/Open-Ended Terms
17=. COSATI F.eld/Group
18. Availability Statement
                                      19.. Security Class (This
                                        Report)
                                      	UNCLASSIFIED
                                                           20. Security Class (This
                                                              Page
                                                           	UNCLASSIFIED
21. No. of Pages
                                                                                 22. Price
FORM NTIS-33 IREV. 3-72)
                                                                                 USCOMM-OC M632-P72
-------
   'INSTRUCTIONS  FOR  COMPLETING FORM  NTIS-35 (10-70) (Bibliographic Data Sheet based on COSATI
   Guidelines to Format Standards for Scientific and Technical Reports Prepared by or  for the Federal Government,
   PB-180 600).

    1.  Report Number.  Each individually bound report shall carry a unique alphanumeric designation  selected by the performing
       organization or provided by the sponsoring organization. Use uppercase letters and Arabic numerals only.  Examples
        FASKB-NS-87 and FAA-RD-68-09-

    2.   Leave blank.

   3.  Recipient's Accession Number. . Reserved for use  by each report recipient.

   4-  Title and Subtitle.  Title should indicate clearly  and briefly the subject coverage of the report, and be displayed promi-
       nently.  Set  subtitle,  if used, in smaller type or otherwise subordinate it to main title.  When a report is prepared in more
       than one volume, repeat the- primary title, add volume  number and include subtitle for the specific volume.

   5.  Report Date. Kaeh report shall carry a date indicating at least month and year.  Indicate the basis on which it was selected
       (e.g., date of issue, date of approval, date of preparation.


   6.  Performing Organization Code.  Leave blank.

   7.  Author(s).  Give name(s) in conventional order  (e.g.,  John K. Doe, or J.Robert Doe).   List author's affiliation if it differs
       from the performing organization.

   8.  Performing Organization Report Number.  Insert if  performing organization wishes to assign this number.

   9-  Performing Organ! ration Name and Address.   Give  name, street, c it y, state, and zip code.   List no more than  two levels of
       an organisational hierarchy.  Display the name  of  the organization exactly as  it should appear  in Government  indexes such
       as  USGRDR-I.

  10.  Project/Task/Work Unit Number.  Use i lie project, tusk and  work unit numbers  under which the report was prepared.

  11.  Controct/Gront Number.  Insert contract or grant number under which report was prepared.

  12*  Sponsoring Agency  Name and  Address.  Include xip code.

  13-  Type of Report and Period Covered.  Indicate interim, final, etc., and, if applicable, dates covered.

  14.  Sponsoring Agency  Code.  Leave blank.

  15.  Supplementary Notes-   Knter  information not  included elsewhere  but  useful,  such n:-: Prepared in cooperation with .  .  .
       Translation of ...  Presented at conference of  . .  . To be published in  ...   Supersedes .  . .        Supplements . . .

  16.  Abstract.   Include a brief  (200 words or less) factual .summary  of the  most significant information contained  in the  report.
       If the report contains a significant bibliography or  literature survey,  mention it here.

  17.  Key  Words and  Document Analysis,  (o).  Descriptors.  Select from the Thesaurus of Knginccring and Scientific Terms the
       proper authorised terms that identify  the major concept of the research and  are  sufficiently specific and precise to be used
       as index entries for cataloging.
       (b).  Identifiers and Open-Ended Terms.  Use identifiers for project  names, code names, equipment designators,  etc.  Use
       open-ended terms written in descriptor form for those subjects for which  no descriptor exists.
       (c).  COSATI Field/Group.   Field and Group assignments  are to be taken from the 1965 COSATI Subject  Category  List.
       Since the majority of documents are multid isciplinary  in nature, the primary Field/Group assignment(s) will be the specific
       discipline, area of human endeavor, or type of physical object. The applicat ion(s) will be cross-referenced with  secondary
       Field/Group  assignments that  will follow the primary posting(s).

  18.  Distribution Statement.  Denote releasability to the public  or  limitation  for  reasons other than  security for example  "Re-
       lease unlimited".  Cite any availability to the public, with address and price.

  19 & 20.  Security Classification.  Do not submit classified reports to the National  Technical

  21.  Number of  Pages.   Insert the  total number of  pages, including this  one  and unnumbered pages, but  excluding distribution
       1 ist, if any.

  22.  Price. Insert the price set hy the National Technical Information Service or the Government Printing Office, if known.
FORM NTIS-35 (REV. 3-72)                                                                                   USCOMM-DC  14932-P72


                         *U.S.  Government Printing Office:  !973"746-772/*t197 Region No.  **
-------