United States Environmental Protection Agency Office of Toxic Substances Office of Toxic Substances Washington, DC 20460 EPA 560/13 80 017B December 1980 Asbestos-Containing Material: in School Buildings Guidance for Asbestos Analytical Programs Statistical Background Document ------- EPA 560/13-80-017B December 1980 ASBESTOS-CONTAINING MATERIALS IN SCHOOL BUILDINGS Guidance for Asbestos Analytical Programs Statistical Background Document by D. Lucas A. V. Rao T. Hartwell Research Triangle Institute Research Triangle Park, North Carolina 27709 EPA Contract Number 68-01-5848 Task Manager: Cindy Stroup Project Officer: Joe Carra Design and Development Branch Exposure Evaluation Division Office of Toxic Substances Washington, DC 20460 OFFICE OF PESTICIDES AND TOXIC SUBSTANCES U. S. ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, DC 20460 ------- DISCLAIMER NOTICE This report was prepared under contract to an agency of the United States Government. Neither the United States Government nor any of its employees, contractors, subcon- tractors, or their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party's use or the results of such use of any information, apparatus, product, or process disclosed in this report, or represents that its use by such third party would not infringe on privately owned rights. Publication of the data in this document does not signify that the contents necessarily reflect the joint or separate views and policies of each sponsoring agency. Men- tion of trade names or commercial products does not consti- tute endorsement or recommendation for use. 11 ------- CONTENTS Page I. INTRODUCTION 1 II. SAMPLING FRIABLE MATERIAL 3 A. Summary of the Sampling Procedure 4 B. Comparison of the Recommended Sampling Procedure with Purposive Sampling 6 C. Recommended Sample Size 8 1. Variance of Asbestos Concentration of Friable Material 8 2. Confidence Intervals 12 3. Power Calculations 16 4. Conclusions 23 III. LABORATORY QUALITY ASSURANCE 27 A. General Description of the Quality Assurance Program 27 B. Probability of Agreement Between the Results Based on the Analysis of Two Parts of a Split Sample 29 C. Sample Sizes 31 D. Initial Laboratory Quality Assurance 33 E. Procedure for Monitoring on an On-Going Basis 34 F. A Central Administrative Structure to Monitor Lab Quality 35 REFERENCES 37 APPENDIX A 39 ------- LIST OF TABLES Number Title _ _ _____ Page 1 Summary of Variability Data ........... 10 2 90% Confidence Intervals for y, when a = .0155 ............................ 14 c 3 90% Confidence Intervals for y, when ac = .04 .............................. 17 4 Power of the Test H : y = .01 Versus H,: y = y, > .01 for Selected Values of y, , when a = .0155 ................ 21 _L Ox 5 Power of the Test H : y = .01 Versus H, : y = y , > .01 for Selected Values of y, , when a = .04 .................. 22 -L v_» 6 False Positive and False Negative Rates ................................. 32 A-l Power of the Test H : y = y = .01 Versus H, : y = y, > .01 for Selected Values of MI .......................... 43 A-2 Power of the Test HQ: y = yQ = .01 Versus H y = y^ > .01 for Selected Values of y, 44 ------- I. INTRODUCTION A Guidance Document, Asbestos-Containing Materials in School Buildings: Guidance for Asbestos Analytical Programs (USEPA 1980) presents detailed sampling procedures and labo- ratory quality assurance measures for bulk samples collected in school buildings. This background document is designed to supplement the Guidance Document by presenting statisti- cal support for these procedures. In this document, the statistical basis for the proposed bulk sampling procedures is given in Chapter II, and Chapter III presents the basis of the proposed quality assurance methods. ------- -2- ------- II. SAMPLING FRIABLE MATERIAL The sampling procedure for friable material in school buildings given in Chapter 2 of the Guidance Document (USEPA 1980) is directed towards meeting the following two require- ments: (1) construction of a probability sample from which meaningful conclusions can be drawn concerning the presence or absence of asbestos and (2) provision of sampling instructions that can be carried out without undue effort and with minimum error by school personnel who are untrained in sampling. This chapter briefly summarizes the proposed sampling methodology and discusses the statistical motivation for its recommendation. The advantages of this methodology over plans involving personal judgment or convenience are given in Section II.B. Section II.C presents statistical consid- erations in determining the necessary sample size to give the amount of confidence that is of practical significance for reaching a decision as to the presence or absence of asbestos. Data collected in two studies conducted by Bat- telle are used in the sample size calculations. These data are described in (Patton et al. 1980) and (Rao et al. 1980). The variance estimation presented in Section II.C.I is a revision of that given in an early draft of this document (Lucas et al. 1980). The reasons for this revision, and its implications, are discussed in Appendix A. The revision does not change the recommended sampling procedure or the recommended number of bulk samples to be collected. -3- ------- A. Summary of the Sampling Procedure All school building areas should first be inspect- ed for friable material. The total friable material area of the school building is then partitioned into Sampling Areas using results of visual inspection, knowledge of the school building's history, and building records if available. A Sampling Area is defined to be a homogeneous area of friable material that is, all friable material in a Sampling Area is of the same type and was applied during the same time pe- riod. A decision as to the presence or absence of asbestos in the friable material is necessary for each Sampling Area. Asbestos is considered "present" in the Sampling Area if the average concentration of asbestos in that area exceeds 1%; otherwise, asbestos is considered to be "absent." A scale diagram of each Sampling Area should be prepar- ed according to the instructions given in Chapter 2 of the Guidance Document (USEPA 1980). Based on area, the number of bulk samples to be collected from the Sampling Area is determined as indicated in the table below. If the size (square Then the number (n) feet) of the Sampling of bulk samples to Area is be collected is Less than 1,000 3 Between 1,000 & 5,000 5 Greater than 5,000 7 The required number of sample locations should be selected from the Sampling Area using the random number pair proced- ure. In addition to detailed instructions on the use of a -4- ------- Table of Random Digits, a Selection of Sample Locations Worksheet is provided in the Guidance Document to aid in completion of this task. The procedure proposed for sampling is simple random sampling; that is, there is one stage of sampling, and all elements of the Sampling Area have equal probabilities of selection, where an element is a subdivision of friable ma- terial that is of the size to be collected in a sampling container. Let y be the true average asbestos concentration in the Sampling Area. Let X, , X» , ..., X be the measured asbestos concentrations in the n collected bulk samples. Assuming that the laboratory analysis and quantitation of asbestos is unbiased, the sample mean n = £ xi/n is an unbiased estimator of y . (Unbiasedness refers to the fact that in repeated sampling the distribution of X is centered around y.) An estimator of the standard deviation of the measured asbestos concentration is n 3 = CX. - X) / (n-l) (2-2) and an estimator of the standard deviation of X is s- = s/-n . (2-3) A (1 - a) -100% confidence interval for y is given by (X " t(n-l), (1-a) SX' X + t(n-l), (1-ct) sx} ' (2~4) -5- ------- where t(n_1)^ (i-a) is the ^"f^ quantile of the t-distri- 2" bution with n-1 degrees of freedom. The interpretation of the confidence interval (2-4) is that in the long run (l-a)'100% of the intervals constructed according to (2-4) will contain y, the true average asbestos concentration in the Sampling Area. Chapter 4 of the Guidance Document gives instructions for computation of the sample mean and the sample standard deviation for the reported asbestos concentrations. A 90% confidence interval for the true average asbestos concen- tration in the Sampling Area is formed. A Statistics Com- putations Worksheet is provided for use in performing these calculations. The following rule is presented for reaching a decision as to the presence or absence of asbestos in the Sampling Area, where asbestos present is taken to mean that the average concentration of asbestos exceeds 1%. (.1) If the entire confidence interval is below 1%, then conclude asbestos absent; (2). If the entire confidence interval is above 1%, then conclude asbestos present; or (.3) If the confidence interval contains 1%, then there remains uncertainty as to the presence or absence of asbestos. B. Comparison of the Recommended Sampling Procedure with Purposive Sampling In one form of purposive sampling the person se- lecting sample locations will, during visual inspection of 6 ------- the Sampling Area, classify certain areas of the friable material as "representative" and then collect bulk samples from these locations. Attempting to make inferences con- cerning y, the true average percentage of asbestos in the Sampling Area/ from data collected in such a manner will lead to many difficulties. First, the judgment of the per- son selecting sample locations is a factor and will differ from person to person, as discussed in (Raj 1968). The probability that a given element of the Sampling Area will be selected is unknown. In this situation it is not possi- ble to determine the distribution of X, the estimator of y. The sampling error cannot be objectively determined, and the accuracy of the resulting estimator cannot be assessed. The above comments also apply to selecting sample lo- cations haphazardly or according to convenience. In conven- ience sampling, all locations not deemed convenient by the person selecting sample locations have probability zero of being included in the sample. Making inferences for the entire Sampling Area from such a sample cannot be statisti- cally justified or defended. Under certain conditions, ei- ther haphazard or convenience sampling may happen to give good results. However, due to the lack of structure of the methods, there is no way of predicting how frequently good results will occur. In the proposed procedure for sampling friable materi- al, it is known that each element of the Sampling Area has -7- ------- an equal probability of selection. An estimate of sampling variability can be calculated from the data obtained. Meth- ods of statistical inference can be used to estimate the true average percentage of asbestos in the Sampling Area, give the precision of this estimate, and test hypotheses concerning the presence or absence of asbestos in the Samp- ling Area. C. Recommended Sample Size 1. Variance of Asbestos Concentration of Fria- ble Material Determining the necessary sample size con- sists of finding the minimum number of observations that will give the amount of precision in inferences about y, the true average concentration of asbestos in the Sampling Area, that is of practical significance for reaching a decision. The desired precision in inferences about y will be discuss- ed in the following sections. Procedures for sample size determinations first require an estimate of the variability to be expected among the observations of asbestos concentra- tion. The variance among observations of asbestos concentra- tion can be partitioned into the following two components: 2 (1) A variance component a attributable to the vari- ability in true (not measured) asbestos concentra- tion among elements of the Sampling Area, where an element is a subdivision of friable material that is of the size to be collected in a sampling con- tainer. -8- ------- (2) A variance component a attributable to the vari- ability in measured asbestos concentration among measurements on a bulk sample of friable material. To obtain numerical estimates of the above variance components, data collected in two studies conducted by Bat- telle are used. These currently available data are very limited, and for future study it would be of interest to have more extensive data. In the Battelle Duplicate Analy- sis Study, described in CRao et al. 1980), asbestos concen- tration measurements were made on split-samples of friable material collected at three locations from each of five buildings. Eight buildings were included in the Battelle Bulk Sampling Variability Study, described in (Patton et al. 1980). Within each of these eight buildings, bulk samples were collected at four randomly selected locations within a 5,000 square feet area of homogeneous ceiling material. Each bulk sample was then split into four parts prior to laboratory analysis. For each area included in these two studies, measurement variability and variability due to sampling location were estimated using analysis of variance techniques. These estimates are displayed in Table 1. It can be seen from Table 1 that estimates of a c (square root of the variance component due to location with- in the Sampling Area) range from 0 to 15.22%. The average a for the thirteen areas studied is 1.55%. Note that there is no reason to assume that a is constant for various types -9- ------- Table 1. Summary of Variability Data Data Sourcea (1) (1) (1) (1) (.1) (21 (.2) (2). (2) (2) (2) (2) t2) Sampling Area ID Number 1 2 3 4 5 6 7 8 9 10 11 12 13 Number of Observa- tions 6 6 6 6 6 16 16 16 16 16 16 16 16 Average Asbestos Concentra- tion, X .075 .397 .042 .545 .548 .481 .116 .131 .134 .028 .004 .184 .289 Standard Devi- . ation Estimates yv CTc .0204 .0000 .0000 .0000 .1522 .0000 .0198 .0000 .0000 .0051 .0039 .0000 .0000 /> °e .0204 .0619 .0385 .1866 .0261 .0661 .0725 .0750 .1197 .0360 .0122 .1008 .1350 Data Source (1) is the Battelle Duplicate Analysis Study, described in (Rao et al. 1980). Data Source (2) is the Battelle Bulk Sampling Variability Study, described in (Patton et al. 1980). ^ a is the square root of the variance component due to C A location within the Sampling Area, and a is the square root of the variance component due to laboratory mea- surement. -10- ------- of friable material. In fact, it is suspected that the var- iability due to location is different for different types of friable material. Some friable materials may be fairly het- erogeneous with respect to asbestos concentration, while others may not be. In the sample size calculations of the l*\ XN. following sections, a = .0155 and a = .04 are considered. C_x O /\ The value a = .04 was arbitrarily selected for considera- tion; it is thought possible that a values in the neighbor- c hood of .04 may be encountered in practice. In (Patton et al. 1980) it was observed that there ap- pears to be a strong positive relationship between asbestos concentration and measurement variability when asbestos con- centration is less than approximately 30%. The following linear regression model is formulated to describe this rela- tionship: ae = 30 + BI y , where BQ and g, are parameters to be estimated. The least squares estimates of these parameters are computed using those points from Table 1 for which X is less than 30% (X. is used to estimate y), yielding the following estimated re- gression equation: ae = .0177 + .45 y, (2-5) with a correlation coefficient of .90. The estimate of to- tal variance among observations of asbestos concentration 222 a = a + a is then given by C G -11- ------- ~2 "2 "2 a = a + a c e a2 = a2 + (.0177 + .45y)2, (2-6) C^ "2 2 ~2 2 where a = (.0155) and a = (.04) are considered. c c It should be noted that there are currently available no data that can be used to examine the relationship between total area and variability of asbestos concentration within the Sampling Area. As mentioned, the Battelle Bulk Sampling Variability Study included areas of 5,000 square feet. Some increase in variability as the area increases might be ex- pected due to increased time span of material application, increased number of batches of material used, or other fac- tors depending upon the application method. However, it is not thought likely that such a relationship between total area and variability would be directly proportional. In other words, doubling the total area would not require doub- ling the sample size to obtain equal estimation precision. Because of this, the rule of sampling at one location for every 5,000 square feet of friable material, recommended in (USEPA 1979), is questionable. The factor of area, however, is taken into account in the sample size recommendations pre- sented in Section II.C.4 of this document. 2. Confidence Intervals Let n denote the number of bulk samples col- lected from a Sampling Area, and let X be the average mea- sured asbestos concentration. According to the discussion -12- ------- of Section II.C.I, assume that the estimate of the standard deviation is s = Ja^ + (.0177 + .45y)2, 2 where a is the variance component due to location within the Sampling Area. Values of this component considered here 2 222 are a = (.0155) and a = (.04) . The standard deviation c c of X is estimated by S:T = s/Jn, and a (1-a) -100% confidence interval for y is given by (X t(n-l)., (1-a) SX' X"l"t(n-l), (1-a) SX} ' as discussed in Section II.A. For selected values of X and n, 90% confidence intervals for y are displayed in Table 2 (a = .0155) and Table 3 (a = .04). C C Consider Table 2, which assumes that cr = .0155. If C the variance model given in Section II.C.I describes the true situation for a Sampling Area fairly well, then the expected 90% confidence interval for y is (.009, .091) when X = .05, computed from measurements on 5 bulk samples. In other words, the true average asbestos concentration is ex- pected to be between .9% and 9.1%, with 90% confidence. (The phrase "with 90% confidence" means that in repeated sampling, 90% of the intervals constructed in this way will contain y.) Again, with X = .05, the expected 90% confidence interval for y.is C.025, .075) when 10 bulk samples were col- lected. The length of this confidence interval is 7.5% - -13- ------- Table 2. 90% Confidence Intervals for y, when a = .0155 c V n\ 2 3 5 7 10 15 20 30 50 .005 .060a .033a .022a .019a a .016a .014a .013a .ona .oioa .010 .069a .039a .029a .025a .022a .019a (.000, .020) (..002, .018) (.004, .016) .015 .078a .047a -035a .031a a .028a (.002, .028) (.004, .026) (.006, .024) (.008, .022) .020 .087a .054a .041a .037a (..002, .038) (.006, .034) (.008, .032) (.010, .030) (.013, .027) \ X A 2 3 5 7 10 15 20 30 50 .030 .106a .068a .054a (.004, .056) (.010, .050) (.014, .046) (.017, .043) (.019, .041) C. 022, .038) .040 .125a .082a (.003, .077) (.011, .069) (.017, .063) (.022, .058) (.025, .055) (..028, .052) (.031, .049) .050 .144a .097a (.009, .091) (.018, .082) (.025, .075) (.030, .070) (.033, .067) (.037, .063) (.040, .060) .075 .192a .133a (.024, .126) (.036, .114) (.044, .106) (.051, .099) (.054, .096) (.058, .092) (.062, .088) continued -14- ------- Table 2. (continued) Y n\ 2 3 5 7 10 15 20 30 50 (. (.. c. (. (. (. c. 038, 053, 063, 071, 075, 080, 085, .100 .240a .170a .162) .147) .137) .129) .125) .120) .115) C.004, (.067, (.086, (.100, (.111, (-117, (.123, (..129, .150 .338a .296) .233) .214) .200) .189) .183) .177) .171) (.017, (.096, (.120, (.137, (.151, (.158, (-166, (.174, .200 .437a .383) .304) .280) .263) .249) .242) .234) .226) (.029, (.125, (.154, (.174, (.190, (.199, (..209, (.219, .250 .535a .471) .375) .346) .326) .310) .301) .291) .281) a 90% Upper Confidence Bound. (.90% Confidence Interval contains zero.) The entries in the above table were calculated according to (* ~ fc(n-l) , .95 SX' X + t(n-l), .95 SX* ' where t, ,« g5 is the 95th quantile of the t-distri- bution with n-1 degrees of freedom, n is the number of bulk samples collected, and s^ - J(..0155)2 + (.0177 + .45X)2 / J~~n~~. This variance model is based on data in (.Patton et al. 1980) and (Rao et al. 1980) . -15- ------- 2.5% = 5.0%, as compared to a length of 9.1% - .9% = 8.2% for a sample size of 5. It can be seen from the table that the length of the confidence interval decreases as the num- ber of bulk samples collected is increased. The desired confidence interval length is a factor in choosing an appro- priate sample size. Table 3 displays confidence intervals for p, again us- ing the variance model of Section II.C.I, but with a = O .04. This larger variability due to location leads to longer confidence intervals, for the same sample size and the same X. As mentioned previously, it is thought likely that the variability due to location is different for dif- ferent friable materials. Considering Table 3, the expected 90% confidence interval for y is (.017, .083) when X = .05, computed from measurements on 10 bulk samples. When 5 bulk samples were collected and X = .05, the expected 90% upper confidence bound is .089. The interpretation of a 90% 4 upper confidence bound is that in repeated sampling, 90% of the bounds constructed in this way will exceed the true av- erage asbestos concentration. A 90% upper confidence bound is presented in Table 2 or 3 whenever the corresponding 90% confidence interval includes 0. 3. Power Calculations It is desired to decide whether or not asbes- tos is present in the Sampling Area, where "asbestos present" is defined to occur when the average asbestos concentration -16- ------- Table 3. 90% Confidence Intervals for y, when a = .04 V -\ 2 3 5 7 10 15 20 30 50 .005 .102a -054a .036a .029a .025a .021a .018a .016a .013a .010 .noa .060a .041a .035a ,030a .026a .024a .021a .018a .015 .117a .066a .047a .041a .036a .031a .029a (.000, .030) (.004, .026) .020 .125a .072a .053a .046a .041a .037a (.001, .039) (.005, .035) (.009, .031) V n\ 2 3 5 7 10 (.001, 15 (.007, 20 (.010, 30 (.014, 50 (0.18, .030 .140a .085a =a .065a .058a .059) .053) .050) .046) .042) .040 .157a .098a .077a (.001, .079) (.009, .071) (.016, .064) (.019, .061) (.023, .057) (.027, .053) .050 .173a .112a .089a (.008, .092) (.017, .083) (..024, .076) (.028, .072) (.032, .068) (.037, .063) .075 .217a .146a (.013, .137) (..027, .123) (.037. ,113) (.045, .105) (.050, .100) (.055, .095) (..060, .090) continued -17- ------- Table 3. (continued) V n\ 2 3 5 7 10 15 20 30 50 (. I. (.. (. (. (. t- 029 045 057 066 071 077 082 .100 .262a .181a ,.171) ,.155) ,.143) ,.134) ,.129) ,.123) ,.118) (.060, (.081, (.095, (-.107, (.114, (.121, (.128, .150 .355a .252a .240) .219) .205) .193) .186) .179) .172) (.006, (.090, (.116, (.133, (-148, (-156, (.164, (.173, .200 .450a .394) .310) .284) .267) .252) .244) .236) .227) (.020, (.120, (.150, (.171, (.188, (.197, (.208, (.218, .250 .546a .480) .380) .350) .329) .312) .303) .292) .282) 90% Upper Confidence Bound. (90% Confidence Interval contains zero.) The entries in the above table were calculated according to CX ~ Hn-l), .95 SX' * + t(n-l), .95 SX) ' where t, , v Q[. is the 95th quantile of the t-distri- in -Lji / . y D bution with n-1 degrees of freedom, and S£ = J (.04)2 + (.0177 + .45X)2 / J~n . This variance model is based on data in (Patton et al. 1980) and (JRao et al. 1980). -18- ------- exceeds 1%. Putting this in terms of statistical hypothesis testing, it is desired to test the null hypothesis HQ: y <_ .01 versus the alternative H,: y > .01, where y is the true average concentration of asbestos in the Sampling Area. A test can be constructed according to the following decision rule: Reject H if X > .01 + z(1_a) a Accept HQ if X <_ .01 + Z(1_ . a/-\J~n, where Z,,_ . is the (1-a) quantile of the normal distribu- tion, a is the standard deviation when H is true, n is the number of bulk samples collected, and X is the average of the measured asbestos concentrations. (Under the assumptions 2 2 presented in Section II.C.I, a = a + (.0177+.45y) , where \*f 2 a is the variance component due to location within the Samp- O ling Area.) The size of the test, denoted by a, is the probability that the test leads to rejection of the null hypothesis H : y <_ .01 when in fact the null hypothesis is true. Denote by B(y-,) the probability that the null hypo- thesis is accepted when actually the alternative y = y, > .01 is true. The power of the test under y-, is defined as l-g(y,), the probability that the test correctly rejects H when y = y. > .01. In other words, the power of the test under y-, is the probability of correctly concluding that asbestos is present when the average asbestos concentration is y , some concentration greater than .01. The power of the test is an important consideration when deciding how -19- ------- many bulk samples to collect. A sufficient number of bulk samples to give good (i.e., high) power for alternatives of interest (.y-^'s) is desired. Let a, the size of the test, be .05. This means that the probability of the test concluding that asbestos is pre- sent when in fact y <_ .01 is 5%. For this a and the vari- ance model presented in Section II.C.I, Tables 4 and 5 give the power of the test for selected values of n and y, . In 2 2 Table 4 it is assumed that a = (.0155) , and in Table 5 it 22 2 is assumed that o = (..04) , where a is the variance com- c c ponent due to location within the Sampling Area. Consider- ing Table 4, suppose 5 bulk samples were collected and actually y = y, = .05. Then, if the variance assumptions describe the situation fairly well, the power is expected to be .851. In other words, with probability .851 the test will conclude that asbestos is present when the true average asbestos concentration is .05. In this case the test will incorrectly conclude that asbestos is absent with probabil- ity .149. It can be observed in Table 4 that, for a fixed alternative y-, , the power increases as the sample size n increases. Also, for a fixed sample size, the power in- creases as y, increases; that is, the test can better dis- tinguish the difference between .01 and y, when y, is far- ther from .01. -20- ------- Table 4. Power of the Test H : y = .01 Versus H, : y = y, > .01 for Selected Values of y,, when a = .0155 V1 "\ 2 3 5 7 10 15 20 30 50 .015 .098 .107 .124 .139 .160 .192 .221 .276 .374 .020 .162 .189 .236 .279 .337 .425 .502 .629 .801 .025 .238 .286 .368 .441 .535 .660 .753 .874 .969 .030 .320 .388 .502 .595 .704 .827 .901 .969 .997 .040 .478 .575 .718 .814 .902 .967 .989 .999 >.999 .050 .610 .717 .851 .922 .971 .995 .999 >.999 .060 .710 .813 .922 .968 .992 .999 >.999 V1 n\ 2 3 5 7 10 15 20 30 50 .070 .783 .875 .959 .987 .998 >.999 .080 .835 .915 .977 .994 .999 >.999 .090 .872 .941 .987 .997 >.999 .100 .900 .958 .992 .999 >.999 .125 .941 .980 .998 >.999 .150 .962 .989 .999 >.999 .200 .980 .996 >.999 The entries in the above table were calculated according to Power of the Test H : y = y = .01 versus = 1 - $ {(..01 - y-L + 1.645 a yo Afli) / a /J~~n} yl * where a = J(.0155)2 + (.0177 + .45y.)2, yi * *- and ${z} is the cumulative distribution function of a standard normal random variable. The variance model is based on data in (Patton et al. 1980) and (Rao et al, 1980). -21- ------- Table 5. Power of the Test H : y = .01 Versus H,: y = y > .01 for Selected Values of y,, when a =.04 V1 n x 2 3 5 7 10 15 20 30 50 .015 .073 .078 .086 .093 .103 .117 .130 .154 .198 .020 .102 .114 .136 .156 .183 .224 .263 .336 .463 .025 .137 .160 .200 .236 .287 .365 .435 .556 .734 .030 .178 .212 .274 .330 .407 .518 .611 .751 .904 .040 .271 .333 .440 .531 .643 .778 .864 .952 .995 .050 .371 .459 .599 .706 .817 .920 .966 .994 >.999 .060 .470 .576 .729 .829 .917 .976 .993 >.999 V1 n\ 2 3 5 7 10 15 20 30 50 .070 .561 .675 .824 .906 .965 .993 .999 >.999 .080 .639 .754 .888 .950 .985 .998 >.999 .090 .705 .815 .929 .973 .994 >.999 .100 .758 .861 .955 .986 .998 >.999 .125 .851 .930 .985 .997 >.999 .150 .904 .962 .994 .999 >.999 .200 .954 .986 .999 >.999 The entries in the above table were calculated according to Power of the Test H : y = y = .01 versus yo = *01 = 1 - $ (C-01 - y, + 1.645 a /J n) / a AHi} J- yQ v y-L v where a = Jt.04)2 + (.0177 + .45vi) , and ${z} is the cumulative distribution function of a standard normal random variable. The variance model is based on data in (Patton et al. 1980) and (Rao et al. 1980). -22- ------- 4. Conclusions The confidence interval calculations of Sec- tion II.C.2 can be used in sample size determination upon establishing the following: (1) the confidence interval length that is of practical significance for estimation and (2). the maximum risk that can be tolerated that the confi- dence interval will not include y. The confidence intervals given in Section II.C.2 are 90% confidence intervals; that is, in the long run 90% of the intervals so constructed will contain y. The power calculations of Section II.C.3 can be used upon establishing the following: (.1) the magnitude of real difference that is of practical significance for detection, (2) the maximum risk that can be tolerated in concluding asbestos is present Cy > .01) when actually there is none (y <_ .01) and (3) the maximum risk that can be tolerated in not detecting presence of asbestos when actually asbestos is present. It is very difficult to assess these factors. In considering di, the relationship between asbestos concen- tration and level of danger to those in the school building is not presently completely understood. Additional factors that increase exposure risk such as water damage and acces- sibility will be considered in making a decision as to the necessity of corrective action. In the power calculations of Section II.C.3, the risk of concluding that asbestos is present when in fact it is not is set at 5%. The power -23- ------- (probability of concluding asbestos is present when it actually is present) for selected sample sizes and alter- native y^s is displayed in Tables 4 and 5. A reasonable requirement is power of at least 90% for alternatives of interest. It is emphasized that a small sample size giving insufficient power would lead to further questioning whenever the test "concludes" that no asbestos is present. Such a situation is obviously undesirable, especially in light of the harmful effects undetected asbestos may be having on school children. It may be that alternatives y, > .05 or y1 > -10 are of interest, and that the sample sizes (expense and effort) necessary for sufficiently powerful tests against alternatives y1 <_ .05 or y, <_ .10 are not justified by pre- sently known risks to health. An additional consideration is the variance of asbestos concentration. As discussed in Section II.C.I, the variance used in calculations for sample size determination is esti- mated using presently available data to the extent possible. This can be refined when more data become available. Taking into account all the factors discussed above, the number of recommended sample locations for a Sampling Area is given in the table below. If the size (square Then the number (n) feet) of the Sampling of samples to be Area is collected is Less than 1,000 3 Between 1,000 & 5,000 5 Greater than 5,000 7 -24- ------- The relationship between area and recommended sample size is in response to the considerations discussed at the end of Section II.C.I. There are no data presently available that could be used to examine the relationship between total area and variability of asbestos concentration within the area of friable material. The proposed procedure is considered the minimal procedure (with respect to effort and expense) capable of producing adequate results for the estimation and testing problems concerning presence or absence of asbestos in friable material. A comparison is given in the table below of the pro- posed sample size and the sample size determined by the rule of sampling at one location for every 5,000 square feet of friable material, previously recommended in (USEPA 1979). Size of Sampling Proposed Sample Size Based on Area (Square Feet) Sample Size 5,000 Square Feet Rule 750 3,000 7,500 20,000 40,000 75,000 125,000 3 5 7 7 7 7 7 1 1 2 4 8 15 25 It can be seen that, for larger areas, the proposed guidance results in the collection of fewer bulk samples than recom- mended by earlier EPA guidance. -25- ------- -26- ------- III. LABORATORY QUALITY ASSURANCE The laboratory quality assurance procedures given in Chapter 3 and Appendix B of the Guidance Document (USEPA 1980). are designed to ensure reliable results for laboratory analyses of bulk samples. In particular, Chapter 3 of the Guidance Document presents procedures to monitor laboratory results on an on-going basis while Appendix B is designed to evaluate the initial performance of an unknown laboratory (.i.e., a laboratory not choosen from the list given in Ap- pendix A of the Guidance Document). In general, the on- going and initial quality assurance evaluation procedures are the same except for the number of split-samples analyzed. This chapter gives a brief summary of the concepts underlying the suggested quality assurance procedures and then presents the statistical bases for the number of split- samples recommended in practice to carry out the quality assurance program. A. General Description of the Quality Assurance Program Results of the analysis of a number of bulk split- samples form the basis of the proposed laboratory quality assurance procedures. For the purpose of this discussion, a split-sample is defined as the two parts of a sample. In general, certain number of split-samples will be sent to the laboratory(s) performing the asbestos analysis; the results of the analysis of these samples will be analyzed to deter- -27- ------- mine whether the laboratory(s) is capable of performing the analysis to the desired level of accuracy. This general procedure applies either initially (i.e., for an unknown lab) or for monitoring a laboratory's performance over time. Specifically, the laboratory report for each sample will include whether the asbestos level in that sample is above a specified level or not (a yes or a no). This data will be analyzed to determine the extent of agreement in the results of the split-samples. The laboratory will be con- sidered to be performing satisfactorily if the number of split-sample disagreements is less than a specified number referred to in this report as the critical number of dis- agreements. This general split-sample procedure is appli- cable for the following situations: (1) When both parts of the split-samples are sent to the same laboratory; (2) When the two parts of the split-sample are sent to different laboratories; (In this case, disagree- ment in the results is indicative of difficulties in one or both of the laboratories.) (3) When the decision is whether asbestos is present in the sample or not; and (4) When the decision is whether the level of asbestos in the sample is above a predetermined value or not. -28- ------- For example, suppose the quality assurance decision is to be based on five split-samples. Then if the conclusions for two or more of these five split-samples are different (be- tween the two splits) , the laboratory procedure is not sa- tisfactory and should be investigated. Following an inves- tigation, when the problem is corrected, all samples analy- zed since the last time the laboratory was determined to be in a satisfactory state should be re-analyzed. The critical number of disagreements allowed for a laboratory depends on the probability of agreement between the results of the analysis of the two parts of the split- sample. The algebraic expression for this probability is discussed in the next section. B. Probability of Agreement Between the Results Based on the Analysis of Two Parts of a Split Sample Let C denote the probability of a positive result when the sample is truly positive (has asbestos) and C de- note the probability of a negative result when the sample is truly negative. Let P (A) denote the probability of the sample being truly positive. If the two analyses are sta- tistically independent, then the probability p of observing an agreement in the results of the two analyses is given by p = Cp + d-Cp) P(A) + cn + d-Cn) 1-P(A). (3-1) If the process of selecting samples is heavily biased to- wards positive samples, then -29- ------- p * cp2 + (1-cp12- If C is very close to 1, i.e., positive samples can be classified as positive with near certainty then In this case, in order for p to be equal to 0.90, C should be of the order 0.95. The above assumptions are used to calculate the number of split-samples required for the quality assurance program. These sample sizes are calculated assuming p = 0.90. Note, if C or C changes with the level of asbestos in the sample, then the assumption of a constant 'p1 implicitly assumed in arriving at' the quality assurance procedures giv- en below is not strictly valid. If, for example, for bor- der line samples C is small and the process of selecting samples is biased toward borderline samples, then one would expect more disagreement between the results of such split- samples than in other situations; p will be smaller than 0.9 in such a case. If the two parts of the split-sample are analyzed in two different laboratories, the above discussion is still appropriate if C and C are the same for both the labs. If they are different for the two labs, the expression for p can be easily modified to account for these differences. -30- ------- C. Sample Sizes Let N be the number of split samples tested in the laboratory and let d denote the number of samples for which there was disagreement. Further let us assume we would like to test the null hypothesis that p = 0.90 against the alter- native hypothesis that p = 0.70. For the purposes of this discussion, we will define the false positive rate as the probability of rejecting p = 0.9 when p is really =0.9 (in statistical terms this is usually referred to as the signi- ficance level). In other words, the false positive rate is the probability of incorrectly rejecting an acceptable batch of bulk samples associated with the split-samples. The false negative rate is defined here as the probability of accepting p = 0.9 when p is really = 0.7 (this is usually referred to as 1-power). The false negative rate is the probability of incorrectly accepting an unacceptable batch of bulk samples. Table 6 gives the false positive and false negative rates for several values of the critical number of split-sample disagreements (d) for sample size (N) equal to 5, 10, 15, 20 and 25. The procedure to be followed in an actual situation is to reject the null hypothesis if the observed number of dis- agreements is equal to or exceeds the critical number of disagreements d for the appropriate sample size. For ex- ample, consider the case where N=15 and the critical number d selected is 5. If the observed number of disagreements is -31- ------- Table 6. False Positive and False Negative Rates Corresponding to Various Critical Numbers of Split-Sample Disagreements No. of Split Samples (N) 5 10 15 20 25 Critical Number of Split- Sample Disagree- ments (d) 1 2 3 2 3 4 5 3 4 5 6 3 4 5 6 7 4 5 6 7 8 False Positive Rate For p=0.90 (significance level) 0.4095 0.0815 0.0086 0.2639 0.0702 0.0128 0.0016 0.1841 0.0556 0.0127 0.0022 0.3231 0.1329 0.0432 0.0127 0.0022 0.2364 0.0980 0.0334 0.0095 0.0023 False Negative Rate For p=0.70 (1-power) 0.1681 0.5282 0.8369 0.1498 0.3828 0.6493 0.8497 0.1268 0.2969 0.5155 0.7216 0.0355 0.1071 0.2375 0.5155 0.7216 0.0332 0.0905 0.1935 0.3407 0.5118 -32- ------- 5 or more, the procedure will call for rejecting p=0.90 and accepting p=0.7. The decision on sample size N and the critical number of split-sample disagreements d should be arrived at by tak- ing into consideration the cost of testing, the acceptable level (p) of agreement, the false positive rate, and the false negative rate for the appropriate alternate hypothe- sis. The usual statistical practice is to choose a critical number for which (1) the false positive rate does not exceed 0.05 or .10 and the false negative rate does not exceed 0.20 and (2) the false negative rate is the smallest among all those satisfying criterion 1. Using this guideline, if the null hypothesis is p=0.9 and the alternate hypothesis is p=0.7, and the split-sample size is 25, then choosing 6 for the critical number of split-sample disagreements is satisfactory. Similarly, the choice of 2, 4, 5, and 6 for samples of size 5, 10, 15, and 20 respectively is satisfactory. Accordingly, the following table provides the suggested critical number of split-sample disagreements for different sample sizes. Critical Number of Split-Sample No. of Split-Samples Disagreements 5 2 6 to 8 3 9 to 14 4 15 to 20 5 21 to 25 6 -33- ------- D. Initial Laboratory Quality Assurance The quality assurance procedure given in Appendix B of the Guidance Document recommends a sample of size 25 for initial quality assurance except in the case of a school or school system where a very small number of samples will be analyzed (i.e., less than 25 samples). With 25 split- samples the critical number of split-sample disagreements is 6. That is, if 6 or more disagreements are noted in an ana- lysis of 25 split samples, the procedure used in the labora- tory Cs)_ involved is unacceptable. The laboratory should be rejected if there are no satisfactory explanations for the large number of disagreements. A minimum sample size of 5 split-samples is recommended initially. Notice from Table 6 that in this case the prob- ability of incorrectly accepting the null hypothesis is high (.0.5282). This suggests that it would be better for schools to combine their efforts for determining the initial quality of the laboratory Cs) . E. Procedure for Monitoring on an On-Going Basis The recommended procedure (for school systems with more than 100 samples) is to decide on the basis of the re- sults of sets of 20 split-samples (see Figure 3.3 of the Guidance Document). If 5 or more disagreements are noted in a set of 20 split-samples, the recommendation is to inves- tigate the laboratory procedure. The false positive rate and false negative rate for this procedure (under the assump- tions) are respectively 0.0432 and 0.2375 (see Table 6). -34- ------- If the null hypothesis is true, then this procedure will call for checking the laboratory incorrectly in 1 out ^ of every 23 split-sample sets of size 20 (1/23 = 0.04); on the other hand, if the alternative hypothesis is really true it will call for checking the lab (correctly!!) in 3 out of every 4 sets of split-samples of size 20 (3/4 = 1 -.2375). In the case of school systems with a limited number of samples to be analyzed (.less than 100) , the procedure recom- mended is to decide on the basis of results of 5 split-sam- ples (see Figures 3.1 and 3.2 of the Guidance Document). If 2 or more disagreements are noted in a set of 5 split-sam- ples, the recommendation is to investigate the laboratory procedure. Note that if the null hypothesis is true, this procedure will call for checking the laboratory incorrectly in 1 out of 12 split-samples of size 5 (1/12 = .0815); on the other hand, if the alternative hypothesis is really true it will call for checking the lab (correctly) in 1 out of s^ every 2 or 3 sets of split-samples of size 5 (1/2 = 1 -.5282) The sampling rate for monitoring on an on-going basis for school systems with more than 100 samples is 1 in 5 split-samples initially up to 100 samples and then at a re- duced rate of 1 in every 10 samples thereafter (see Figure 3.3 of the Guidance Document). F. A Central Administrative Structure to Monitor Lab Quality The quality assurance procedure as is proposed calls for every school or school system to monitor labora- -35- ------- tory quality. This- results in an enormous multiplication of quality assurance efforts. The effort in terms of the over- all number of split-samples required to be analyzed can be reduced considerably if the quality assurance efforts are centralized. As an example, consider a group of six schools with expected number of samples of 10, 20, 30, 40, 50, and 60. If each school requires independent initial and on- going quality assurance procedures, then 151 of the 210 sam- ples should be split-samples. If initial quality assurance is not required at all, then 46 of the 210 samples should be split-samples. However, if the group combines their quality assurance program, then with initial quality assurance the group will include 51 split-samples (compared 151) and with- out it they will include only 21 (compared to 46) split- samples. The increased efficiency achieved by centralizing the quality assurance program would result in a significant sav- ings in the overall cost of the asbestos analytical program for a school district. It would require, however, the com- mitment or designation of an individual with overall respon- sibility for the program. -36- ------- REFERENCES Lucas D, Hartwell T, Rao AV. Research Triangle Institute, 1980. Background Document. Asbestos-Containing Materials in School Buildings: Guidance for Asbestos Analytical Programs. Draft Report. Washington, DC: Office of Pesticides and To-xic Substances, U.S. Envi- ronmental Protection Agency- Contract no. 68-01-5848. Patton JL, Price BP, Ogden JS. Battelle Columbus Labora- tories. 1980. Asbestos in Schools: Bulk Sampling Variability Study. Draft Report. Washington, DC: Office of Pesticides and Toxic Substances, U.S. En- vironmental Protection Agency. Contract no. 68-01- 3858. Raj D. 1968. Sampling Theory. New York: McGraw-Hill Book Company - Rao AV, Myers LE, Lentzen DE, Hartwell TD. Research Tri- angle Institute. 1980. Analysis of Battelle Bulk Asbestos Duplicate Samples. Draft Report. Washington, DC: Office of Pesticides and Toxic Substances, U.S. Environmental Protection Agency. Contract no. 68-01- 5848. USEPA. 1979. U.S. Environmental Protection Agency- Office of Toxic Substances. Asbestos-Containing Materials in School Buildings: A Guidance Document, Part 1. USEPA. 1980. U.S. Environmental Protection Agency. Office of Toxic Substances. Asbestos-Containing Materials in School Buildings: Guidance for Asbestos Analytical Programs. Washington, DC: Office of Toxic Substances, USEPA. EPA 560/13-80-017A. -37- ------- -38- ------- APPENDIX A REVISIONS IN SECTION 2.3.1 -39- ------- -40- ------- A. REVISIONS IN SECTION II.C.I Estimation of the variability to be expected among the observations of asbestos concentration is described in Sec- tion II.C.I. The variance estimation differs from that pre- sented in an early draft of this document (Lucas et al. 1980). The following two changes have been made: (1) Data from (Patton et al. 1980) are used, in addition to data from (Rao, Myers et al. 1980) , to estimate the variance compon- ents. (2) A variance component attributable to variability among laboratories is not included in the variance model. This appendix discusses the reasons for these changes and their implications. In the early draft of this document, data from the Bat- telle Duplicate Analysis Study, described in (Rao, Myers et al. 1980), were used to estimate measurement variability 2 2 (a ) and variability due to sampling location (a ). After 6 C preparation of the early draft, data from the Battelle Bulk Sampling Variability Study (Patton et al. 1980) became available. This study, designed to estimate measurement variability and variability due to sampling location, gives more extensive information than the Duplicate Analysis Study. The results are used in the calculations of Section II.C.I to give better variance component estimates. The variance component model in the early draft includ- ed a component attributable to variability among laborator- ies. This component was estimated using data made available -41- ------- to EPA by the Bureau of Mines (Rao, Hartwell et al. 1980). For most Sampling Areas, however, it is thought that all bulk samples will be sent to the same laboratory (with the exception of the quality assurance procedures described in Chapter III of this document). When only one laboratory is involved, it is not appropriate to include a variance com- ponent attributable to variability among laboratories. Thus, this component was omitted from the model, and the 2 2 revised model includes only a and a . c e Tables A-l and A-2 are power tables from (Lucas et al. 1980). The calculations were based on data from (Rao, Hartwell et al. 1980) and (Rao, Myers et al. 1980), and the variance model included a laboratory variability term. The size of the test, defined in Section II.C.3, was set equal to .10. (For Tables 4 and 5 in Chapter II, the size was set equal to .05.) Under the assumptions of Table A-l (smaller laboratory variability), power of .913 is expected when 3 bulk samples are collected and actually y = .05. In other words, with probability .913 the test will correctly conclude that asbestos is present, when 3 bulk samples are collected and y = .05. Under the assumptions of Table A-2 (larger laboratory variability), power of .914 is expected when 3 bulk samples are collected and y = .15. Power of .952 is expected when 5 bulk samples are collected and y = .10. Tables A-l and A-2 can be compared with Tables 4 and 5 in Chapter II of this document. Under the assumptions of -42- ------- Table A-l. Power of the Test H : y = y = .01 versus H, : y = y, > .01 for Selected Values of y-, (assuming smaller laboratory variability and the variance model from (Lucas et al. 1980)) X n \ 3 5 7 10 12 15 17 20 30 50 X1 3 5 7 10 12 15 .015 .379 .441 .493 .558 .595 .644 .673 .712 .810 .916 .050 .913 .968 .988 .997 .999 >.999 .020 .571 .662 .730 .805 .842 .884 .905 .930 .975 .997 > .060 .075 .931 .946 .977 .984 .992 .995 .998 .999 .999 >.999 >.999 .025 -682 .776 .839 .901 .928 .995 .967 .979 .995 .999 .100 .991 .999 >.999 .030 .748 .838 .893 .942 .961 .978 .985 .992 .999 >-999 .040 .819 .898 .941 .973 .984 .993 .995 .998 >.999 The above table is taken from (Lucas et al. 1980). The entries were calculated according to Power of the test H : y = y = .01 versus o o = 1 - $ { (.01 - y, + 1.282 a //n)/a //n} , yo yl where a = y± V.H25 + k2 , i = 0, 1, yi yi k = 1.00 for y± < .05, k = 0.75 for .05 <_ y . < .10, k = 0.50 for .10 <_ y. < .20, k = 0.25 for y. >_ .20, yj_ ! y^_ i and ${z> is the cumulative distribution function of a standard normal random variable. -43- ------- Table A-2. Power of the Test H : y = y = .01 versus o o HL: y = (assuming y-L > -01 larger for Selected Values of y. laboratory variance model from (Lucas variability et al. 1980) ) and the nX 3 5 7 10 12 15 17 20 30 50 .015 .284 .313 .337 .369 .388 .414 .430 .452 .518 .621 020 415 464 505 555 584 623 646 678 761 865 .025 .500 .559 .606 .664 .696 .737 .760 .791 .866 .943 .030 .556 .621 .671 .730 .762 .801 .823 .851 .915 .971 040 626 694 745 802 832 867 886 909 956 989 nX 3 5 7 10 12 15 17 20 30 50 .050 .713 .795 .851 .904 .928 .953 .965 .977 .994 >.999 >. 075 774 850 897 940 958 975 982 989 998 999 .100 .891 .952 .978 .993 .997 .999 .999 >.999 .150 .914 .965 >. .985 .996 .998 .999 >.999 200 994 999 The above table is taken from (.Lucas et al. 1980). The entries were calculated according to Power of the test H : y = yQ = .01 versus = 1 - $ {(..01 - u1 + 1.282 a //n)/a //n}, where a = yi J.1125 + 4k2 , i = 0, 1, i i k = 1.00 for U:L < .05, k = 0.75 for .05 <_ Vi < .10, ui yi k = 0.50 for .10 <_ yi < .20, k = 0.25 for y.^ >_ .20, and ${z} is the cumulative distribution function of a standard normal random variable. -44- ------- Table 4, power of .915 is expected when 3 bulk samples are collected and p = .08. Power of .922 is expected when 5 bulk samples are collected and p = .06. From Table 5, for 5 bulk samples and p = .09, power of .929 is expected. For 7 bulk samples and p = .07, power of .906 is expected. It can be seen from comparing these tables that the re- sults of the power calculations using the revised variance estimation are not identical to the results in the early draft. These differences are not thought to be appreciable in light of the qualifying assumptions made Ce.g., unbiased laboratory measurement, variance estimation based on limited data) and the uncertainty as to alternatives (i.e., p,) of interest. The revised variance estimation does not merit any change in the recommended number of bulk samples to collect. -45- ------- REFERENCES (Appendix A) Lucas D, Hartwell T, Rao AV. Research Triangle Institute. 1980. Background Document. Asbestos-Containing Ma- terials in School Buildings: Guidance for Asbestos Analytical Programs. Draft Report. Washington, DC: Office of Pesticides and Toxic Substances, U.S. En- vironmental Protection Agency- Contract no. 68-01- 5848. Patton JL, Price BP, Ogden JS. Battelle Columbus Labora- tories. 1980. Asbestos in Schools: Bulk Sampling Variability Study. Draft Report. Washington, DC: Office of Pesticides and Toxic Substances, U.S. En- vironmental Protection Agency. Contract no. 68^01- 3858. Rao AV, Hartwell TD, Myers LE, Lentzen DE, Breen J, Campbell W, Breeden CH, Gustafson NF. Research Triangle Insti- tute. 1980. Analysis of the Data from the Roundrobin Evaluation of Miscroscopic Procedures for Identifica- tion and Quantification of Asbestos in Sprayed Con- struction Materials. Washington, DC: Office of Pest- icides and Toxic Substances, U.S. Environmental Pro- tection Agency. Contract no. 68-01-5848. Rao AV, Myers LE, Lentzen DE, Hartwell TD. Research Tri- angle Institute. 1980. Analysis of Battelle Bulk Asbestos Duplicate Samples. Draft Report. Washington, DC: Office of Pesticides and Toxic Substances, U.S. Environmental Protection Agency. Contract no. 68-01- 5848. -46- ------- |