EPA 600/R-14/331 I October 2014 I www.epa.gov/research
United States
Environmental Protection
Agency
Evaluation of Options for
Interpreting Environmental
Microbiology Field Data Results
having Low Spore Counts
Office of Research and Development
National Homeland Security Research Center
-------
EPA/600/R-14/331
October 2014
Evaluation of Options for Interpreting Environmental Microbiology Field
Data Results having Low Spore Counts
U.S. Environmental Protection Agency
Cincinnati, Ohio
-------
Disclaimer
The U.S. Environmental Protection Agency, through its Office of Research and Development, funded and
managed the research described here under Contract No. SP0700-00-D-3180, Delivery Order 0729,
Technical Area Task CB-11-0232 with the Defense Threat Reduction Agency and the Department of
Homeland Security under the Battelle/Chemical, Biological, Radiological, and Nuclear Defense
Information and Analysis Center. It has been subjected to the Agency's review and has been approved for
publication. Note that approval does not necessarily signify that the contents reflect the views of the
Agency. Mention of trade names, products, or services does not convey official EPA approval,
endorsement, or recommendation.
Questions concerning this document or its application should be addressed to:
Erin Silvestri
U.S. Environmental Protection Agency
National Homeland Security Research Center
26 W. Martin Luther King Drive, MS NG16
Cincinnati, OH 45268
513-569-7619
Silvestri.Erin@EPA.gov
-------
Table of Contents Page
Disclaimer i
List of Tables iii
List of Figures iv
List of Acronyms and Abbreviations v
Acknowledgments vi
Executive Summary vii
1.0 Introduction 1
1.1 Terminology Used in This Report 3
2.0 Methods 5
2.1 Source of Paired Spread Plate and Filter Plate Data 5
2.2 Overview of Spread Plating and Filter Plating 5
2.3 Options for Interpreting Censored Microbiological Data 8
2.4 Equations for Calculating Sample Concentrations 12
2.5 Overview of Statistical Approach to Calculating Summary Statistics and
95%UCLsontheMean 13
3.0 Results 16
3.1 Summary Statistics and Histograms 17
3.2 95%UCLsontheMean 28
4.0 Discussion 32
4.1 Statistical Approach 32
4.2 All Spread Option 34
4.3 Substitution Options 35
4.4 "Less-Than" Options 36
4.5 Data Validation 36
4.6 Data Groupings 38
5.0 Summary 39
6.0 References 42
Appendix A: Listings of Individual Sample Concentrations under the Six Data Interpretation
Options 48
Appendix B: Distributional Goodness-of-Fit Tests Applied to Each Data Interpretation
Option 56
-------
Appendix C: Estimates for 95% Upper Confidence Limit (UCL) on the Mean, Applying Various
Statistical Methods for Each Data Interpretation Option 60
List of Tables Page
Table 1. Six Data Interpretation Options Evaluated for Censored Microbiological Data 8
Table 2a. Summary Statistics for Air Sample Concentrations (CFU/m3), by Data
Interpretation Option 18
Table 2b. Summary Statistics for Surface Sample Concentrations (CFU/m2), by Data
Interpretation Option 19
Table 3a. Recommended 95% Upper Confidence Limit (UCL) on the Mean, Using Air Sample
Concentrations (CFU/m3) for Each of the Six Data Interpretation Options 29
Table 3b. Recommended 95% Upper Confidence Limit (UCL) on the Mean, Using Surface Sample
Concentrations (CFU/m2) for Each of the Six Data Interpretation Options 30
Table A-l. Listing of Individual Air Sample Concentrations under the Six Data Interpretation
Options 49
Table A-2. Listing of Individual Surface Sample Concentrations under the Six Data
Interpretation Options 50
Table B-l. Results of Distributional Goodness-of-Fit Tests Applied to the Air Sample Data
(n=18 samples) for Each Data Interpretation Option 58
Table B-2. Results of Distributional Goodness-of-Fit Tests Applied to the Surface Sample Data
(n=136 samples) for Each Data Interpretation Option 58
Table C-la. Estimates for 95% Upper Confidence Limit (UCL) on the Mean, Applying Various
Statistical Methods That Rely on a Specific Distributional Form, for Air Sample Data
(CFU/m3) 61
Table C-lb. Estimates for 95% Upper Confidence Limit (UCL) on the Mean, Applying Various
Nonparametric Statistical Methods, for Air Sample Data (CFU/m3) 61
Table C-2a. Estimates for 95% UCL on the Mean, Applying Various Statistical Methods That
Rely on a Specific Distributional Form, for Surface Sample Data (CFU/m2) 62
Table C-2b. Estimates for 95% UCL on the Mean, Applying Various Nonparametric Statistical
Methods, for Surface Sample Data (CFU/m2) 62
in
-------
Table C-3. Summary of ProUCL Recommended Approaches for Calculating the 95% Upper
Confidence Limit (UCL) on an Unknown Mean When All Results are Positive and Detected
and Taken from a Skewed Dataset Without a Discernable Distribution 63
List of Figures Page
Figure la. Histograms of Air Sample Concentrations (CFU/m3) for Four Data Interpretation
Options (n= 18) 24
Figure Ib. Histogram of Air Sample Concentrations (CFU/m3) for the Substitute 15 Data
Interpretation Option (n=18) 25
Figure Ic. Histogram of Air Sample Concentrations (CFU/m3) for the < Quantification -
Both Methods Option (n= 18) 25
Figure 2a. Histograms of Surface Sample Concentrations (CFU/m2) for Four Data
Interpretation Options (n=136) 26
Figure 2b. Histogram of Surface Sample Concentrations (CFU/m2) for the Substitute 15
Treatment Option (n=136) 27
Figure 2c. Histogram of Surface Sample Concentrations (CFU/m2) for the < Quantification
- Both Methods Option (n= 13 6) 27
IV
-------
List of Acronyms and Abbreviations
95% UCL 95% upper confidence limit
BBT Butterfield Buffer with Tween
BCA bias-corrected accelerated
Bg Bacillus atrophaeus subspecies globigii
CPU colony forming units
CLT central limit theorem
cm2 square centimeter
EPA U.S. Environmental Protection Agency
L liter
min minute
MLE maximum likelihood estimation
MVUE minimum variance unbiased estimate
n number of samples
PBST phosphate-buffered saline with Tween® 20
ROS regression on order statistics
Sd standard deviation
SKC vendor of air sampling equipment (SKC Inc.)
sponge cellulose sponge-stick
swab macrofoam swab
TNTC too numerous to count
TSA tryptic soy agar
UCL upper confidence limit
(im micrometer
vacuum vacuum sock
wipe Versalon® wipe
-------
Acknowledgments
The following individuals and organizations are acknowledged for their contributions to this report:
U.S. Environmental Protection Agency, Office of Research and Development, National Homeland
Security Research Center
Worth Calfee
Kevin Garrahan
Erin Silvestri
Sarah Taft
Cynthia Yund
Battelle, Contractor for the U.S. Environmental Protection Agency
VI
-------
Executive Summary
Following a widespread environmental release of a biological agent, such as Bacillus anthracis,
remediation of contaminated facilities or areas may be needed to eliminate or reduce the risk of exposure.
Decision makers may look to microbial exposure assessment using field data collected during remediation
efforts (site characterization and/or post-decontamination sampling) to better inform decisions regarding
identifying exposures, reducing hazards, selecting decontamination strategies, and facility clearance
(Parkin, 2007; Nichols et al, 2006). However, estimating the magnitude of potential exposure using
microbial data collected from the field can be complicated by the lack of guidelines for interpreting such
data, especially when sample results fall below the limits of detection or quantification of the analytical
method used to analyze the samples.
The number of bacterial spores in an environmental sample is often estimated by culturing bacteria from
the sample extract on an appropriate growth medium and observing the number of colonies (colony
forming units [CPU]) that grow through spread plating and/or filter plating. Conventionally, only spread
plates with colony counts in the range of 30-300 CPU are used (although some method ranges differ
slightly) because high colony counts might prevent accurate counting, which can lead to under-
representing the actual count, and high variability is expected with low colony counts (Breed and
Dotterrer, 1916). The countable range for filter plating is often reported as 20 to 200 colonies (SMC,
2011) although some methods have established slightly different ranges. Both spread plate and filter plate
analyses can detect 1 CPU. If replicate plates are used, the detection limit is 1 CPU divided by the number
of replicate plates used.
In cases where a sample result is reported as "not detected", "below the detection limit", or "below the
limit of quantitation", there is little information on how that result should be interpreted. An analytical
measurement that can be expressed only as less than the established quantification limit is classified as
"censored" (more precisely, "left censored") at that limit. A "not detected" result is considered to be less
than the method detection limit, or the lowest value for which it is known with high confidence that the
characteristic is present in the sample and is classified as "censored" at that limit. Similar to a result that is
less than the quantification limit, a non-detected result does not necessarily imply that the actual sample
value is zero (Gilbert, 1987). When encountering censored data within an exposure assessment, EPA
(1992) noted that a variety of data interpretation options could be used. Some researchers have compared
various options for treating censored data, including but are not limited to; substitution, imputation
methods, maximum likelihood estimation, regression on order statistics, and Kaplan-Meier methods.
vn
-------
This report documents the evaluation of six options for representing culture-based/microbial count data
when no colonies were observed and/or when colonies were observed but were below the limits of
quantification of the filter plating or spread plating techniques (i.e., censored data). The six options
included: use of the mean spread plate count, even if under the limit of quantitation; two options for
substitution; and three options for left censoring data at the quantitation and/or the detection limit.
Secondary data that were used for this evaluation were generated from a previous interagency
decontamination study (EPA, 2013). These data included indoor air and surface samples that were
collected post-decontamination (when low numbers of viable and culturable Bacillus atrophaeus
subspecies globigii (Bg) spores were expected within the samples) and analyzed for Bg spores using both
spread plating and filter plating techniques. Mean plate counts were adjusted by multiplying by the
elution spore suspension (for filter and spread plating) and serial dilution (for spread plating) to estimate
the number of CPU in the sample. The sample concentration was also determined for both air (CFU/m3)
and surface samples (CFU/m2). The higher filter plate or spread plate result was used to represent the
sample. Each of the six data interpretation options evaluated in this report were applied to the paired
spread plate and filter plate Bg spore data to compare summary statistics and to evaluate which options
might be more useful for interpreting data when low spore counts and left censoring are present.
Based on the criteria set out in this study, results of this evaluation suggest that when the reported
(unadjusted) mean spread plate count is nonzero but <30 CPU, the actual CPU value should be used if
possible, rather than substituting a different value (e.g., 0 or 15 CPU) based on quantification limits. That
is, all nonzero results should be used instead of using substitution methods. The reason is that substituting
0 CPU when spores are present understates the results. Substituting 15 CPU can understate or overstate
the results, depending on whether the actual CPU is greater or less than 15 CPU. In addition, based on the
results from the data included in this evaluation, if high variability and uncertainty in low concentration
estimates is considered acceptable, then the use of a censoring option in which all nonzero, unadjusted
mean spread plate counts are used in addition to censoring spread plate and filter plate results that were
reported as 0 CPU at the detection limit could be the best option for handling censored observations. This
option maximally utilizes all available information to provide conservative estimates of concentrations
and indicates uncertainty associated with non-detection. However, if high variability and uncertainty in
low concentration estimates is considered unacceptable, then censoring both spread plates and filter plates
with counts below the quantitation limit at the quantitation limit could be the most useful option for
handling censored observations. This option would require appropriate justification for the quantification
and detection limits that are used to represent censored outcomes.
Vlll
-------
1.0 Introduction
Following a widespread environmental release of a biological agent, such as Bacillus anthracis,
remediation of contaminated facilities or areas may be needed to eliminate or reduce the risk of exposure.
Remediation efforts might include both site characterization sampling, to determine the extent of
contamination, and post-decontamination sampling to determine decontamination efficacy (EPA, 2012a).
Decision makers may look to microbial exposure assessment using data collected from the field to better
inform decisions regarding identifying exposures, reducing hazards, selecting decontamination strategies,
and facility clearance (Parkin, 2007; Nichols et al., 2006). However, estimating the magnitude of potential
exposure using microbial data collected from the field can be complicated by the lack of guidelines for
interpreting such data, especially when sample results fall below the limits of detection or quantification
of the analytical method used to analyze the samples.
The number of bacterial spores in an environmental sample is often estimated by culturing bacteria from
the sample extract on an appropriate growth medium and observing the number of colonies (colony
forming units [CPU]) that grow through spread plating and/or filter plating. Conventionally, only spread
plates with colony counts in the range of 30-300 CPU are used, although some method ranges differ
slightly, e.g., 25-250 CPU (Sutton, 2006). The practice of using a specified countable range for spread
plates originates from the historic bacterial examination of milk, as high colony counts (e.g., >300 CPU)
might prevent accurate counting, which can lead to under-representing the actual count, and low colony
counts (e.g., <30 CPU) were associated with high variability (Breed and Dotterrer, 1916). Filter plating
provides a direct bacterial count based on the development of colonies that grow on the surface of a
membrane filter. A sample is filtered through the membrane, which retains the bacteria. The membrane is
then placed on medium where colony forming units are counted. The countable range for filter plating is
often reported as 20 to 200 colonies (SMC, 2011) although some methods have established slightly
narrower ranges (20-80 CPU; ASTM, 2004) or slightly higher ranges (50 to 300 CPU/filter; Clark et al.,
1951).
Both spread plate and filter plate analyses can detect 1 CPU. As the U.S. Environmental Protection
Agency's (EPA) Method Validation of U.S. Environmental Protection Agency Microbiological Methods
of Analysis (2009a) document noted, when no organisms are observed upon applying culture methods to a
particular plate, the result is reported as <1 CPU rather than 0 CPU. If replicate plates are used, the
detection limit is 1 CPU divided by the number of plates. For example, because spread plate analysis of a
given sample involves use of three replicate plates, the detection limit for spread plating is 0.33 CPU (i.e.,
1
-------
1 CPU detected on one of three replicate plates). For filter plate analysis, the detection limit is either 1
CPU or 0.5 CPU, depending on whether one or two replicate filter plates are used to analyze a sample.
In cases where a sample result is reported as "not detected", "below the detection limit", or "below the
limit of quantitation", there is little information on how that result should be interpreted. An analytical
measurement that can be expressed only as less than the established quantification limit is classified as
"censored" (more precisely, "left censored") at that limit. Likewise, an analytical measurement that is
classified as "not detected" is considered to be "censored" at the detection limit. A "not detected" result is
considered to be less than the method detection limit, or the lowest value for which it is known with high
confidence that the characteristic is present in the sample. Similar to a result that is less than the
quantification limit, a non-detected result does not necessarily imply that the actual sample value is zero
(Gilbert, 1987).
When encountering censored data within an exposure assessment, EPA (1992) noted that a variety of data
interpretation options could be used, and "selecting the appropriate method requires consideration of the
degree of censoring, the goals of the exposure assessment, and the accuracy required." Some researchers
have compared various options for treating censored data, and these options have continued to evolve in
recent years beyond traditional (and problematic) substitution approaches as the ability to handle more
computer-intensive analytical techniques has increased. For example, Gleit (1985) evaluated multiple
options for small (n = 5 to 15) normal environmental data sets and found that a "fill-in with expected
values" approach worked better than a maximum likelihood estimation (MLE) or "fill-in with constants"
approach. Smeets et al. (2007) used various approaches for dealing with non-detect Cryptosporidium
oocyst concentrations in water (including non-detects being set to zero and non-detects being set to the
detection limit); however, the best non-detect concentration estimate was identified from log-linear
extrapolation. For non-detects of Campylobacter spp. in water, Signer et al. (2005) compared a
"nonparametric modified log-probability regression model" presented by El-Shaarawi (1989) and the
"fill-in with expected values" technique developed by Gleit (1985). Although both methods generated
similar mean concentrations, Signer et al. (2005) reported that the method proposed by El-Shaarawi
(1989) produced a more conservative probability distribution function in the upper tail then the method
proposed by Gleit (1985) and was the more preferred method. However, given a small dataset, Signer et
al. (2005) noted that the preferred method may result in excessive rather than conservative estimates, and
that an educated judgement would have to be made to determine the appropriateness of the variability
analysis results.
-------
Wong et al. (2009) used regression on order statistics (ROS) for non-detects in developing predictive
models of enteric virus contamination at recreational beaches. Wong et al. (2009) noted that imputation
methods, "which fill in values for non-detects without assigning them all the same value," provide better
descriptions of censored data than substitution approaches. Imputation methods include MLE, Kaplan-
Meier, and ROS. Wong et al. (2009) noted that ROS imputation is known to work better than MLE for
data sets with <50 detected values. As noted by Helsel (2005), the more accurate methods for computing
statistics (modern MLE, ROS, and Kaplan-Meier methods) are now available in statistical software.
This report documents the evaluation of six options for interpreting culture-based/microbial count data
sets that include left censored data, or measurements that are less than established quantification limits
and/or detection limits. The six options attempt to use the filter plating result along with the spread
plating result for a given sample in order to improve the precision of the final sample result, and to
consider different approaches to handling censored outcomes. However, while the evaluation considered
paired spread plate and filter plate results, these options are applicable even when only one set of results
is available for a sample.
The secondary data that were provided to Battelle for analysis in this study were generated from a
previous interagency decontamination study (EPA, 2013), in which indoor air and surface samples were
collected and split samples were analyzed for Bacillus atrophaeus subspecies globigii (Bg) spores using
both spread plating and filter plating techniques (Section 2.2). Each of the six data interpretation options
evaluated in this report were applied to paired spread plate and filter plate Bg spore data associated with
air and surface samples collected post-decontamination in this study to compare summary statistics and to
evaluate which options are more appropriate for use in making conclusions from the data when low spore
counts and left censoring are present. (These six options are introduced in Section 2.3)
1.1 Terminology Used in This Report
The terms "count" and "mean plate count" refer to the number of CPU observed on replicate plates under
a given method, then averaged across the plates. These counts are considered "unadjusted" until
multiplied by the volume of the elution suspension and the dilution factor, as applicable, resulting in
"adjusted CPU." (Mean filter plate counts were multiplied by the volume of the elution suspension; mean
spread plate counts were multiplied by the dilution factor and volume of the elution suspension.) The term
"result" (or equivalently, "concentration") refers to the adjusted CPU per volume of air sampled or per
area of the surface sampled. Thus, in this context, "result" is equivalent to a Bg spore concentration.
-------
Among the summary statistics considered in this evaluation of data interpretation options in the presence
of censored data is a 95% upper confidence limit (95% UCL) on the mean concentration. The 95% UCL
on the mean is defined as the lowest value that is expected to equal or exceed the true (unknown) mean of
the distribution 95% of the time, if the experiment was to be continuously repeated under the same
conditions. The 95% UCL is a measure of uncertainty in the mean, rather than a measure of variability in
the data, which makes it distinct from the 95th percentile of the data distribution. As described in EPA's
Risk Assessment Guidance for Superfund (EPA, 1989), the 95% UCL on the mean is traditionally used in
human health risk assessments as a point estimate of reasonable maximum exposure, or equivalently, the
exposure point concentration (i.e., the contaminant concentration at the point of contact by humans). Haas
et al. (1993) have noted that microbial exposures could be conducted under the same framework used for
chemical risk assessments. Although distributional data are generally preferred over point estimates in
microbial assessments (EPA, 2012b), 95% UCL on the mean values have been used to describe
environmental microbiology data. For example, Goodwin et al. (2012) used the 95% UCL on the mean to
describe infrequently detected methicillin-resistant Staphylococcus aureus in seawater and beach sand
samples. Hamilton et al. (2006) calculated the 95% UCL on the mean for the annual probability of enteric
virus infection associated with the ingestion of uncooked vegetables grown using reclaimed water.
-------
2.0 Methods
2.1 Source of Paired Spread Plate and Filter Plate Data
Paired spread plate and filter plate data from an EPA study (EPA, 2013) of the decontamination of Bg
released inside of a facility were used for this investigation involving the six data interpretation options.
In the EPA decontamination study, indoor air samples were collected with SKC BioSampler® (SKC,
Eighty Four, PA) units (liquid impingers) with the sample contents deposited directly into phosphate-
buffered saline with Tween® 20 (PBST). Air samples were collected from two rooms within the
contaminated building at three distinct heights and at three locations per room. These air samples were
collected over 15 minutes in a staggered fashion (0-15, 15-30, and 30-45 minutes) to represent the level of
Bg air contamination following decontamination. Indoor surface samples were collected with cellulose
sponge-sticks (sponge), macrofoam swabs (swab), vacuum socks (vacuum), and Versalon® (Pall
Corporation, Port Washington, NY) wipes (wipe). The sampled surface area varied considerably among
the methods: 0.00258 m2 for the swab, 0.0645 m2 for the sponge and wipe, and 0.3716 m2 for the
vacuum. Bg spores were extracted from surface samples in PBST, and the volume of the elution spore
suspension was determined. For sponge, vacuum, and wipe samples, the elution suspension was
centrifuged and concentrated to reduce the volume. For both air and surface samples, the EPA study
cultured portions of the elution suspension via spread plating and filter plating to estimate the number of
Bg spores as CPU (EPA, 2013).
This data investigation used paired spread plate and filter plate results for 18 indoor air samples and 136
surface samples from the EPA study, for a total of 154 samples. These samples were collected post-
decontamination and resulted in a high prevalence of low Bg counts (compared to pre-decontamination
levels), including results falling below quantification or detection limits (i.e., left censored at these limits).
Therefore, in order for a data investigation option to be considered among the better performers in this
evaluation, it needed to be relevant and valid when samples contained low spore counts.
2.2 Overview of Spread Plating and Filter Plating
Spread Plating. For spread plating, 0.1 mL of the elution suspension (or subsequent 10-fold dilution) was
applied to each of three replicate tryptic soy agar (TSA) plates. The sample was mechanically spread
across the TSA plates using a cell spreader. Once the TSA plates were incubated, the CPU were counted
on each plate. A series of 10-fold dilutions of the initial sample were also prepared and spread plated as
-------
well. The use of dilutions in the spread plating approach allowed estimation of the number of Bg spores
when densities were high in the undiluted sample. Spread plating was performed in triplicate (that is,
three spread plates prepared for each dilution).
Equation 1 shows the formula for calculating the adjusted CPU from spread plating (i.e., the estimated
number of Bg spores collected in the sample after adjustment per the volume of the elution suspension
and the serial dilution). Equation 1 was used to estimate the number of Bg spores collected in the total
elution suspension sample by accounting for the dilution factor and volume of the elution suspension.
While the adjusted CPU is a count variable, the calculation does not necessarily result in an integer value,
and in such situations, the result is rounded to the nearest integer.
Equation 1. Calculation of Adjusted CFU from Spread Plating
Adjusted CFU = Mean Spread Plate Count (CFU/mL) x Serial Dilution (unitless)
x Volume (mL)
where:
Adjusted CFU = estimated number of Bg spores (CFU) collected in the sample; rounded to the
nearest integer.
Mean Spread Plate Count = (unadjusted) average CFU across three replicate spread plates, per
volume of elution suspension plated.
Serial Dilution = dilution factor (unitless); a factor of 10 (from 10 to 10,000) that accounted for
plated volume (0.1 mL) and any associated 10-fold dilutions used for spread plating. The serial
dilution with an unadjusted mean spread plate count of 30 to 300 CFU was selected for calculation
of the adjusted CFU. Note: when the mean spread plate count was identified as censored (i.e.,
below the quantification limit or detection limit), a dilution factor of 10 was applied, reflecting the
most undiluted suspension plated.
Volume = volume of elution suspension (mL).
Historically for spread plating, the dilution that led to an unadjusted mean spread plate count between 30
and 300 CFU (with some guidance recommending slightly different ranges. If a dilution led to counts
above this range, it was not used. This was because colony overlap or crowding may prevent accurate
counting (Breed and Dotterrer, 1916). Alternatively, if a dilution led to an unadjusted mean spread plate
count of less than 30 CFU, variability becomes high. Therefore, to help maximize accuracy and precision,
the dilution with an unadjusted mean spread plate count of 30 to 300 CFU (quantifiable range) was
selected for reporting the final spread plating result for a sample. In practice, however, it is possible to
detect a single spore on one of the three spread plates, and thus, to report a nonzero CFU result for the
sample.
-------
Colony counts (CPU) provide an estimate of spore levels in a sample and could arise from one or several
spores (Sutton, 2006). Colony plate counts can underestimate the true number of organisms in the sample
if they are clumped together (Almeida et al., 2008). However, Baron et al. (2008) noted that when flow-
enhanced spores were allowed to settle on agar plates and subsequently dispersed using a spreader and
Butterfield Buffer with Tween (BBT), this provided a better estimate of spore numbers compared to un-
manipulated agar plates and agar plates exposed to BBT solution after spore settling. Spreading likely
separated spore clumps, thereby reducing the chance that multiple spores were contributing to a single
CPU. Baron et al. (2008) hypothesized that spreading/BBT techniques would be similar to approaches
where surface samples were collected and then cultured from liquid suspensions.
Filter Plating. For filter plating, 1 mL or more of undiluted sample (i.e., the elution suspension) was
vacuum filtered through a 0.45 (im Microfunnel that prevented the passage of Bg spores. After filtration
was completed, the filter was placed onto a TSA plate and incubated to support colony growth. The
elution suspension was then applied to either one or two replicate TSA filter plates. As with spread
plating, all CPU counts <300 were reported on a given plate. If the CPU count was >300 on a plate, the
result would be reported as "too numerous to count" (TNTC), although this outcome did not occur among
the sample results considered in this evaluation.
Equation 2 shows the calculation for the adjusted CPU from filter plating. Like spread plating, the
adjusted CPU is a count variable and thus is rounded to the nearest integer if necessary. Clark et al.
(1951) noted that filter plating is advantageous when determining the number of CPU in samples
containing low densities of culturable bacteria (<30 CFU/mL). When spore numbers are high in the
elution suspension, filter plating will result in too many colonies to achieve accurate counts (Clark et al.,
1951).
Equation 2. Calculation of Adjusted CFU from Filter Plating
Adjusted CFU = Mean Filter Plate Count (CFU/mL) x Volume (mL)
where:
Adjusted CFU = estimated number of Bg spores (CFU) collected in the sample; rounded to the
nearest integer.
Mean Filter Plate Count = the (unadjusted) CFU count if a single filter plate was used for the
sample, or the average CFU count if two filter plates were used for the sample, relative to the
volume of elution suspension plated.
Volume = volume of elution suspension (mL).
-------
2.3 Options for Interpreting Censored Microbiological Data
For a given air or surface sample considered in this assessment, the reported (adjusted) CPU value was
often too low to be quantifiable under a given analysis technique (filter plating or spread plating). This
assessment considered six different options for handling these non-quantifiable outcomes when
calculating summary statistics and the 95% UCL on the mean. Results of these calculations were
compared across these six options, called "data interpretation options," and the outcome of this
comparison provided knowledge to inform future data decisions. These options were applied to paired
spread plate and filter plate data for a given air or surface sample, although having paired data is not
required to implement these options. Table 1 defines these six options.
Table 1. Six Data Interpretation Options Evaluated for Censored Microbiological Data
Data
Interpretation
Option
Description
1. Substitute 0
- Treat as
Detect
0 CPU was substituted for unadjusted mean spread plate counts <30 CPU.
Mean filter plate counts and mean spread plate counts of 0 CPU were treated
as reported.
The filter plate result was used to represent the sample when greater than the
spread plate result.
Results of 0 CFU/m2 (or CFU/m3) were possible, and all results were treated
as detects for the calculation of summary statistics and 95% UCL on the
mean.
2. All Spread
- Treat as
Detect
No substitution was made for unadjusted mean spread plate counts, even if
<30 CPU.
Mean filter plate counts and mean spread plate counts were treated as
reported.
The filter plate result was used to represent the sample when greater than the
spread plate result.
Results of 0 CFU/m2 (or CFU/m3) were possible, and all results were treated
as detects for the calculation of summary statistics and 95% UCL on the
mean.
3. Substitute 15
- Treat as
Detect
15 CPU (i.e., half the quantification limit) was substituted for unadjusted
mean spread plate counts <30 CPU.
Mean filter plate counts were treated as reported.
The filter plate result was used to represent the sample when greater than the
spread plate result.
All results were nonzero and treated as detects for the calculation of
summary statistics and 95% UCL on the mean.
-------
Data
Interpretation
Option
Description
4. < Quantification
- Identify if
"Less-Than"
Unadjusted mean spread plate counts less than the quantification limit were
identified as being <30 CPU, and the associated results were calculated
assuming a dilution factor of 10 and identified as being censored.
Unadjusted filter plate counts of 0 CPU were identified as either <1 CPU (if
one filter plate was used) or <0.5 CPU (if two plates were analyzed), and the
associated results were identified as being censored.
The sample result was represented by the analysis method (spread plate or
filter plate) with the higher detected result. If both methods yielded results
identified as censored, the lower result (i.e., filter plate) was used to represent
the sample.
Summary statistics were based on the detected results only, while the 95%
UCL on the mean was based on the detected and censored results.
5. < Detection
- Identify if
"Less-Than"
When nonzero, unadjusted mean spread plate counts, even if <30 CPU, were
used as reported and treated as detected.
Unadjusted mean spread plate counts of 0 CPU were identified as being
<0.33 CPU and the associated results were calculated assuming a dilution
factor of 10 and identified as censored.
Unadjusted filter plate counts of 0 CPU were identified as either <1 CPU (if
one filter plate was used) or <0.5 CPU (if two plates were analyzed), and the
associated results were identified as being censored.
The sample result was represented by the analysis method (spread plate or
filter plate) with the higher detected result. If both methods yielded results
identified as censored, the lower result was used to represent the sample.
Summary statistics were based on the detected results only, while the 95%
UCL on the mean was based on the detected and censored results.
6. < Quantification
- Both Methods -
Identify if
"Less-Than"
Unadjusted mean spread plate counts less than the quantification limit were
identified as being <30 CPU, and the associated results were calculated
assuming a dilution factor of 10 and identified as being censored.
Unadjusted mean filter plate counts less than a quantification limit of 20
CPU were identified as being <20 CPU, and the associated results were
identified as being censored.
The sample result was represented by the analysis method (spread plate or
filter plate) with the higher quantifiable result. If both methods yielded
results identified as censored, the lower result (i.e., filter plate) was used to
represent the sample.
Summary statistics were based on the quantifiable results only, while the
95% UCL on the mean was based on the quantifiable and censored results.
Each of these data interpretation options yielded a separate set of sample results (as described in Section
2.4). Summary statistics including the 95% UCL on the mean were generated for each set using EPA's
ProUCL software (as described in Section 2.5). Each of the six data interpretation options is described in
more detail below, as well as the historic basis for its consideration.
-------
(1.) Substitute 0 - Treat as Detect. This option, also referred to as "Substitute 0", treated samples with
unadjusted mean spread plate counts <30 CPU as having 0 CPU. However, in determining the sample
result, the sample's filter plate result was used if it was detected and greater than the spread plate result.
Because nonzero filter plate counts <30 CPU are considered "detects", this option generally defaulted to
the filter plate result whenever the unadjusted mean spread plate count was <30 CPU.
Brown et al. (2007a, 2007b, and 2007c) reportedly only considered spread plates with CPU counts of 30-
300 when measuring Bg surface contamination following aerosol deposition. Several researchers
sampling for spores on surfaces used filter plate results alone or in combination with other plating
techniques. Hodges et al. (2006) and Estill et al. (2009) evaluated various surface sampling methods for
Bacillus anthracis spores based on counting CPU on filter plates. Krauter et al. (2012) studied the
recovery of Bg spores with wipe samples and used filter plating if no growth occurred after standard serial
dilution and plating. Calfee et al. (2012) studied the decontamination of Bg spores deposited on surfaces,
and when <30 CPU were counted on a TSA plate, the remaining wipe sample extract was analyzed by
filter plating.
(2.) All Spread - Treat as Detect. This option, also referred to as "All Spread", used the reported
unadjusted mean spread plate count, even when the value was less than quantification limits (<30 CPU),
and treated all samples as having detected results. For a given sample, this option assigned the final
sample result as the larger of the filter plate or spread plate result.
ASTM (2004) methods for water state that all colonies on spread plates should be counted when
microbial counts are low. Sutton (2006) also noted that plate counting guidance varies by organization,
and some report colonies below the countable range, e.g., <30 CPU for spread plates as an estimated
count.
Some research involving surface sampling of spores failed to document approaches with regard to a
quantification limit or minimum acceptable spread plate count, simply indicating that CPU were counted
(Sanderson et al., 2002; Frawley et al., 2008; Valentine et al., 2008). Sanderson et al. (2002) noted that
when the CPU counts were >300, the results were reported as too numerous to count.
(3.) Substitute 15 - Treat as Detect. For this option (also referred to as "Substitute 15"), unadjusted
mean spread plate counts less than the quantification limit (i.e., <30 CPU) were substituted with one-half
10
-------
of the quantification limit (15 CPU), and the result was always treated as detected. For a given sample,
this option assigned the final sample result as the larger of the filter plate result or spread plate result.
Rodda et al. (1993) evaluated the substitution of one-half the detection limit for a risk assessment of
enteric viruses in water. The associated daily risk of enteric virus infection was considerably higher than
the mean yearly risk recommended by EPA. Rodda et al. (1993) acknowledged that the identification of
low risk levels in drinking water was inhibited by small sample volumes and high detection limits. Signer
et al. (2005) used one-half of the detection limit for one result (one out of 16 daily samples) in a study to
quantify the impact of rain events on Cryptosporidium concentrations in surface water. Substituting with
one-half of the detection limit was a common practice for purposes of conducting a chemical risk
assessment historically (for example, when analyzing measured concentrations of chemicals in the
environment), as it simplified the data analysis, but this practice is diminishing as recent increases in
standard computing power allow for more sophisticated data analysis techniques to be implemented.
(4.) < Quantification -Identify if "Less-Than". Unlike the previous three options that treated each
sample result as detected when calculating summary statistics (including 95% UCL on the mean), this
option (also referred to as "< Quantification") retained information on whether the sample result was less
than the quantification limit of 30 CPU (for the spread plate) or less than the detection limit of either 1.0
or 0.5 CPU (for the filter plate). (Filter plates generally do not have an associated quantification limit, as
all countable CPU on a filter plate are considered valid.) For the data summaries and analyses, such
unadjusted mean plate counts were identified as censored at either <1 CPU for one filter plate, <0.5 CPU
for two replicate filter plates, or <30 CPU for the spread plates. The censored counts were then adjusted
to account for elution suspension volumes and dilution factor per Equations 1 and 2. If both the associated
spread plate and filter plate results were censored, the lower result (i.e., the filter plate result) was used to
represent the sample and was identified as censored. Otherwise, the higher detected result was used, and
the sample was identified as a detect. ProUCL includes statistical methods such as Kaplan-Meier (Kaplan
and Meier, 1958) for computing the 95% UCL on the mean for data sets with censored data.
(5.) < Detection -Identify if "Less-Than". This option, also referred to as "< Detection", utilized the
same approach as the < Quantification option (4), except that the unadjusted mean spread plate count was
censored only when less than the detection limit (i.e., <0.33 CPU). Unadjusted mean spread plate counts
of at least 1 CPU but less than 30 CPU (i.e., less than the quantification limit) were not treated as
censored, and therefore, the reported mean spread plate count was used. If both the associated spread
11
-------
plate and filter plate results were identified as being censored, the lower result was used to represent the
sample. Otherwise, the higher detected result was used.
(6.) < Quantification - Both Methods - Identify if "Less-Than ". This option, also referred to as
-------
Equation 4. Calculation of Surface Sample Concentration (CFU/m2)
r ,, Adjusted CPU
Surface Sample Concentration (CFU/irr) = —r-
F v / ; Sample Area (m2)
where:
Adjusted CPU = estimated number of Bg spores (CPU) collected in the sample from Equation 1 for
spread plating or Equation 2 for filter plating.
Sample Area (m2) = surface area sampled: 0.00258 m2 for the swab, 0.0645 m2 for the sponge and
wipe, and 0.3716 m2 for the vacuum.
Note that the sample concentrations calculated from Equations 3 and 4 are continuous in nature over the
set of non-negative real numbers and are not rounded to the nearest integer.
2.5 Overview of Statistical Approach to Calculating Summary Statistics and 95% UCLs on the
Mean
Summary statistics of the sample concentration results were generated separately for air and surface
samples and for each of the six data interpretation options using Version 4.1 of EPA's ProUCL software
(EPA, 2010)1. In addition, ProUCL was used to calculate the 95% UCL on the mean. ProUCL provides
several state-of-the-art parametric and nonparametric statistical methods for calculating the 95% UCL on
the mean from data sets containing both uncensored and censored data. These methods, which are
detailed in EPA (2007), include the following:
Parametric Methods (all results detected/quantifiable)
• Student's t-statistic - assumes normality or approximate normality
• Approximate gamma upper confidence limit (UCL) - assumes gamma distribution
• Adjusted gamma UCL - assumes gamma distribution
• Land's H-Statistic - assumes lognormality
• Chebyshev Theorem using the minimum variance unbiased estimate (MVUE) of the parameters of a
lognormal distribution (denoted by Chebyshev (MVUE)) - assumes lognormality
Nonparametric Methods (all results detected/quantifiable)
• Modified t-statistic - modified for skewed distributions
1 Downloaded from http://www.epa.gov/osp/hstl/tsc/software.htm.
13
-------
• Central limit theorem (CLT) - to be used for large samples
• Adjusted central limit theorem (adjusted-CLT) - adjusted for skewed distributions and to be used for
large samples
• Chebyshev Theorem using the sample arithmetic mean and standard deviation (Sd) (denoted by
Chebyshev (Mean, Sd))
• Jackknife method - yields the same result as Student's t-statistic for the UCL of the population mean
• Standard bootstrap
• Percentile bootstrap
• Bias-corrected accelerated (BCA) bootstrap
• Bootstrap t
• Hall's bootstrap
Nonparametric Methods (some non-quantifiable results)
Different techniques are available in ProUCL to estimate the 95% UCL under the Kaplan-Meier method,
including the following:
• Using a percentile bootstrap method
• Using a BCA bootstrap method
• Using the Chebyshev inequality
• Using a Student-^ cutoff value
Results of statistical tests for goodness-of-fit for the normal, lognormal, and gamma distributions are
presented in Appendix B for each data interpretation option. As a result of these tests and upon viewing
plots of the data, because the data were inherently count-based rather than continuous in nature, and
because it was preferred to make a common distributional assumption for all analyses if possible, a
nonparametric approach was taken to estimate the 95% UCL on the mean for each option and each
sample type.
The Substitute 0, All Spread, and Substitute 15 options use substitution techniques and/or actual observed
plate counts to represent the sample. When calculating summary statistics or the 95% UCL on the mean,
these options treat all data as detected, including results of 0 CFU/m3 or 0 CFU/m2. Therefore, only those
methods in ProUCL that are relevant to data sets containing 100% detected results are considered for
these three options. Furthermore, because the first two of these three options can yield results of 0 CPU,
14
-------
parametric techniques for calculating 95% UCL on the mean that require all data to be nonzero (e.g.,
lognormal, gamma) were not applicable.
For data interpretation options that identify results as censored limits (< Quantification and < Detection),
ProUCL calculates summary statistics on detected results only, while the 95% UCL on the mean is
calculated using statistical techniques that account for the presence of censored values. These methods
include Kaplan-Meier and ROS, both of which allow for multiple detection/quantification limits among
samples within the same data set.
Statistical Outliers. The statistical approaches used in this report to calculate a 95% UCL on the mean
were applied both with and without identified statistical outliers included in the analysis. Outliers are
those sample values which are identified as extreme in a particular direction relative to the distribution of
the remaining sample values; they are labeled as statistical outliers based on the results of a statistical
hypothesis test (Gilbert, 1987). For each data interpretation option having at least three detected values,
an outlier test available within ProUCL was applied to data that were not labeled as censored limits.
Dixon's extreme value test (Dixon, 1950) was applied to the indoor air data due to the small sample size
(n=18), while Rosner's test (Rosner, 1983) was applied to the surface sample data (n=136). Note that both
tests assume a normal distribution to the data, which does not generally hold for these data interpretation
options. Therefore, the results of these tests were used only as a guide for identifying extreme data values.
When outliers with large values are included in data analyses, they can contribute to inflated summary
statistics and can unduly impact the results of goodness-of-fit tests and the decision on the approach used.
However, they do not warrant automatic exclusion from analysis simply due to their magnitude - some
verification that their values are invalid is typically needed. Thus, if there is insufficient evidence that
outliers are invalid, analyses are typically performed, as was done here, with and without the outliers
included, to assess the impact on the analysis outcomes (e.g., 95% UCL on the mean).
15
-------
3.0 Results
For each of the six data interpretation options in Table 1, the results of paired spread plate and filter plate
analyses were evaluated for each sample. In general, the higher detected result between the two analysis
methods was adopted as the count (CPU) estimate for the sample, and sample concentration was
calculated from the CPU value. Of the two, the higher result when detected was considered a more
precise estimate of the actual count. However, if both spread plate and filter plate results were left
censored (that is, both results were expressed as a "less-than" value), the lower value was selected and
treated as censored. Here, the lower value was selected because that result provided more information on
the actual sample count. For example, if two methods reported values of <30 and <60 CPU for a given
sample, the value of <30 is considered more precise given neither method can quantify the result. Air
sample results (Bg spores in the air) were presented in units of CFU/m3 based on the air sampling flow
rate and sample duration. Surface sample results (Bg spores on surfaces) were presented in units of
CFU/m2 based on the surface area sampled.
Appendix A contains tables of the results under each of the six data interpretation options for each
individual sample. Table A-l contains results for the 18 indoor air samples, while Table A-2 contains the
136 surface sample results. The following findings were noted before any substitutions or designations of
censoring were made within a particular data interpretation option:
• Unadjusted mean spread plate counts were <30 CPU for all 18 indoor air samples and for all but
one of the 136 surface samples.
o Only one surface sample did not have a mean spread plate count of <30 CPU. Its
unadjusted count equaled 30 CPU; after adjusting for dilution factor and sample volume
and dividing by sample area, its result was 3,991 CFU/m2.
• Seven of the 18 indoor air samples reported mean spread plate counts >0 CPU (before the
application of any substitutions) - the remaining 11 samples reported a result of 0 CPU.
o The largest mean spread plate count among these seven samples was 1.33 CPU (i.e., one
count in each of two plates, and two counts in the third plate), prior to multiplying by the
sample volume.
o One of these seven samples had a single CPU observed in the second serial dilution.
Although no colonies were observed at the first dilution, application of a dilution factor
of 100 was required, leading to the largest result (for an option not associated with
substitution) of 1,758 CFU/m3). All others air samples were based on a dilution factor of
10, and the next largest result (for an unsubstituted option) was 714 CFU/m3.
16
-------
• 41 of the 136 surface samples (30%) reported mean spread plate counts >0 CPU (before the
application of any substitutions) - the remaining samples reported a result of 0 CPU.
o Excluding the one detected surface sample noted above (with an unadjusted mean spread
plate count = 30 CPU), the largest unadjusted mean spread plate count was 10 CPU.
o Two of these 41 samples required a dilution factor of 100, while all others used a dilution
factor of 10. As described above for the air sample based on a dilution factor of 100, no
colonies were observed at the first dilution. The adjusted CPU for both samples was 167
CPU, which was within the range observed for the other 39 samples (15 to 1,483 CPU).
However, with the application of the small sampling area for a swab (0.00258 m2), one
sample based on a dilution factor of 100 had the largest unsubstituted result (which
occurs by implementing the All Spread option) - 64,729 CFU/m2. The next largest
sample result was 38,760 CFU/m2, also for a swab sample. The other sample based on a
dilution factor of 100 was from a vacuum sample with a result of 449 CFU/m2.
• Only two of the 18 indoor air samples (11%) had a filter result of 0 CPU. In contrast, 55% of the
surface samples had a filter result of 0 CPU, or 75 of the 136 surface samples.
o The filter results for these 77 samples (i.e., the two indoor air and 75 surface samples
with filter result of 0 CPU) were identified as less-than values for the < Quantification
options (4 and 6).
o Of these 77 samples, the two indoor air sample filter results and 12 of the surface samples
representing wipe samples were analyzed using a single filter plate and thus were based
on a detection limit of 1 CPU.
o All of the remaining surface samples (i.e., 63 vacuum, sponge, and swab samples) were
analyzed using two plates and thus were based on a detection limit of 0.5 CPU.
• Only one of the 18 indoor air samples (6%) had a result of 0 CPU for both spread plate and filter
plate, compared to 66 of the 136 surface samples (49%).
3.1 Summary Statistics and Histograms
Tables 2a and 2b present summary statistics, calculated by ProUCL, on the results for the indoor air
samples and the surface samples, respectively. As noted in Table 1, under the < Quantification and
< Detection options, the summary statistics in these tables were calculated only on detected results, while
all sample results are used to calculate the summary statistics for the other three options. Several
parameters were included in Tables 2a and 2b to help describe/differentiate the data sets being evaluated
including: the number of samples, the number of samples represented by the spread plate and filter plate
17
-------
Table 2a. Summary Statistics" for Air Sample Concentrations (CFU/m3), by Data Interpretation Option
# Samples
# Filter/* Spread"
% Detected
# of 0 CFU/m3 Results
# of Censored Results
Mean (CFU/m3)
Standard Deviation
Coefficient of Variation
Skewness
Minimum
25th Percentile
50th Percentile
75th Percentile
90th Percentile
95th Percentile
Maximum (CFU/m3)
Standard Deviation of
Log-Transformed Results
1. Substitute 0
- Treat as Detect
18
16/2
100%
2
NA
104
78
0.75
0.87
0
52
86
147
213
229
288
NA
2. All Spread
- Treat as Detect
18
12/6
100%
1
NA
278
409
1.47
3.12
0
56
158
271
523
871
1,758
NA
3. Substitute 15
- Treat as Detect
18
0/18
100%
0
NA
8,323
531
0.06
1.77
7,733
7,956
8,110
8,623
8,742
8,963
9,957
0.061
4. < Quantification
- Identify if
"Less-Than"
18
18/0
88.9%
NA
2C
117
72
0.62
1.07
51
55
108
158
215
236
288
0.595
5. < Detection
- Identify if
"Less-Than"
18
13/5
94.4%
NA
ld
294
416
1.41
3.08
51
57
159
288
550
923
1,758
1.015
6. < Quantification
- Both Methods -
Identify if
"Less-Than"
18
18/0
0.0%
NA
18
~
~
~
~
~
~
~
~
~
~
~
~
CPU, colony forming units, NA = not applicable.
a Calculated only on detected results. For the Substitute 0, All Spread, and Substitute 15 options, all data were considered to be detected (including results of 0
CFU/m3) and therefore all results were included in the calculation of the summary statistics. The less-than values associated with the < Quantification and
< Detection options were not included in the summary statistics as these results were not specifically known.
b The number of filter plate results and spread plate results selected to represent the 18 air samples.
0 The two less-than samples were <55 and <59 CFU/m3; see Table A-l, which identifies detected and less-than results.
d The one less-than sample was <59 CFU/m3; see Table A-l, which identifies detected and less-than results.
18
-------
Table 2b. Summary Statistics" for Surface Sample Concentrations (CFU/m2),, by Data Interpretation Option
# Samples
# Filter/* Spread"
% Detected
# of OCFU/m2 Results
# of Censored Results
Mean (CFU/m2)
Standard Deviation
Coefficient of Variation
Skewness
Minimum
25thPercentile
50th Percentile
75th Percentile
90th Percentile
95th Percentile
Maximum (CFU/m2)
Standard Deviation of Log-
Transformed Results
1. Substitute 0
- Treat as Detect
136
60/76
100%
75
NA
465
2,810
6.04
10.83
0
0
0
95
582
1,977
32,171
NA
2. All Spread
- Treat as Detect
136
36 / 100
100%
66
NA
1,181
6,506
5.51
8.47
0
0
8
249
1,372
2,791
64,729
NA
3. Substitute 15
- Treat as Detect
136
0/136
100%
0
NA
36,785
76,912
2.09
2.93
2,018
2,018
11,395
15,407
46,512
290,698
290,698
1.397
4. < Quantification
- Identify if
"Less-Than"
136
135/1
44.8%
NA
75C
1,037
4,143
3.99
7.32
8
31
170
450
2,093
2,481
32,171
1.936
5. < Detection
- Identify if
"Less-Than"
136
102/34
51.5%
NA
66C
2,295
8,957
3.90
6.06
8
47
233
1,039
2,481
4,745
64,729
2.017
6. < Quantification
- Both Methods -
Identify if
"Less-Than"
136
135/1
3.7%
NA
131d
2,159
1,202
55.67
0.68
681
1,705
2,093
2,326
3,991
3,991
3,991
0.645
CPU, colony forming units; NA = not applicable.
a Calculated only on detected results. For the Substitute 0, All Spread, and Substitute 15 options, all data were considered to be detected (including results of 0
CFU/m2) and therefore all results were included in the calculation of the summary statistics. The less-than values associated with the < Quantification and
< Detection options were not included in the summary statistics as these results were not specifically known.
b The number of filter plate results and spread plate results selected to represent the 136 surface samples.
0 The less-than samples ranged from <8 to <1,163 CFU/m2; see Table A-2, which identifies detected and less-than results.
d The results for the five samples with quantifiable outcomes were 681, 1,705, 2,093, 2,326, and 3,991 CFU/m2.
19
-------
results, the percentage of results treated as detects, the number of zero results (i.e., 0 CFU/m3 or CFU/m2),
and the number of censored results.
The following information can be gained from the summary statistics in Tables 2a and 2b:
• The Substitute 0 option was equivalent to using the filter plate result for all samples but the one
surface sample where the unadjusted mean spread plate count equaled 30 CPU.
o Because all unadjusted mean spread plate results were <30 CPU except for one surface
sample, all nonzero filter plate results were used to determine the sample result except for
the one surface sample.
o Nonzero filter plate results represented 16 of 18 air samples and 60 of 136 surface
samples.
o More than half of the surface samples (75 of 136) were assigned a result of 0 CFU/m2
under this option. Two of 18 air samples were assigned a result of 0 CFU/m3.
o While the largest observed surface result (swab) was 32,171 CFU/m2 under this option,
the next largest result (vacuum sock) was 3,991 CFU/m2 (i.e., the sample using the spread
plate result).
o As a note, if filter plate data were not available and spread plate results <30 CPU were
considered to have 0 CPU, then all air sample results would be 0 CFU/m3, and only one
surface sample result would be nonzero (i.e., 3,991 CFU/m2). Without consideration of
the filter plate data, the associated mean level of Bg spore contamination on the indoor
surfaces would be 29 CFU/m2 compared to 465 CFU/m2 with the filter plate data.
• The All Spread option showed results of 0 CFU/m3 or 0 CFU/m2 only when both the spread plates
and filter plates had unadjusted plate counts of 0 CPU. Compared to the Substitute 0 option, the
sample result under the All Spread option was obtained more frequently from the spread plate
results, and fewer results of zero were encountered.
o Nonzero filter plate results were selected to represent 12 of 18 air samples and 36 of 136
surface samples.
o Slightly under half of the surface samples (66 of 136) were assigned a result of 0 CFU/m2
under this option. One of 18 air samples was assigned a result of 0 CFU/m3.
o For five of the seven indoor air samples having nonzero values for unadjusted mean
spread plate count, and for 34 of the 41 such surface samples, the result was based on the
spread plate outcome.
20
-------
• The Substitute 15 option used substitution techniques in all cases where unadjusted mean spread
plate counts were less than quantification limits, and thus, there were no outcomes equal to zero.
o Spread plate results were selected to represent all air samples and all 136 surface
samples.
o These substitutions imply that samples having true 0 CPU counts will be represented by
some positive count, leading to overestimates. As a result, the summary statistics were
orders of magnitude higher than for the other options.
o For the surface samples, the range of results did not overlap among samples with
different sampling methods:
• Swab samples: Each of the 11 swab samples had a result of 290,698 CFU/m2
(the spore content of each sample was <30 CPU which was therefore substituted
by 15 CPU, each had an elution suspension volume of 5 mL, and each had a
sample area of 0.00258 m2).
• Wipe samples: The result for each of the 20 wipe samples was 46,512 CFU/m2
(the spore content of each sample was <30 CPU thus substituting 15 CPU for the
result, each had an elution suspension volume of 20 mL, and each had a sample
area of 0.0645 m2).
• Sponge samples: Results for 69 samples ranged from 6,744 to 16,512 CFU/m2
(the result for each of the 69 sponge samples was non-detected and thus was
substituted with 15 CPU; each had a sample area of 0.0645 m2, but elution
suspension volumes ranged from 2.9 to 7.1 mL among these samples).
• Vacuum samples: Except for one vacuum sample, each sample result equaled
2,018 CFU/m2 (which resulted from a substitution of 15 CPU due to being a non-
detect, each had an elution suspension volume of 5 mL, and each had a sample
area of 0.3716 m2) - the one vacuum sample with a detected outcome had a result
of 3,991 CFU/m2 (See table A-2).
o For the air samples, the results ranged from 7,733 to 9,957 CFU/m3; all 18 samples
underwent substitution with 15 CPU and all sample durations were 15 minutes, but
elution suspension volumes ranged from 10.2 to 11.6 mL and air sampling flow rates
ranged from 11.65 to 13.39 L/min.
• Like the Substitute 0 option , the < Quantification (Spread only) option was equivalent to using
the filter plate result for all samples but the one surface sample where the unadjusted mean spread
plate count equaled 30 CPU. (This was because the other spread plate sample results were
21
-------
identified as less than the quantification limit, and the detection limit for the filter plate was
always less than the quantification limit for the spread plate.)
o For 2 of 18 air samples and 75 of 136 surface samples, the results were identified as
being censored. Censored values were identified only with those samples that have non-
detected outcomes for the filter analysis. The nonzero results of the Substitute 0 option
matched the detected results used in the < Quantification option.
o For some samples, results were identified as censored at limits that fell above the
outcomes of some detected samples, and the censored limits differed considerably among
samples. For example, occurrences of censored results among surface samples (taken
from filter analysis) were as follows:
• Vacuum samples: <8 CFU/m2 (0.3716m2 sample area, elution suspension
volume of 5 mL, average of two plates - i.e., the detection limit was based on a
potential CPU count of 0.5)
• Sponge samples: <31, <47, or <62 CFU/m2 (0.0645 m2 sample area, elution
suspension volumes ranged from 3.4 to 7.0 mL, average of two plates)
• Wipe samples: <310 CFU/m2 (0.0645 m2 sample area, elution suspension
volume of 20 mL, only one plate - i.e., the detection limit was based on a
potential CPU count of 1)
• Swab samples: <1,163 CFU/m2 (0.00258 m2 sample area, elution suspension
volume of 5 mL, average of two plates)
o Less-than values based on the spread plates at the quantification limit, were at least two
orders of magnitude greater than the associated less-than values based on the filter plate
detection limits.
Under the < Detection option , all results identified as detected matched those nonzero results
used in the All Spread option for the given sample, while all results labeled as censored (i.e., not
detected for both spread plate and filter plate analyses) were censored at the same limit as in the <
Quantification option. This is due to the filter plate analysis always resulting in the lower
censoring limit for samples under both options.
o Despite the spread plate censored value declining from the < Quantification option to the
< Detection option (as its interpretation shifts from a quantification limit to a detection
limit), the censored value associated with the filter plate was still used in all instances
when the result was non-detected under both analyses.
22
-------
o As expected, the < Detection option resulted in fewer sample results identified as
censored compared to the < Quantification option, due to the decline in the number of
samples whose spread plate result is classified as censored.
o Generally, when the sample outcome changed from the < Quantification option (either
from a censored result to a detected result, or among detected results), the outcome was
higher for the < Detection option. As a result, the summary statistics were higher under
the < Detection option than for the < Quantification option, especially in the upper tail of
the distribution.
• The < Quantification - Both Methods option resulted in outcomes falling below quantifiable
limits (i.e., left-censored) for all 18 indoor air samples, and for all but five of the 136 surface
samples. Because no quantifiable outcomes occurred among the indoor air samples under this
option, no summary statistics are presented for this option in Table 2a (and no 95% UCL
calculations can be performed), and the summary statistics in Table 2b are based on only five
sample measurements.
o All vacuum samples below quantification limits are portrayed as <269 CFU/m2. (Two
vacuum samples had quantifiable results: 681 CFU/m2 and 3,991 CFU/m2.)
o All sponge samples fall between 501 and 2,500 CFU/m2. (Three sponge results were
quantifiable: 1,705 CFU/m2, 2,093 CFU/m2, and 2,326 CFU/m2.)
o All wipe samples are portrayed as <6,202 CFU/m2.
o All swab samples are portrayed as <38,760 CFU/m2.
Figures la through Ic (for air samples) and Figures 2a through 2c (for surface samples) present the results
in histograms by data interpretation option. In these histograms, the vertical axis represents the number of
samples having results within the range specified on the horizontal axis (beneath each bar). In these
histograms, results falling below quantification or detection limits are portrayed by their respective limits.
To facilitate comparison of the data distribution and trends across the four data interpretation options in
Figures la and 2a, the range of the vertical axis and the categories on the horizontal axis are consistent
among the four histograms. However, for both air and surface samples, the histograms for the Substitute
15 option (Figures Ib and 2b) and the < Quantification - Both Methods option (Figures Ic and 2c) have
different axis ranges from the other options as their data were considerably higher in magnitude and thus
different from the others.
23
-------
Result )
V^ v^ -^ J" •*
'&'•&•$'•&•&•$
Result (CFU/m5)
c?
*
Result (CFU/m'(
(Note: The < Quantification option had two censored results (<55 and <59 CFU/m3) that were reported as 0
CFU/m3 for the Substitute 0 option. The < Detection option had one less-than result (<59 CFU/m3) that was reported
as 0 CFU/m3 for the All Spread option. These samples are represented by their censored results in the bottom two
histograms.)
Figure la. Histograms of Air Sample Concentrations (CFU/m3) for Four Data Interpretation
Options (n=18)
24
-------
Result (CFU/m3)
Figure Ib. Histogram of Air Sample Concentrations (CFU/m3) for the Substitute 15 Data
Interpretation Option (n=18)
10
9
< Quantification - Both Methods
V* Vs"
\> \> \v \y
Concentration (CFU/m3)
Figure Ic. Histogram of Air Sample Concentrations (CFU/m3) for the < Quantification - Both
Methods Option (n=18)
25
-------
100
90
SO
70
jj so
a.
I 5°
» 40
30
20
10
0
< Quantification /
Values equal 3 991 and
32,171 CFU/m2
\
Result (CFU/m!)
Values range from 3,721-9,922,
38,760, and 64,729 CFU/m
V S' V V V V V
Result (CFU/m2)
(Note: The < Quantification option had 75 censored results, all of which had results of 0 CFU/m2 for the Substitute
0 option. The < Detection option had 66 censored results, all of which had results of 0 CFU/m2 for the All Spread
option. These samples are represented by their censored results [<8, <31, <47, <62, <310, and <1,163 CFU/m2] in
the bottom two histograms.)
Figure 2a. Histograms of Surface Sample Concentrations (CFU/m2) for Four Data Interpretation
Options (n=136)
26
-------
Result (CFU/m2)
Figure 2b. Histogram of Surface Sample Concentrations (CFU/m2) for the Substitute 15
Treatment Option (note: horizontal axis not to scale) (n=136)
< Quantification - Both Methods
Concentration (CFU/m2)
Figure 2c. Histogram of Surface Sample Concentrations (CFU/m2) for the < Quantification - Both
Methods Option (note: horizontal axis not to scale) (n=136)
27
-------
The histograms for Substitute 15 option (Figures Ib and 2b) and the < Quantification - Both Methods
option (Figures Ic and 2c) show that these options were prone to generating very high results for some
samples. These very high results tended to be clustered, or separated from the distribution of results for
the other samples, suggesting that all results do not adhere to a common normal, lognormal, or gamma
distribution. For example, under the Substitute 15 option, the result for one air sample (9,957 CFU/m3)
was about 13% higher than the next highest air sample result. The separation was much more prominent
for the surface sample results, where, as noted earlier, the swab results (290,698 CFU/m2 for all 11
samples under the Substitute 15 option) were an order of magnitude higher than for all other sample types
(due to very small swab surface area sampled), and the wipe results (46,512 CFU/m2 for all 20 samples)
were at least three times higher than sponge and vacuum samples. Thus, it is apparent from these
histograms and data investigation that the method of substitution used in the Substitute 15 and <
Quantification - Both Methods options not only yielded much higher results than the other options, but
they were much more prone to yielding outliers that would have a large impact on the 95% UCL on the
mean calculation. The data distribution depends heavily on the range of surface areas sampled under both
options.
The All Spread option and the < Detection option were prone to generating more large outliers compared
to the Substitute 0, < Quantification -Identify if Less then, and < Quantification-Both Methods options.
Under the All Spread and < Detection options, the air sample with the highest result (1,758 CFU/m3)
exceeded the sample with the next highest result by more than double. The result of 1,758 CFU/m3 was
due in part to the need to take a 100-fold dilution when analyzing this sample, versus a 10-fold dilution
that was taken for all other air samples. Two surface samples had results that were more than 4 times
higher than the other samples; both samples had small surface areas, and one of the two samples had a
100-fold dilution. This shows that when any counts are noted among the replicate plates for a sample of a
relatively small area, the result under either of these options will likely be very high when expressed as
CPU per unit area.
3.2 95% UCLs on the Mean
ProUCL was used to generate estimates for the 95% UCL on the mean for each data interpretation option
and each data set (indoor air and surface), using the data summarized in Section 3.1 (and presented in
Appendix A). While ProUCL offers many varied techniques that require various types and degrees of
distributional assumptions, all of which are detailed in EPA (2010), the final set of recommended 95%
UCL estimates was obtained without assuming a specific data distribution for the results, as a single
28
-------
distribution type cannot be discerned to hold across all options and for the post-decontamination samples
being utilized in this assessment. Estimates of the 95% UCL of the mean under other statistical
techniques (and assuming different distributional assumptions) are presented in Appendix C.
For indoor air samples, Table 3a presents estimates of the 95% UCL on the mean, while estimates for
surface samples are presented in Table 3b. Within these tables, the Substitute 0 option and the
-------
Table 3b. Recommended 95% Upper Confidence Limit (UCL) on the Mean, Using Surface
Sample Concentrations (CFU/m2) for Each of the Six Data Interpretation Options
Recommended
UCL
Calculation
Method
95% UCL on
the mean
(CFU/m2)
95% UCL on
the mean
(CFU/m2)
Data Interpretation Options
1.
Substitute 0
- Treat as
Detect
Chebyshev
4.
-------
95% UCL estimates for these three options are those that are relevant under a lognormal distribution
assumption.
In general, for the samples included in this evaluation, using results identified as censored had a relatively
small effect on the 95% UCL on the mean calculation, compared to how the spread plate results were
used to represent the sample result (i.e., unadjusted mean spread plate counts less than the quantification
limit (30 CPU) treated as zeroes, all counts <30 CPU treated as valid, or treated as 15 CPU). This was
seen by noting that the 95% UCL on the mean calculations were similar between the two paired data
interpretation options (i.e., the options within bolded lines within Tables 3a and 3b), but differed
considerably between different pairs of options. The 95% UCL on the mean under the All Spread option
was over twice the size of the UCL under the Substitute 0 option for surface samples (over four times the
size when excluding the outlier), and over five times the size of the Substitute 0 option result for air
samples.
The 95% UCLs on the mean results under Substitute 15 option were much higher (by an order of
magnitude) than for the other options. For air samples, removal of the outlier had a minor effect on the
calculation of the UCL. For surface samples, the 95% UCL estimates under < Quantification - Both
Methods option are highly variable, as they are based on only five detected outcomes out of the 136
sample results. Thus, the two recommended estimates from ProUCL, both nonparametric-based, are an
order of magnitude different (Table 3b). (To ensure appropriate coverage, the higher of the two
recommended estimates should be selected as a conservative estimate.)
31
-------
4.0 Discussion
4.1 Statistical Approach
The generation of summary statistics on the Bg spore data set, including a 95% UCL on the mean, needed
to account fortwo issues: (1) the presence of censored results, and (2) the count-based nature of the
unadjusted data. Within this report, filter plate results were considered censored if no CPU were observed
on the plates (i.e., non-detect), or in the case of one option, if unadjusted filter plate counts were <20 CPU
(i.e., below a quantification limit). Spread plate results were considered censored if either no CPU were
observed on the plates (i.e., non-detect), or if the unadjusted mean spread plate counts were <30 CPU
(i.e., below the quantification limit).
If censored data are not handled appropriately (e.g., treated as detected, or use of various substitution
approaches), this can result in descriptive statistics that do not adequately represent the true underlying
data distribution (Helsel, 2005). For censored data sets, Helsel (2005) indicated that the following
methods are among those preferred for generating statistics over substitution methods:
• MLE - Uses detected observations and censored limits to generate summary statistics that are
expected to have produced both the detect and non-detect data.
• Imputation (e.g., ROS) - Non-detects are assigned values determined from the distribution of the
detected data (e.g., probability plots of detects). Not all non-detects are assigned the same value.
• Kaplan-Meier - A nonparametric approach (i.e., does not assume a specific distributional model)
that estimates the cumulative distribution function of data in the presence of multiple detection
limits, then generates statistics that are based on this estimated distribution.
These and other methods have been incorporated into EPA's ProUCL software (Version 4.1), which was
used in this evaluation. As described by EPA (2010), ProUCL provides several parametric and
nonparametric methods that can be used with uncensored (100% detected) and censored (censored limits)
observations at multiple quantification/detection limits, such as the Kaplan-Meier and ROS. ProUCL
helps identify an appropriate statistical method by applying goodness-of-fit tests relative to a specified
distributional model (e.g., normal, lognormal, gamma) and making recommendations on a method based
on the outcome of these tests and other properties of the data (e.g., standard deviation, skewness, sample
size, number of censored limits). Nonparametric methods are available and preferred if none of the
distributional models are deemed adequate for the data being analyzed.
32
-------
Methods for computing the 95% UCL on the mean using ProUCL are well established for chemical
contamination in environmental samples. However, Brattin et al. (2012) remarked that some of the more
common statistical procedures in ProUCL may not be applicable for calculating a 95% UCL on the mean
for parameters measured using count-based analytical methods. In particular,
• Counts of zero prohibit ProUCL from evaluating gamma and lognormal data distributions or
applying statistical procedures that assume these underlying distributions hold.
• ProUCL tests for goodness-of-fit to normal, lognormal, and gamma distributions which are
continuous in nature, while count-based data are inherently discrete.
• ProUCL assumes only inter-sample variation, while count-based data may also consist of random
Poisson counting variation.
The complicating nature of analyzing count-based data was also described by Petterson et al. (2001):
"Statistical analysis of data from microbial count experiments has traditionally involved converting all
counts to concentrations by dividing by sample volumes, followed by analysis assuming a continuous
statistical distribution. At low concentrations, stochastic variability in sampling is not negligible and
microbial counts from a single well-mixed suspension may not be assumed to be uniform. Analysis of the
data using counting statistics (discrete rather than continuous distributions) allows for consideration of
sampling variability, and differences in information content (low counts versus high counts) to be
properly addressed."
If a Poisson distribution is assumed due to the counting nature of the unadjusted data, the mean is
estimated by the arithmetic mean (as usual), but the standard deviation is estimated by the square root of
the arithmetic mean, which tends to underestimate the actual standard deviation in these data when the
mean is small. Furthermore, when the mean is large, the 95% UCL would be calculated in the same way
as if a normal distribution was assumed, using the mean and standard deviation calculated under the
Poisson distributional assumption.
The necessity of considering discrete microbial distributions versus continuous distributions may depend
on the average level of contamination. For example, as just noted, a normal approximation to the Poisson
(discrete) distribution can hold when the average contamination level is high. At very low average levels
of contamination, heterogeneity becomes more important, as, for example, in exposure analyses.
33
-------
4.2 All Spread Option
The All Spread option for interpreting the spore data selects the higher result from filter plating or spread
plating to represent the sample, without qualifying the result as being below detection or quantification
limits (i.e., all data are treated as detected). As such, this option is expected to yield more accurate results
than the two substitution options (Substitute 0 or Substitute 15), although estimates of accuracy are made
without explicitly known CPU levels. Substituting 0 CPU when spores are present understates the results.
Substituting 15 CPU may understate or overstate the results, depending on whether the actual number of
viable spores in samples with non-quantifiable spread-plate outcomes is greater than or less than 15 CPU.
Because the data examined in this study were post-decontamination, the number of residual spores
detected was low - substantially less than 15 CPU - and therefore substituting 15 CPU for non-
quantifiable outcomes yielded results much higher than was observed as colonies on plates. Thus, the
Substitute 15 option is most likely not appropriate for settings in which very low spore counts are
typically present.
Filter plates are suitable for detection and quantification of spores especially at low environmental
concentrations. For example, filter plates were used to analyze surface samples collected after a low (3 to
200 CPU/100 cm2) loading with Bacillus anthracis spores (Estill et al., 2009). Krauter et al. (2012) used
filter plating to maximize the detection of low numbers of Bg spores from surface samples; if no spores
were observed on spread plates, a portion of the elution suspension was then filter plated.
ASTM (2004) methods for water state that all colonies on spread plates should be counted when
microbial counts are low. Sutton (2006) also noted that plate counting guidance varies by organization,
and some report colonies below the countable range for spread plates (e.g., <30 CPU) as an estimated
count.
Helsel (2005) and other researchers have cautioned against using analytical data below the reporting limit
as uncensored data, or using any type of substitution approach to analyze non-detects or non-quantifiable
outcomes. From an analytical chemistry perspective, Helsel argued that readings below the reporting limit
cannot be identified as being different from zero or from each other. It is uncertain if this concern is valid
for microbiological spread plate and filter plate data as well, but for similar reasons, some researchers
have avoided using spread plate results below the quantification limit, using filter plate results instead. A
validation study (Rose et al. 2011) for the recovery of Bacillus anthracis spores from surfaces used filter
plate results if the spread plate results fell below the quantification limit (<25 CPU). Rose et al. (2011)
34
-------
filtered two 1-ml aliquots of spore elution suspensions to detect low numbers of spores. If the spread plate
counts were <25 CPU, then the filter plate CPU counts were used for quantification. The authors
considered this analysis of cellulose sponge wipe processing to be a validated method with oversight
being provided by the Centers for Disease Control and Prevention Laboratory Response Network review
committee. Calfee et al. (2012) used filter plates to detect Bg spores collected from surface samples if
fewer than 30 CPU were detected on spread plates.
Given the bias noted for substitution approaches, it appears appropriate to use the reported unadjusted
mean spread plate counts <30 CPU in determining the sample CPU results, without substituting alternate
values for these outcomes.
4.3 Substitution Options
The options of substituting mean spread plate counts <30 CPU (i.e., below the quantification limit) with
either 0 CPU or 15 CPU appeared to underestimate and overestimate results, respectively. For example,
when using the Substitute 0 option, the mean sample results for air and surface samples were
approximately half the results associated with the All Spread option. When using the Substitute 15 option,
the mean sample results were 30 times the results associated with the All Spread option. When
substituting results for all samples below the quantification limit with 15 CPU and treating them as
detected outcomes, the results became considerably inflated, as CPU counts per spread plate and filter
plate analyses were actually often below the substituted value. In this study, substituting 15 CPU for
samples with <30 CPU on spread plates biased the results high. Depending on the actual number of CPU
counts in the sample, substituting 15 CPU may result in a high or low bias.
EPA (2010) acknowledged that substituting one-half of the detection limit for non-detected outcomes can
introduce bias, even with few (5-10%) non-detects. Substituting zero for <30 CPU on spread plates biases
results in underestimates for the affected samples and thus does not reflect a conservative approach for
post-decontamination sampling. Helsel (2005) also noted that substitution approaches can hide real trends
while potentially introducing false trends in the data, and therefore, does not recommend substituting
values for non-detects or non-quantified values and treating the outcome as detected. Helsel (2010)
referred to substitution as a flawed method (except possibly when estimating a mean for a data set with
only one less-than threshold) and clarified that substitution does not equate to imputation.
35
-------
4.4 "Less-Than" Options
The three < Quantification and < Detection options qualified sample results as censored (less-than) if
below the associated quantification limits or detection limit. Incorporating this qualification into the data
analysis required alternative statistical approaches to calculate 95% UCL on the mean (e.g., Kaplan-Meier
methods within ProUCL) compared to when all sample results were treated as detected. However, with
the data set evaluated, 95% UCL on the mean values were similar to one of the other options that did not
account for censoring. For example, the surface sample 95% UCL on the mean for the Substitute 0 option
was 1,516 CFU/m2 while the < Quantification option was 1,533 CFU/m2; the 95% UCL on the mean
values were 3,613 CFU/m2 forthe All Spread option and 3,638 CFU/m2 forthe < Detection option.
If quantification limits are associated with both filter and spread plate analyses (as with the
-------
validation is traditionally an analyte- and sample-specific evaluation of data quality often conducted by a
party independent of data generation and data use (EPA, 2002 and 2009b). Data validation, as noted here,
primarily refers to the assigning of qualifiers (or flags) to the data in order to identify potential
deficiencies about data quality. For example, data qualifiers typically associated with chemical data
include:
• U - The analyte was not detected above the quantification limit
• J - The analyte is determined to be present in the sample, but its concentration is uncertain
• R - Sample results rejected
Data qualifiers for chemical data may also be associated with method blanks, surrogate recoveries,
holding times, matrix spikes and matrix spike duplicates, and field duplicates (EPA, 2002 and 2009b).
Unique data qualifiers may be needed for microbiological data, although procedures for validating
microbiological data were not identified in the literature. A data validation step where microbiological
analytical results are qualified and flagged may aid in the interpretation of environmental microbiology
data. For example, a validation process could distinguish microbiological results below quantification
limits. Appropriate quality control procedures, such as use of split samples or duplicate samples, could
also contribute to information on the quality of microbiological results.
Validation could also draw attention to somewhat anomalous results that might influence the data set. As
described in Section 3.0, there were three samples that each had a single CPU observed in the second
serial dilution during spread plating. Although no colonies were observed at the first dilution, application
of a dilution factor of 100 was required, (leading to the some of the largest results observed for the
unsubstituted options). Two of the associated results were also identified as potential outliers (1,758
CFU/m3 and 64,729 CFU/m2) in Tables 3a and 3b. Data validation flags associated with these results
could have alerted the user to the uniqueness of these data. Processes for handling such unusual plate
counting situations (such as two dilutions with countable colonies) can vary and whichever method is
desired should be documented and justified within a standard operating procedure (Sutton, 2006). The
approach in the current evaluation used the dilution with the largest mean spread plate count. Other than
the three samples described in this paragraph that were based on a dilution factor of 100, all spread plate
results were based on a dilution factor of 10.
37
-------
4.6 Data Groupings
For this assessment, all surface data were combined across sampling methods (swab, wipes, etc.) when
generating summary statistics, including the 95% UCL on the mean. However, large differences were
observed in the data values among different types of surface samples, primarily due to the differences in
sampled areas that are linked to the sampling method. Thus, in order to ensure the calculated 95% UCL is
not dominated by differences among sampling methods, it may be necessary to select a single sampling
method that most appropriately yields samples and results that will address the sampling objectives. For
example, when CPU are present in swab and wipe samples, the very small area associated with these
samples will lead to very high results when expressed per square meter, compared to vacuum and sponge
samples that are collected over a wider area, and subsequently, the CPU counts on a per area basis are
lower. Recovery efficiency (likely influenced by sampling method and material sampled) may also need
to be accounted for when analyzing the data.
38
-------
5.0 Summary
This study analyzed data from air and surface samples for which CPU of Bg spores were determined both
by spread plate and filter plate methods. Having paired results for each sample representing the two
methods provided an opportunity to assess the impact of alternative methods of addressing low or no
CPU counts on filter plates and spread plates used to analyze environmental microbiological samples.
Paired spread plate and filter plate results were available for 154 samples (18 indoor air samples collected
with SKC BioSampler® units and 136 surface samples collected with sponges, swabs, vacuums, or
wipes). These samples were collected following decontamination and resulted in relatively low level Bg
detections with many results below detection or quantification limits. The data associated with these
samples were used to evaluate each of six data interpretation options for analyzing censored
microbiological data. The data interpretation options are summarized in Table 1 (Section 2.3).
Comments on the various data interpretation options are provided below and are based on comparison of
the means and 95% UCL on the mean calculated using each option. The findings of this assessment are
subject to the type of data that were considered - for example, the unadjusted mean spread plate count for
all but one sample in the data set was reported to be less than the quantification limit (i.e., <30 CPU).
Therefore, results and conclusions may be different if spread plate results (based on counts above the
quantification limit) were more prevalent in the data. In addition, although conclusions related to
estimation bias (i.e., underestimation, overestimation) were made for these data interpretation options, a
more complete assessment of bias needs to consider data with a higher prevalence of samples with known
CPU counts. Selection of a recommended option will depend on the type of analysis being conducted
(e.g., a screening level assessment versus a more detailed analysis), whether the data are being treated as
discrete or continuous, and whether results less than the quantification limit are deemed suitable for use.
Results for the Substitute 0 option indicated results were biased low as this option had the lowest means
and 95% UCL on the mean results among all options evaluated. Here, all nonzero sample results (except
one surface sample) were based on the filter plate results.
The All Spread option reflected the level of Bg spores collected more accurately than the Substitute 0 or
Substitute 15 option. The All Spread mean and 95% UCL on the mean results were more than double that
of the Substitute 0 option. Higher results imply greater accuracy because, in theory, colonies would only
be present if culturable bacteria were actually present. Relative to the Substitute 0 option, fewer sample
results were based on filter plate results and the number of zero results was reduced.
39
-------
The Substitute 15 option biased the results rather high. The Substitute 15 option mean and 95% UCL on
the mean results were an order of magnitude higher than the results of all other data interpretation options.
All Substitute 15 results were based on spread plating, and other than the one surface sample with a mean
spread plate count of 30 CPU, the sample results were based on a substituted mean spread plate count of
15 CPU. Although not used for this option, the reported unadjusted mean spread plate counts were all <10
CPU and most (146 of 154 samples) were <2 CPU, which is considerably lower than the substituted value
of 15 CPU. If the true mean of the samples had been higher than 15 CPU, the Substitute 15 option would
bias the results low.
The < Quantification option generated values for the 95% UCL on the mean that were similar to the
Substitute 0 approach (likely biased low). Mean results were higher, especially for the surface samples,
for the < Quantification option than the Substitute 0 option as mean results for the < Quantification option
were only based on detected values. This option might be beneficial for generating 95% UCL on the mean
for data sets where results below the quantification limit are deemed unusable.
The < Detection option generated values for the 95% UCL on the mean that were similar to the All
Spread option (expected to most accurately reflect Bg spore contamination). The calculated means were
higher (especially for the surface samples) for the < Detection option than the All Spread option as the
mean results for the < Detection option were only based on detected values. This option may be beneficial
to avoid the presentation of zero results, which might imply more certainty in the data than what actually
may exist.
The < Quantification - Both Methods option yields unstable estimates for the 95% UCL, as this option
leads to a large proportion of samples with non-quantifiable results. All air sample results were non-
quantifiable in this analysis, and all but five surface sample results were non-quantifiable. Thus, 95%
UCL estimates could be made only for surface samples. However, this option would be preferred if, in
fact, any sample outcome should be deemed non-quantifiable when either spread plate or filter plate
analysis are below their respective quantification limits.
Overall, in estimating the 95% UCL on the mean, ProUCL can handle spore concentration data at
multiple censoring limits and offers nonparametric approaches (Kaplan-Meier-based) when distributional
assumptions are not achieved.
40
-------
Based on the results from the data included in this evaluation:
• If high variability and uncertainty in low concentration estimates is considered acceptable, then
the < Detection option could be the best option for handling censored observations. The <
Detection option maximally utilizes all available information to provide conservative estimates of
concentrations and indicates uncertainty associated with non-detection.
• If high variability and uncertainty in low concentration estimates is considered unacceptable, then
the < Quantification - Both Methods option could be the most useful option for handling censored
observations This option would require appropriate justification for the quantification and
detection limits that are used to represent censored outcomes.
Note that it was desired to identify an option that yielded the best estimate for the 95% UCL in the
presence of censored outcomes while taking advantage of all available information on spread plate and
filter plate results for each sample. Thus, this investigation favored one option over another based on its
ability to generate an upper confidence limit on the mean whose expected coverage percentage was
closest to 95%. It did not consider, for example, which option led to the most conservative estimate that it
likely to have a higher coverage percentage than 95%.
Future work to further evaluate the accuracy and precision of these or other options for data interpretation
is warranted because of the potential importance of low-level human exposures to biological agents.
Accurately knowing the concentration of biological agents on surfaces or in air is necessary for
performing meaningful exposure assessment. The design of the study could include larger numbers of
samples containing accurately known concentrations of Bg spores that could be evaluated using
alternative data interpretation options.
41
-------
6.0 References
Almeida, J.L., B. Harper, and K.D. Cole, 2008. Bacillus anthracis spore suspensions: determination of
stability and comparison of enumeration techniques. Journal of Applied Microbiology 104(5): 1442-1448.
ASTM, 2004. Standard Practice for Determining Microbial Colony Counts from Waters Analyzed by
Plating Methods. ASTM Standard D5465, 1993 (2004), ASTM International, West Conshohocken, PA.
Baron, P.A., C.F. Estill, G.J. Deye, M.J. Hein, J.K. Beard, L.D. Larsen, and G.E. Dahlstrom, 2008.
Development of an aerosol system for uniformly depositing Bacillus anthracis spore particles on
surfaces. Aerosol Science and Technology 42(3): 159-172.
Brattin, W., T. Barry, and S. Foster, 2012. Estimation of the upper confidence limit on the mean of
datasets with count-based concentration values. Human and Ecological Risk Assessment: An
International Journal 18(2):435-455.
Breed, R.S. and W.D. Dotterrer, 1916. The number of colonies allowable on satisfactory agar plates.
Journal of Bacteriology 1(3):321-331.
Brown, G.S., R.G. Betty, J.E. Brockmann, D.A. Lucero, C.A. Souza, K.S. Walsh, R.M. Boucher, M.
Tezak, M.C. Wilson, and T. Rudolph, 2007a. Evaluation of a wipe surface sample method for collection
of Bacillus spores from nonporous surfaces. Applied and Environmental Microbiology 73(3):706-710.
Brown, G.S., R.G. Betty, J.E. Brockmann, D.A. Lucero, C.A. Souza, K.S. Walsh, R.M. Boucher, M.S.
Tezak, and M.C. Wilson, 2007b. Evaluation of vacuum filter sock surface sample collection method for
Bacillus spores from porous and non-porous surfaces. Journal of Environmental Monitoring 9(7):666-
671.
Brown, G.S., R.G. Betty, J.E. Brockmann, D.A. Lucero, C.A. Souza, K.S. Walsh, R.M. Boucher, M.S.
Tezak, M.C. Wilson, T. Rudolph, H.D.A. Lindquist, and K.F. Martinez, 2007c. Evaluation of rayon swab
surface sample collection method for Bacillus spores from nonporous surfaces. Journal of Applied
Microbiology 103(4): 1074-1080.
42
-------
Calfee, M.W., S.P. Ryan, J.P. Wood, L. Mickelsen, C. Kempter, L. Miller, M. Colby, A. Touati, M.
Clayton, N. Griffm-Gatchalian, S. McDonald, and R. Delafield, 2012. Laboratory evaluation of large-
scale decontamination approaches. Journal of Applied Microbiology 112(5):874-882.
Clark, H.F., E.E. Geldreich, H.L. Jeter, and P.W. Kabler, 1951. The membrane filter in sanitary
bacteriology. Public Health Reports 66(30): 951-977.
Dixon, W.J., 1950. Analysis of extreme values. The Annals of Mathematical Statistics 21(4):488-506.
Edmonds, J.M., 2009. Efficient methods for large-area surface sampling of sites contaminated with
pathogenic microorganisms and other hazardous agents: current state, needs, and perspectives. Applied
Microbiology and Biotechnology 84(5):811-816.
El-Shaarawi, A.H., 1989. Inferences about the mean from censored water quality data. Water Resources
Research 25(4):685-690.
EPA, 2013. Bio-response Operational Testing and Evaluation (BOTE) Project - Phase 1:
Decontamination Assessment. U.S. Environmental Protection Agency, Office of Research and
Development, National Homeland Security Research Center, Washington, DC, EPA/600/R-13/168,
December.
EPA, 2012a. Selected Analytical Methods for Environmental Remediation and Recovery (SAM) - 2012.
U.S. Environmental Protection Agency, Office of Research and Development, National Homeland
Security Research Center, Cincinnati, OH, EPA/600/R-12/555, July.
EPA, 20l2b.Microbial Risk Assessment Guideline: Pathogenic Microorganisms with Focus on Food and
Water., Prepared by Interagency Microbial Risk Assessment Guideline Workgroup July 2012, U.S.
Environmental Protection Agency and U.S. Department of Agriculture/Food Safety and Inspection
Service, EPA/ 100/J-12/001; USDA/FSIS/2012-001.
EPA, 2010. ProUCL Version 4.1 User Guide (Draft), Statistical Software for Environmental Applications
for Data Sets with and without Nondetect Observations. U.S. Environmental Protection Agency, Office of
Research and Development, Washington, DC, EPA/600/R-07/041, May.
43
-------
EPA, 2009a. Method Validation of U.S. Environmental Protection Agency Microbiological Methods of
Analysis. The EPA Forum on Environmental Measurements (FEM) Microbiology Action Team. FEM
Document Number 2009-01.
EPA, 2009b. Guidance for Labeling Externally Validated Laboratory Analytical Data for Superfitnd Use.
U.S. Environmental Protection Agency, Office of Solid Waste and Emergency Response, Washington,
DC, EPA/540/R-08/005, January.
EPA, 2007. ProUCL Version 4.0 Technical Guide. U.S. Environmental Protection Agency, Office of
Research and Development, Washington, DC, EPA/600/R-07/041, April.
EPA, 2002. Guidance on Environmental Data Verification and Data Validation. U.S. Environmental
Protection Agency, Office of Environmental Information, Washington, DC, EPA/240/R-02/004,
November.
EPA, 1992. Guidelines for Exposure Assessment. U.S. Environmental Protection Agency, Risk
Assessment Forum, Washington, DC, EPA/600/Z-92/001, May.
EPA, 1989. Risk Assessment Guidance for Superfund Volume I Human Health Evaluation Manual (Part
A). U.S. Environmental Protection Agency, Office of Emergency and Remedial Response, Washington,
DC, EPA/540/1-89/002, December.
Estill, C.F., P.A. Baron, J.K. Beard, M.J. Hein, L.D. Larsen, L. Rose, F.W. Schaefer III, J. Noble-Wang,
L. Hodges, H.D.A. Lindquist, G.J. Deye, and M.J. Arduino, 2009. Recovery efficiency and limit of
detection of aerosolized Bacillus anthracis Sterne from environmental surface samples. Applied and
Environmental Microbiology 75(13):4297-4306.
Frawley, D.A., M.N. Samaan, R.L. Bull, J.M. Robertson, A.J. Mateczun, and P.C.B. Turnbull, 2008.
Recovery efficiencies of anthrax spores and ricin from nonporous or nonabsorbent and porous or
absorbent surfaces by a variety of sampling methods. Journal of Forensic Sciences 53(5): 1102-1107.
Gilbert, R.O., 1987. Statistical Methods for Environmental Pollution Monitoring. New York: Van
Nostrand Reinhold Company.
44
-------
Gleit, A., 1985. Estimation for small normal data sets with detection limits. Environmental Science and
Technology 19(12): 1201-1206.
Goodwin, K.D., M. McNay, Y. Cao, D. Ebentier, M. Madison, and J.F. Griffith, 2012. A multi-beach
study of Staphylococcus aureus, MRSA, and enterococci in seawater and beach sand. Water Research
46(13):4195-4207.
Haas, C.N., J.B. Rose, C. Gerba, and S. Regli, 1993. Risk assessment of virus in drinking water. Risk
Analysis 13(5):545-552.
Hamilton, A.J., F. Stagnitti, R. Premier, A.-M. Boland, and G. Hale, 2006. Quantitative microbial risk
assessment models for consumption of raw vegetables irrigated with reclaimed water. Applied and
Environmental Microbiology 72(5):3284-3290.
Helsel, D., 2010. Much ado about next to nothing: incorporating nondetects in science. The Annals of
Occupational Hygiene 54(3):257-262.
Helsel, D.R., 2005. More than obvious: better methods for interpreting nondetect data. Environmental
Science and Technology 39(20):419A-423A.
Hodges, L.R., L.J. Rose, A. Peterson, J. Noble-Wang, and M.J. Arduino, 2006. Evaluation of a
macrofoam swab protocol for the recovery of Bacillus anthracis spores from a steel surface. Applied and
Environmental Microbiology 72(6): 4429-4430.
Kaplan, E.L. and P. Meier, 1958. Nonparametric estimation from incomplete observations. Journal of the
American Statistical Association 53(282):457-481.
Krauter, P.A., G.F. Piepel, R. Boucher, M. Tezak, E.G. Amidan, and W. Einfeld, 2012. False-negative
rate and recovery efficiency performance of a validated sponge wipe sampling method. Applied and
Environmental Microbiology 78(3):846-854.
45
-------
Nichols, T, I. Baumel, and C. Sonich-Mullin, 2006. Framework for incident-based risk assessment for
biothreat agents: Proceedings of the Sixth International Chemical and Biological Medical Treatment
Symposium. Spiez Laboratory; paper no. 49, Spiez, Switzerland, April 30-May 5.
Parkin, R., 2008. Foundations and Frameworks for Microbial Risk Assessment. Center for Risk Science
and Public Health, School of Public Health and Health Services, The George Washington University
Medical Center Washington, B.C. Prepared for the U.S. Environmental Protection Agency, National
Center for Environmental Assessment.
Petterson, S., R. Signer, N. Ashbolt, and D. Roser, 2006. QMRA Methodology. The MicroRisk
Consortium.
Petterson, S.R., P.F.M. Teunis, and N.J. Ashbolt, 2001. Modeling virus inactivation on salad crops using
microbial count data. Risk Analysis 21(6): 1097-1108.
Rodda, N., A. Amory, and R. Kfir, 1993. The application of risk assessment techniques to microbial
monitoring data: a South African perspective. Water Science and Technology 27(3-4): 145-150.
Rose, L.J., L. Hodges, H. O'Connell, and J. Nobel-Wang, 2011. National validation study of a cellulose
sponge wipe-processing method for use after sampling Bacillus anthracis spores from surfaces. Applied
and Environmental Microbiology 77(23):8355-8359.
Rosner, B., 1983. Percentage points for a generalized ESD many-outlier procedure. Technometrics
25(2): 165-172.
Sanderson, W.T., M.J. Hein, L. Taylor, B.D. Curwin, G.M. Kinnes, T.A. Seitz, T. Popovic, H.T. Holmes,
M.E. Kellum, S.K. McAllister, D.N. Whaley, E.A. Tupin, T. Walker, J.A. Freed, D.S. Small, B. Klusaritz,
and J.H. Bridges, 2002. Surface sampling methods for Bacillus anthracis spore contamination. Emerging
Infectious Diseases 8(10): 1145-1151.
Signer, R.S., D.J. Roser, N.J. Ashbolt, and J.E. Ball, 2005. Quantifying the impact of runoff events on
microbiological contaminant concentrations entering surface drinking source waters. Journal of Water and
Health 3(4):453-468.
46
-------
SMC [Standard Methods Committee] American Public Health Association, American Water Works
Association, Water Environment Federation, 2011. 9275 Heterotrophic Plate Count. Standard Methods
for the Examination of Water and Wastewater. 22nd Edition. A.D. Eaton, E.W. Rice, R.B. Baird, A.D.
Eaton, L. S. Clesceri, Editors.
Smeets, P.W.M.H., J.C. van Dijk, G. Stanfield, L.C. Rietveld, and G.J. Medema, 2007. How can the UK
statutory Cryptosporidium monitoring be used for quantitative risk assessment of Cryptosporidium in
drinking water? Journal of Water and Health 5(Suppl 1): 107-118.
Sutton, S., 2006. Counting colonies. Pharmaceutical Microbiology Forum Newsletter 12(9):2-11.
Valentine, N.B., M.G. Butcher, Y.-F. Su, K.H. Jarman, M. Matzke, B.-J. Webb-Robertson, E.A. Panisko,
B.A.B. Seiders, and K.L. Wahl, 2008. Evaluation of sampling tools for environmental sampling of
bacterial endospores from porous and nonporous surfaces. Journal of Applied Microbiology 105(4): 1107-
1113.
Wong, M., L. Kumar, T.M. Jenkins, I. Xagoraraki, M.S. Phanikumar, and J.B. Rose, 2009. Evaluation of
public health risks at recreational beaches in Lake Michigan via detection of enteric viruses and a human-
specific bacteriological marker. Water Research 43(4): 1137-1149.
47
-------
Appendix A
Listings of Individual Sample Concentrations under the Six Data Interpretation Options
48
-------
Table A-l. Listing of Individual Air Sample Concentrations under the Six Data Interpretation Options
Barcode
4375
5028
5030
5033
5104
5110
5117
5133
5214
5258
5267
5300
5364
5382
5384
5409
5432
5959
Volume
(mL)
10.40
10.20
10.20
11.60
11.60
10.20
10.80
10.20
10.60
10.40
10.20
11.20
11.20
11.40
10.40
10.40
10.80
10.80
Flow
Rate
(L/min)
12.35
12.89
12.94
11.65
13.20
13.19
12.52
12.80
13.39
13.08
12.75
12.98
13.00
13.07
12.98
12.96
13.18
12.78
Dilution
Factor
10
100
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
Un-
adjusted
Mean
Spread
Plate
Count
<30 CFU?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Adjusted CFU
Spread
Plate
0
340
0
77
0
0
0
0
35
0
0
37
0
76
139
0
36
0
Filter
Plate
21
20
41
12
23
31
0
10
0
10
10
56
11
23
10
31
43
11
Concentration
(CFU/m3)
1.
Substitute 0
- Treat as
Detect
113
103
211
69
116
157
0
52
0
51
52
288
56
117
51
159
218
57
2. All
Spread
- Treat as
Detect
113
1758
211
441
116
157
0
52
174
51
52
288
56
388
714
159
218
57
3. Sub-
stitute 15 -
Treat as
Detect
8421
7913
7883
9957
8788
7733
8626
7969
7916
7951
8000
8629
8615
8722
8012
8025
8194
8451
4.
-------
Table A-2. Listing of Individual Surface Sample Concentrations under the Six Data Interpretation Options
Barcode
1481
1482
1520
1597
1605
1634
1637
1647
1653
1676
1682
1807
1812
1815
1819
1820
1825
1829
1837
1843
1846
1848
1851
1877
1910
1913
Volume
(mL)
5
5
5
5
5
5
5
5
5
5
5
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
Sampled
Area (m2)
0.00258
0.00258
0.00258
0.00258
0.00258
0.00258
0.00258
0.00258
0.00258
0.00258
0.00258
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
Dilution
Factor
10
10
10
100
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
Unadjus-
ted Mean
Spread
Plate Count
<30 CFU?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Adjusted CFU
Spread
Plate
0
0
0
167
0
0
0
0
0
100
0
67
267
0
67
0
0
0
0
67
0
0
0
0
67
0
Filter
Plate
0
0
0
8
0
0
0
0
0
83
5
20
140
60
20
0
0
0
0
120
0
0
0
20
20
0
Concentration
(CFU/m3)
1.
Substitute 0
- Treat as
Detect
0
0
0
3101
0
0
0
0
0
32171
1938
310
2171
930
310
0
0
0
0
1860
0
0
0
310
310
0
2. All
Spread
- Treat as
Detect
0
0
0
64729
0
0
0
0
0
38760
1938
1039
4140
930
1039
0
0
0
0
1860
0
0
0
310
1039
0
3. Sub-
stitute 15 -
Treat as
Detect
290698
290698
290698
290698
290698
290698
290698
290698
290698
290698
290698
46512
46512
46512
46512
46512
46512
46512
46512
46512
46512
46512
46512
46512
46512
46512
4. Quanti-
fication
- Identify if
"Less-Than"
1163
1163
1163
3101
1163
1163
1163
1163
1163
32171
1938
310
2171
930
310
310
310
310
310
1860
310
310
310
310
310
310
5. <
Detection
- Identify
if "Less-
Than"
1163
1163
1163
64729
1163
1163
1163
1163
1163
38760
1938
1039
4140
930
1039
310
310
310
310
1860
310
310
310
310
1039
310
6.
-------
Table A-2. (cont.)
Barcode
1930
1937
1942
1950
1955
2070
2072
2231
2233
2234
2235
2236
2240
2241
2266
2436
2566
2577
2582
2599
2601
2606
2652
2653
2654
2679
2721
Volume
(mL)
20
20
20
20
20
5.5
6.1
4.5
6.4
6.4
5.9
7.1
6.4
5.3
5.2
4.8
4.5
7
5.2
4.9
4.4
5.1
6
5.7
5.1
4.8
6.1
Sampled
Area (m2)
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
Dilution
Factor
10
10
10
10
10
10
10
100
100
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
100
10
Unadjus-
ted Mean
Spread
Plate Count
<30 CFU?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Adjusted CFU
Spread
Plate
0
133
133
0
0
0
20
345
640
0
0
0
0
0
0
0
15
0
0
0
0
0
0
57
0
240
0
Filter
Plate
0
160
0
0
0
0
40
0
150
0
0
14
0
0
0
0
0
0
0
0
0
0
0
29
0
0
0
Concentration
(CFU/m3)
1.
Substitute 0
- Treat as
Detect
0
2481
0
0
0
0
620
0
2326
0
0
217
0
0
0
0
0
0
0
0
0
0
0
450
0
0
0
2. All
Spread
- Treat as
Detect
0
2481
2062
0
0
0
620
5349
9922
0
0
217
0
0
0
0
233
0
0
0
0
0
0
884
0
3721
0
3. Sub-
stitute 15 -
Treat as
Detect
46512
46512
46512
46512
46512
12791
14186
10465
14884
14884
13721
16512
14884
12326
12093
11163
10465
16279
12093
11395
10233
11860
13953
13256
11860
11163
14186
4. Quanti-
fication
- Identify if
"Less-Than"
310
2481
310
310
310
47
620
31
2326
47
47
217
47
47
47
31
31
62
47
31
31
47
47
450
47
31
47
5. <
Detection
- Identify
if "Less-
Than"
310
2481
2062
310
310
47
620
5349
9922
47
47
217
47
47
47
31
233
62
47
31
31
47
47
884
47
3721
47
6.
-------
Table A-2. (cont.)
Barcode
2722
2723
2724
2725
2741
2742
2743
2749
2794
2825
2827
2839
2840
2844
2900
2914
2916
2957
3013
3063
3065
3066
3076
3077
3218
3238
3239
Volume
(mL)
5.9
3.4
6.5
5.5
4.7
5.8
6
5
5.2
5.2
4.5
3.8
4.9
4.5
4.4
5
6
3.7
3.5
3.9
3.8
4.3
6
5.5
4.4
2.9
5.4
Sampled
Area (m2)
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
Dilution
Factor
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
Unadjus-
ted Mean
Spread
Plate Count
<30 CFU?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Adjusted CFU
Spread
Plate
39
0
0
0
0
0
0
0
0
0
0
63
16
0
0
0
0
0
0
0
0
100
0
0
0
19
18
Filter
Plate
24
0
0
0
0
0
0
0
0
0
0
40
20
0
2
0
3
7
28
0
2
110
6
11
0
3
24
Concentration
(CFU/m3)
1.
Substitute 0
- Treat as
Detect
372
0
0
0
0
0
0
0
0
0
0
620
310
0
31
0
47
109
434
0
31
1705
93
171
0
47
372
2. All
Spread
- Treat as
Detect
605
0
0
0
0
0
0
0
0
0
0
977
310
0
31
0
47
109
434
0
31
1705
93
171
0
295
372
3. Sub-
stitute 15 -
Treat as
Detect
13721
7907
15116
12791
10930
13488
13953
11628
12093
12093
10465
8837
11395
10465
10233
11628
13953
8605
8140
9070
8837
10000
13953
12791
10233
6744
12558
4. Quanti-
fication
- Identify if
"Less-Than"
372
31
47
47
31
47
47
47
47
47
31
620
310
31
31
47
47
109
434
31
31
1705
93
171
31
47
372
5. <
Detection
- Identify
if "Less-
Than"
605
31
47
47
31
47
47
47
47
47
31
977
310
31
31
47
47
109
434
31
31
1705
93
171
31
295
372
6.
-------
Table A-2. (cont.)
Barcode
3242
3243
3244
3246
3247
3252
3254
3273
3277
3282
3283
3287
3495
3505
3506
3509
3513
3528
3529
3530
3549
3568
3569
3585
3592
3593
3594
Volume
(mL)
3.3
4.7
4
3.9
4.7
5.1
4.6
5.5
4.5
3.6
4
5
4
3.5
4.1
4.5
5
6
7
5
5
5
5
5
5
5
5
Sampled
Area (m2)
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.0645
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
Dilution
Factor
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
100
10
10
10
10
10
Unadjus-
ted Mean
Spread
Plate Count
<30 CFU?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Adjusted CFU
Spread
Plate
0
94
0
0
0
0
0
73
0
0
0
0
0
0
0
45
83
0
0
0
0
417
17
17
0
0
33
Filter
Plate
2
35
0
14
0
0
0
135
0
0
2
5
4
0
4
0
10
15
28
0
0
253
8
3
0
0
0
Concentration
(CFU/m3)
1.
Substitute 0
- Treat as
Detect
31
543
0
217
0
0
0
2093
0
0
31
78
62
0
62
0
155
233
434
0
0
681
22
8
0
0
0
2. All
Spread
- Treat as
Detect
31
1457
0
217
0
0
0
2093
0
0
31
78
62
0
62
698
1287
233
434
0
0
1122
46
46
0
0
89
3. Sub-
stitute 15 -
Treat as
Detect
7674
10930
9302
9070
10930
11860
10698
12791
10465
8372
9302
11628
9302
8140
9535
10465
11628
13953
16279
11628
2018
2018
2018
2018
2018
2018
2018
4. Quanti-
fication
- Identify if
"Less-Than"
31
543
31
217
31
47
31
2093
31
31
31
78
62
31
62
31
155
233
434
47
8
681
22
8
8
8
8
5. <
Detection
- Identify
if "Less-
Than"
31
1457
31
217
31
47
31
2093
31
31
31
78
62
31
62
698
1287
233
434
47
8
1122
46
46
8
8
89
6.
-------
Table A-2. (cont.)
Barcode
3595
3598
3599
3612
3616
3621
3629
3631
3633
3644
3648
3655
3658
3660
3663
3668
3674
3676
3686
3689
3714
3718
3719
3720
3722
3725
3868
Volume
(mL)
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
Sampled
Area (m2)
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
0.3716
Dilution
Factor
10
10
10
10
10
10000
10
10
10
100
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
Unadjus-
ted Mean
Spread
Plate Count
<30 CFU?
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Adjusted CFU
Spread
Plate
0
0
167
0
50
1483
67
33
0
167
17
0
0
117
0
0
0
17
0
0
0
17
50
0
0
67
0
Filter
Plate
5
0
88
18
13
465
0
3
0
0
5
3
0
63
0
0
5
0
8
3
10
10
8
0
0
38
3
Concentration
(CFU/m3)
1.
Substitute 0
- Treat as
Detect
13
0
237
48
35
3991
0
8
0
0
13
8
0
170
0
0
13
0
22
8
27
27
22
0
0
102
8
2. All
Spread
- Treat as
Detect
13
0
449
48
135
3991
180
89
0
449
46
8
0
315
0
0
13
46
22
8
27
46
135
0
0
180
8
3. Sub-
stitute 15 -
Treat as
Detect
2018
2018
2018
2018
2018
3991
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
2018
4. Quanti-
fication
- Identify if
"Less-Than"
13
8
237
48
35
3991
8
8
8
8
13
8
8
170
8
8
13
8
22
8
27
27
22
8
8
102
8
5. <
Detection
- Identify
if "Less-
Than"
13
8
449
48
135
3991
180
89
8
449
46
8
8
315
8
8
13
46
22
8
27
46
135
8
8
180
8
6.
-------
Table A-2. (cont.)
Barcode
3978
3981
Volume
(mL)
5
5
Sampled
Area (m2)
0.3716
0.3716
Dilution
Factor
10
10
Unadjus-
ted Mean
Spread
Plate Count
<30 CFU?
Yes
Yes
Adjusted CFU
Spread
Plate
0
0
Filter
Plate
5
5
Concentration
(CFU/m3)
1.
Substitute 0
- Treat as
Detect
13
13
2. All
Spread
- Treat as
Detect
13
13
3. Sub-
stitute 15 -
Treat as
Detect
2018
2018
4. Quanti-
fication
- Identify if
"Less-Than"
13
13
5. <
Detection
- Identify
if "Less-
Than"
13
13
6.
-------
Appendix B
Distributional Goodness-of-Fit Tests Applied to Each Data Interpretation Option
56
-------
Goodness-of-fit Testing. In order to ensure that the calculated UCL actually exceeds the unknown mean
with 95% likelihood, the statistical method used to calculate this value must be appropriate for the given
set of data. More specifically, if a statistical method requires the data to originate from a given statistical
distribution such as a normal distribution, then that method is not appropriate for the given data if it can
be demonstrated that the distribution of the data varies greatly from that assumed distribution. This is
typically demonstrated through applying a goodness-of-fit statistical test and assessing graphs of the data.
Therefore, ProUCL (Version 4.1 used for this evaluation) helps identify an appropriate statistical method
by applying goodness-of-fit tests for normal, lognormal, and gamma distributions and making
recommendations on a method based on the outcome of these tests and other properties of the data such as
standard deviation, skewness, sample size, and number of censored values. These goodness-of-fit tests
were performed on data for each of the six data interpretation options, as each option is based on the same
amount of data (that is, no one option yields a larger dataset compared to the others). Samples with values
falling below quantification or detection limits were represented by extrapolated values from a ROS
method. Within ProUCL, the specific test applied for a given distributional model depended on whether
or not the sample size exceeded 50; air samples had n=18, while surface samples had n=136:
• Test for Normality:
o Shapiro-Wilk test for air samples (Shapiro and Wilk, 1965); Lilliefors test for surface
samples (Lilliefors, 1967)
o Normal ROS estimates were used to represent non-quantifiable sample results in these
tests
• Test for Lognormality:
o Shapiro-Wilk test on log-data for air samples; Lilliefors test on log-data for surface
samples
o Lognormal ROS estimates were used to represent non-quantifiable sample results in these
tests
• Test for Gamma Distribution:
o Kolmogorow-Smirnov test for both air and surface samples (D'Agostino and Stephens,
1986).
o Gamma ROS estimates were used to represent non-quantifiable sample results in these
tests
These tests assume the (null) hypothesis that the given distributional model holds and requires sufficient
evidence from the data to reject this hypothesis. All goodness-of-fit tests were performed at the 0.05
significance level.
57
-------
Table B-l. Results of Distributional Goodness-of-Fit Tests Applied to the Air Sample Data (n=18
samples) for Each Data Interpretation Option
Data Interpretation
Option
Substitute 0 -
treat as detect
All Spread -
treat as detect
Substitute 15 -
treat as detect
< Quantification
(Spread only) -
Identify if "Less Than"
< Detection -
Identify if "Less Than"
< Quantification -
Both Methods -
Identify if "Less Than"
% Detected
or
Quantifiable
100%
100%
100%
89%
94%
0%
Outcome of Distributional Tests
Test for
Normality
Normality is Not
Rejected
Data Not Normal
Data Not Normal
Data Not Normal
Data Not Normal
-
Test for
Lognormality
-
-
Data Not Lognormal
Data Not Lognormal
Lognormality is Not
Rejected
-
Test for Gamma
Distribution
-
-
Data Not Gamma
Distributed
Data Not Gamma
Distributed
Data Not Gamma
Distributed
-
Test for Normality: Shapiro-Wilk test; Normal ROS estimates for non-quantifiable outcomes
Test for Lognormality: Shapiro-Wilk test on log-data; Lognormal regression on order statistics (ROS) estimates for
non-quantifiable outcomes
Test for Gamma: Kolmogorow-Smirnov test; Gamma ROS estimates for non-quantifiable outcomes
All tests were performed at the 0.05 significance level.
Table B-2. Results of Distributional Goodness-of-Fit Tests Applied to the Surface Sample Data
(n=136 samples) for Each Data Interpretation Option
Data Interpretation
Option
Substitute 0 -
treat as detect
All Spread -
treat as detect
Substitute 15 -
treat as detect
< Quantification
(Spread only) -
Identify if "Less Than"
< Detection -
Identify if "Less Than"
< Quantification -
Both Methods-
Identify if "Less Than"
% Detected
or
Quantifiable
100%
100%
100%
45%
51%
4%
Outcome of Distributional Tests
Test for
Normality
Data Not Normal
Data Not Normal
Data Not Normal
Data Not Normal
Data Not Normal
Data Not Normal
Test for
Lognormality
~
~
Data Not Lognormal
Lognormality is Not
Rejected
Lognormality is Not
Rejected
Data Not Lognormal
Test for Gamma
Distribution
~
~
Data Not Gamma
Distributed
Data Not Gamma
Distributed
Data Not Gamma
Distributed
Data Not Gamma
Distributed
Test for Normality: Lilliefors test; Normal regression on order statistics (ROS) estimates for non-quantifiable
outcomes
Test for Lognormality: Lilliefors test; Lognormal ROS estimates for non-quantifiable outcomes
Gamma: Kolmogorow-Smirnov test; Gamma ROS estimates for non-quantifiable outcomes
All tests were performed at the 0.05 significance level.
58
-------
References for Appendix B
D'Agostino, R.B. and MA. Stephens, 1986. Goodness-of-Fit Techniques. Marcel Dekker, Inc.
Lilliefors, H., 1967. On the Kolmogorov-Smirnov test for normality with mean and variance unknown.
Journal of the American Statistical Association. 62(318):399-402.
Shapiro, S.S. and M.B. Wilk, 1965. An analysis of variance test for normality (complete samples).
Biometrika 52(3-4):591-611.
59
-------
Appendix C
Estimates for 95% Upper Confidence Limit (UCL) on the Mean, Applying Various Statistical
Methods
for Each Data Interpretation Option
60
-------
Table C-la. Estimates for 95% Upper Confidence Limit (UCL) on the Mean, Applying Various
Statistical Methods That Rely on a Specific Distributional Form, for Air Sample
Data (CFU/m3)
95% UCL
Calculation Method
Data Interpretation O
Substitute 0 -
treat as
detect
All Spread -
treat as detect
Substitute 15 -
treat as detect
)tions
< Quantification
(Spread only) -
Identify if "Less
Than"
< Detection -
Identify if
"Less Than"
Methods Assuming a Normal Distribution
Student-t3
136
446
8,540
137
447
Methods Assuming a Positively Skewed Distribution with 100% Quantifiable Outcomes
Adj. Central Limit
Theorem
Modified-t
138
136
513
458
8,584
8,549
Methods Assuming a Lognormal Distribution
Chebyshev (MVUE)
H-Statistica
8,844
158
549
Methods Using Lognormal ROS Extrapolation for Non-Quantifiable Results
Student-t
Percentile Bootstrap
BCA Bootstrap
H-UCL
139
138
141
149
448
452
564
515
Methods Assuming a Gamma Distribution
Adjusted Gamma.
8,561
147
641
BCA, bias-corrected accelerated; CPU, colony forming units
a The Student-t (under the Normal Distribution assumption) and H-Statistic approaches assume that left-censored
(non-quantifiable) observations are substituted by one-half of the detection or quantification limit.
Table C-lb. Estimates for 95% Upper Confidence Limit (UCL) on the Mean, Applying Various
Nonparametric Statistical Methods, for Air Sample Data (CFU/m3)
95% UCL
Calculation Method
Central Limit Theorem
Student-t
Normal-Z
Jackknife
Standard Bootstrap
Hall's Bootstrap
Bootstrap t
BCA
Percentile bootstrap
Chebyshev
Data Interpretation Options3
Substitute 0 -
treat as
detect
134
136
133
142
141
138
133
184
All Spread -
treat as detect
437
446
432
1,086
713
525
446
699
Substitute 15 -
treat as detect
8,528
8,540
8,526
8,752
8,626
8,574
8,536
8,868
< Quantification
(Spread only) -
Identify if "Less
Than"
139
137
139
146
136
138
183
< Detection -
Identify if
"Less Than"
448
439
448
704
469
450
700
BCA, bias-corrected accelerated; CPU, colony forming units
a The options represented by the last two columns of the table, which lead to left-censored data, utilize a Kaplan-
Meier approach to estimating the underlying distribution of the data.
61
-------
Table C-2a. Estimates for 95% UCL on the Mean, Applying Various Statistical Methods That
Rely on a Specific Distributional Form, for Surface Sample Data (CFU/m2)
95% UCL
Calculation Method
Data Interpretation Options
Substitute 0
- treat as
detect
All Spread -
treat as
detect
Substitute 15
- treat as
detect
< Quantification
(Spread only) -
Identify if "Less
Than"
< Detection
- Identify if
"Less Than"
< Quantification
- Both Methods
- Identify if
"Less Than"
Methods Assuming a Normal Distribution
Student-ta
864
2,105
47,708
918
2,157
3,237
Methods Assuming a Positively Skewed Distribution with 100% Quantifiable Outcomes
Adj. Central Limit
Theorem
Modified-t
1,101
902
2,532
2,173
49,405
47,984
Methods Assuming a Lognormal Distribution
Chebyshev (MVUE)
H-Statistica
51,807
41,905
584
1,381
2,954
Methods Using Lognormal ROS Extrapolation for Non-Quantifiable Results
Student-t
Percentile Bootstrap
BCA Bootstrap
H-UCL
873
947
1,262
1,380
2,116
2,210
2,646
4,083
238
240
262
192
Methods Assuming a Gamma Distribution
Adjusted Gamma.
45,117
877
2,152
179
BCA, bias-corrected accelerated; CPU, colony forming units
a The Student-t (under the Normal Distribution assumption) and H-Statistic approaches assume that left-censored
(non-quantifiable) observations are substituted by one-half of the detection or quantification limit.
Table C-2b. Estimates for 95% Upper Confidence Limit (UCL) on the Mean, Applying Various
Nonparametric Statistical Methods, for Surface Sample Data (CFU/m2)
95% UCL
Calculation Method
Central Limit Theorem
Student-t
Normal-Z
Jackknife
Standard Bootstrap
Hall's Bootstrap
Bootstrap t
BCA
Percentile bootstrap
Chebyshev
Data Interpretation Options3
Substitute 0
- treat as
detect
862
864
867
2,209
2,170
1,381
932
1,516
All Spread
- treat as
detect
2,099
2,105
2,082
6,041
6,078
2,674
2,218
3,613
Substitute 15
- treat as
detect
47,633
47,708
47,296
49,029
49,556
50,003
48,292
65,533
< Quantification
(Spread only) -
Identify if "Less
Than"
879
876
877
2,143
975
941
1,533
< Detection
- Identify if
"Less Than"
2,125
2,119
2,122
6,067
2,282
2,244
3,638
< Quantification -
Both Methods -
Identify if "Less
Than"
826
825
1,396
818
2,344
2,326
944
BCA, bias-corrected accelerated; CPU, colony forming units
a The options represented by the last three columns of the table, which lead to left-censored data, utilize a Kaplan-
Meier approach to estimating the underlying distribution of the data.
62
-------
Table C-3 is taken from Table 2-5 of EPA (2010). It indicates those approaches for estimating the 95%
UCL that ProUCL recommends when all results are considered positive and detected, and no specific
distributional form is assumed for these results other than the presence of skewness. The
recommendations are based on the value of the standard deviation of the log-transformed results and the
sample size (n).
Table C-3. Summary of ProUCL Recommended Approaches for Calculating the 95% Upper
Confidence Limit (UCL) on an Unknown Mean When All Results are Positive and Detected and
Taken from a Skewed Dataset Without a Discernable Distribution
Standard Deviation of
Log-Transformed
Results
Less than or equal to 0.5
Between 0.5 and 1.5
Between 1.5 and 2.0
Between 2.0 and 2. 5
Between 2. 5 and 3.0
Between 3.0 and 3.5
Greater than 3. 5
Sample Size (n)
All sample sizes
All sample sizes
Less than 20
Equal or greater than 20
Less than 10
Between 10 and 20
Between 20 and 50
Equal or greater than 50
Less than 10
Between 10 and 30
Between 30 and 70
Equal or greater than 70
Less than 15
Between 15 and 50
Between 50 and 100
Equal or greater than 100
All sample sizes
ProUCL Recommended Approach
Student-t, modified-t, or H-UCL
95% Chebyshev (mean, SD) UCL
99% Chebyshev (mean, SD) UCL
95% Chebyshev (mean, SD) UCL
Hall's Bootstrap UCL
99% Chebyshev (mean, SD) UCL
97.5% Chebyshev (mean, SD) UCL
95% Chebyshev (mean, SD) UCL
Hall's Bootstrap UCL
99% Chebyshev (mean, SD) UCL
97.5% Chebyshev (mean, SD) UCL
95% Chebyshev (mean, SD) UCL
Hall's Bootstrap UCL
99% Chebyshev (mean, SD) UCL
97.5% Chebyshev (mean, SD) UCL
95% Chebyshev (mean, SD) UCL
99% Chebyshev (mean, SD) UCL
Source: Table 2-5 of EPA (2010).
When censored data are present and no discernable distribution is assumed for the results, ProUCL
indicates that the UCL computational method recommended for a normal distribution could be used if the
distribution of observed results resembles a symmetric distribution, and for a lognormal or gamma
distribution if the distribution of observed results is skewed. Sections 4.10.3 and 4.10.4 of EPA (2010)
provide the ProUCL-recommended approaches for the gamma and lognormal distribution, respectively.
63
-------
United States
Environmental Protection
Agency
PRESORTED STANDARD
POSTAGE & FEES PAID
EPA
PERMIT NO. G-35
Office of Research and Development (8101R)
Washington, DC 20460
Official Business
Penalty for Private Use
$300
------- |