vvEPA
United States
Environmental Protection
Agency
Assessment of False Positive and False
Negative Confirmation Rates for Fecal
Coliforms by EPA Methods 1680 and
1681 in Sewage Sludge Matrices
September 2008
-------
U.S. Environmental Protection Agency
Office of Water (4303T)
1200 Pennsylvania Avenue, NW
Washington, DC 20460
EPA821-R-08-005
-------
Acknowledgments
The contributions of the following volunteer participants and organizations to this study are gratefully
acknowledged:
Volunteer Participant Laboratories
• Albuquerque Water Utility Authority - Water Quality Laboratory: Steve Glass, Bill Lindberg, and
Lauren Tapps
• Alexandria Sanitation Authority: John Benard, Leulu M. Gebremedhin, Lisa Racey, Paul Carbary,
and George Floyd
• Amtest Laboratories: Kathy Fugiel, Neila Glidden, and Melinda Woomer
• Analytical Laboratories, Incorporated: Sandy Koch, Robert Voermans, and Lynn Murray
• Barton Laboratory, Jefferson County Commission: J. Lynn King, Henry Word, Ronstead Claughton,
Don Lovell, and Bob Spigner
• Bay County Laboratory Services Division: Carol Monti and Anna Wright
• City of Everett Water Pollution Control Facility: Jeff Wright and Tim Rickman
• County Sanitation Districts of Los Angeles County, Joint Water Pollution Control Plant - Water
Quality Laboratory (JWPCP-WQL): Kathy Walker, Debra Leachman, and Mark Patterson
• Edge Analytical: Larry Henderson, Kent Oostra, and Shannon Kizer
• Energy Laboratories, Incorporated - Casper Branch: Sheryl Garling, N. Lou Miller, Sherri L.
Boatman, and Randy Ogden
• Environmental Protection Agency, National Risk Management Research Laboratory: Mark Meckes,
Laura Boczek, and Cliff Johnson
• Environmental Science Corporation: Rodney Shinbaum, Kim Johnson, and Rachel Freeman
• The Industrial Laboratories Company, Incorporated: Tania Vogel, Geoff Henderson, and Lenka
Teodorovic
• Katahdin Analytical Services, Incorporated: Deborah Nadeau, Shelly Brown, Greg Lull, Amy
Broadbent, Jane April, and Mandi Greenleaf
• King County Environmental Laboratory: Kate Leone, Despina Strong, Colin Elliot, Joe Calk, Tami
Alley, Bobbie Anderson, Karl Bruun, Eyob Mazengia, Robin Revelle, Debbie Turner, and Jodeen
Wieser
• Madison Metropolitan Sewerage District: Kurt Knuth, Montgomery Baker, Kris Farrar, and Carol
Mielke
• Monroe County Environmental Laboratory at the Frank E. VanLare Waste Treatment Plant: Drew
Smith, Mary Merner, Dave Spanganberg, and Stephen Bland
• Nova Biologicals, Incorporated: Paul Pearce, Brenda Bates, Donna Reioux, and Amber Sutton
• Orange County Utilities Central Laboratory: Terri Slifko, Shelley Patterson, Vanessa Perez, Scott
Rampenthal, and Theresa Slifko
• St. George Regional Water Reclamation Facility Laboratory: Leslie Wentland and Amy Howe
-------
• SVL Analytical, Incorporated - Microbiology Laboratory: Linda Johann
• Universal Laboratories: Carol Kleemeier, Stacie Splinter, and Linda McFarland
• Wichita Water and Sewer Wastewater Laboratory: Becky Gagnon and Karen Roberts
Volunteer Verification Laboratory
• Orange County Sanitation Districts, Environmental Sciences Laboratory: Charles McGee and Kim
Patton
-------
Disclaimer
Mention of company names, trade names, or commercial products does not constitute endorsement or
recommendation for use.
Questions concerning this report should be addressed to:
Robin K. Oshiro
Engineering and Analysis Division (4303T)
U.S. EPA Office of Water, Office of Science and Technology
1200 Pennsylvania Avenue, NW
Washington, DC 20460
oshiro.robin(g),epa.gov or OSTCWAMethods@ epa.gov
in
-------
Table of Contents
SECTION 1.0 INTRODUCTION 3
1.1 Summary of EPA Method 1680 for Fecal Conforms 3
1.2 Summary of EPA Method 1681 for Fecal Conforms 3
SECTION 2.0 STUDY OBJECTIVES AND STUDY DESIGN 4
2.1 Study Objectives 4
2.2 Technical Approach: Identification of Laboratories 4
2.3 Technical Approach: Sample Collection, Storage Conditions, and Holding
Times 4
2.4 Technical Approach: Methods 5
2.5 Technical Approach: Study Design and Analyses 5
2.6 Quality Control (QC) Analyses 6
SECTION 3.0 STUDY IMPLEMENTATION 7
3.1 Study Management 7
3.2 Study Schedule 7
3.3 Participant Laboratories 7
SECTION 4.0 DATA REPORTING AND VALIDATION 9
4.1 Data Reporting 9
4.2 Data Validation 9
SECTION 5.0 RESULTS 10
5.1 Method 1680 Class A Sewage Sludge Results 10
5.2 Method 1680 Class B Sewage Sludge Results 10
5.3 Method 1681 Class A Sewage Sludge Results 11
5.4 Method 1681 Class B Sewage Sludge Results 12
5.5 Alkaline-Stabilized Results 12
5.6 Overall Results by Matrix and Method 13
5.7 Overall Results by Method 13
SECTION 6.0 ASSESSMENT OF FALSE POSITIVE AND FALSE NEGATIVE CONFIRMATION RATES:
DISCUSSION 15
6.1 Method 1680 Confirmation Rates for Class A Matrices 15
6.2 Method 1680 Confirmation Rates for Class B Matrices 15
6.3 Method 1681 Confirmation Rates for Class A Matrices 16
6.4 Method 1681 Confirmation Rates for Class B Matrices 16
6.5 Methods 1680 and 1681 Confirmation Rates for Alkaline-Stabilized Matrices
16
SECTION 7.0 CONCLUSIONS 18
SECTION 8.0 REFERENCES 19
SECTION 9.0 ACRONYMS 20
IV
-------
List of Tables
Table 1. Study Analysis Summary 5
Table 2. Positive and Negative Control Cultures 6
Table 3. Laboratories Participating in the Assessment of Methods 1680 and 1681 False Positive and
False Negative Confirmation Rates 8
Table 4. Laboratory-Specific False Positive and False Negative Rates for Method 1680 Class A
Matrices 10
Table 5. Laboratory-Specific False Positive and False Negative Rates for Method 1680 Class B
Matrices 11
Table 6. Laboratory-Specific False Positive and False Negative Rates for Method 1681 Class A
Matrices 11
Table 7. Laboratory-Specific False Positive and False Negative Rates for Method 1681 Class B
Matrices 12
Table 8. Laboratory-Specific False Positive and False Negative Rates for Alkaline-Stabilized
Matrices 12
Table 9. Overall False Positive and False Negative Rates for Methods 1680 and 1681 by Matrix 13
Table 10. Overall False Positive and False Negative Rates for Methods 1680 and 1681 by Method 14
-------
List of Appendices
Appendix A: Verification Instructions
VI
-------
False Positive and False Negative Assessment Report
Executive Summary
In 2004, EPA completed the report for the interlaboratory validation of EPA Methods 1680 (LTB/EC)
and 1681 (A-l) for fecal coliforms in sewage sludge (biosolids) (EPA 821-B-04-007). In 2005 (FR
70:48256) EPA proposed these methods for use in sewage sludge. As a part of the proposal, EPA also
published the results of a three laboratory interlaboratory holding time study using these methods to
determine whether sewage sludge samples could be held for 24 hours without significant change in
bacterial densities for these analytes (Reference 8.1, EPA-821-R-04-029). Between the proposal and the
final rule, EPA conducted a more extensive (23 laboratory) holding time study to confirm the results from
the preliminary study in a wider variety of sewage sludge matrices (2006, Reference 8.2, EPA-821-R-07-
003). The methods and the 24 hour holding time for certain media and sewage sludge treatment type
combinations were promulgated in 2007 (FR 72:14220).
In addition to the holding time study, culture data for the false positive and false negative confirmation
rates for these two media were collected during the 2006 study but not statistically analyzed at that time.
This report presents the statistical assessment of false positive and false negative confirmation rates for
Methods 1680 and 1681 in sewage sludge samples analyzed during the U.S. Environmental Protection
Agency's (EPA's) second sewage sludge holding time study (2006, Reference 8.2). The purpose of this
statistical assessment is to characterize EPA Methods 1680 and 1681 false positive and false negative
confirmation rates across multiple laboratories and sewage sludge matrices analyzed during the sewage
sludge holding time study.
EPA qualified 23 participating laboratories to study several matrix/method combinations. The
laboratories streaked growth from tubes for isolation onto mEndo plates, then shipped the plates to the
verification laboratory for confirmation between February 2006 and August 2006. Nine hundred ninety
six (767 typical and 229 atypical) colonies were verified as target or non-target colonies. Usable data was
obtained from 21 of the 23 laboratories.
Overall false positive rates for Method 1680 ranged from 0% - 17% for all six matrices (aerobically
digested, alkaline-stabilized, anaerobically digested, composted, heat-dried, and thermophilically
digested). Overall false positive rates for Method 1681 were relatively low for three of the matrices
(aerobically digested, alkaline-stabilized, and anaerobically digested) ranging from 3% - 9%. The false
positive rate for composted matrices was considerably higher, at 37%, likely due to tubes that appeared
turbid (not due to growth) that also produced air bubbles (not gas) when agitated, which thus appeared
positive.
Overall false negative rates were very low for Method 1680, ranging from 0% - 5% for all six matrices.
In addition, overall false negative rates for Method 1681 were also very low, ranging from 0% - 3% for
aerobically digested, alkaline-stabilized, anaerobically digested, and composted matrices. Although the
false negative rates were adjusted to include the number of "true" negatives [clear tubes (not submitted
for confirmation)], a significant number of negative tubes (turbidity and no gas) submitted to verification
verified as false negatives (82/152). This indicates that laboratories may be incorrectly reading the tubes.
In some instances gas production may be very weak (especially as sample volume analyzed is decreased)
and may be recorded incorrectly as a negative.
Based on the results of this statistical assessment, overall the false positive and negative rates were
acceptably low. The high false positive and negative rates observed for some method/matrix
combinations may be due to lack of familiarity with the matrices rather than a limitation of the analytical
methods. To ensure that analysts are familiar with the different matrices evaluated for fecal coliforms
-------
False Positive and False Negative Assessment Report
using Method 1680 and/or Method 1681, we recommend that analysts verify a minimum often percent of
all positive and negative tubes for each matrix evaluated to increase proficiency.
-------
False Positive and False Negative Assessment Report
SECTION 1.0 INTRODUCTION
Land application of sewage sludge is a critical component of solid waste management. Under Subpart D
of Title 40 Code of Federal Regulations (CFR) Part 503, sewage sludges (biosolids) are required to be
processed prior to land application in order to minimize pathogen levels and the potential public health
risks associated with contact or exposure. Subpart D further defines and classifies sewage sludge
(biosolids) for land application purposes based on pathogen concentrations. In 2004, EPA validated
Methods 1680 (Reference 8.3) and 1681 (Reference 8.4) for fecal coliforms in Class A and B sewage
sludge. As a part of the proposal to approve these methods for use in sewage sludge, EPA also published
the results of a three laboratory interlaboratory holding time study using these methods to determine
whether sewage sludge samples could be held for 24 hours without significant change in bacterial
densities for these analytes (Reference 8.1, EPA-821-R-04-029). False positive or false negative rates
were not collected for these media in this initial study. Between the proposal and the final ruling, EPA
conducted a more extensive (23 laboratory) holding time study to confirm the results from the preliminary
study in a wider variety of sewage sludge matrices and to collect data to determine false positive and false
negative rates for fecal coliforms (2006, Reference 8.2, EPA-821-R-07-003). The methods and the
extended holding time for certain media and sewage sludge treatment type combinations were
promulgated in 2007 (FR 72:14220).. The objective of this statistical assessment report was to assess
false positive and false negative confirmation rates for Methods 1680 and 1681 as they were not
characterized via biochemical verification during the validation study.
1.1 Summary of EPA Method 1680 for Fecal Coliforms
Fecal coliforms were evaluated in Class A and Class B sewage sludge using Method 1680 (Reference
8.3). Sewage sludge samples were homogenized and inoculated into lauryl tryptose broth (LTB) a
presumptive medium. Following incubation at 35.0°C ± 0.5°C for 24 ± 2 and 48 ± 3 hours, growth from
positive tubes was transferred to EC broth (confirmatory medium) and incubated at 44.5°C ± 0.2°C for 24
± 2 hours. All tubes exhibiting turbidity and gas production were considered positive for fecal coliforms.
All tubes that did not exhibit both turbidity and gas production were considered negative for fecal
coliforms.
1.2 Summary of EPA Method 1681 for Fecal Coliforms
Fecal coliforms were evaluated in Class A and Class B sewage sludge using Method 1681 (Reference
8.4). Sewage sludge samples were homogenized and inoculated into A-l medium and incubated at
35.0°C ± 0.5°C for 3 hours and 44.5°C ± 0.2°C for 21 ± 2 hours. All tubes exhibiting turbidity and gas
production were considered positive for fecal coliforms. All tubes that did not exhibit both turbidity and
gas production were considered negative for fecal coliforms.
-------
False Positive and False Negative Assessment Report
SECTION 2.0 STUDY OBJECTIVES AND STUDY DESIGN
2.1 Study Objectives
The following study objective was established for the biochemical verification study:
• Characterize EPA Methods 1680 and 1681 false positive and false negative confirmation rates across
multiple laboratories and sewage sludge matrices.
To accomplish this objective, qualified volunteer laboratories submitted mEndo plates to verification that
were streaked from positive and negative EC and/or A-l tubes that were analyzed during the holding time
study.
The following data quality objective was established for this study:
• Data produced under this study must be generated according to the analytical and QA/QC procedures
as described in each of the analytical methods or approved changes to these procedures in order to
ensure that data will be of known and reliable quality.
2.2 Technical Approach: Identification of Laboratories
The study required two types of laboratories: participant laboratories and a verification laboratory. The
role of these laboratories is described below.
2.2.1 Participant Laboratories
Participant laboratories were representative of the general user community, with experience analyzing
sewage sludge samples for fecal coliforms and had access to representative sewage sludge matrices within
driving distance (2 hours).
To reduce cost, volunteer laboratories were recruited from a pool of 400 laboratories contacted for
potential participation in the study. To reduce the burden on the participant laboratories and encourage
volunteer participants, EPA provided the media, reagents, and disposable supplies required for the study.
EPA also provided a contractor [Computer Services Corp (CSC)] to manage the study.
2.2.2 Verification Laboratory
The Orange County Sanitation Districts, Environmental Sciences Laboratory in Fountain Valley, CA
served as the verification laboratory. To assess false positive and negative rates, the verification
laboratory speciated typical and atypical colonies on mEndo plates submitted by the participant
laboratories. Colonies were identified using the Vitek® automated biochemical identification system
with bioMerieux's industrial/environmental database. Verification laboratory instructions are in
Appendix A.
The verification laboratory was also recruited as a volunteer in an effort to reduce costs. To reduce the
burden on the verification laboratory, EPA provided all necessary verification media, reagents, and
supplies.
2.3 Technical Approach: Sample Collection, Storage Conditions, and Holding Times
-------
False Positive and False Negative Assessment Report
A single bulk sample of at least 1000 g was collected and transported to the laboratory on ice and
maintained at less than 10°C and above freezing. At the laboratory, the bulk sample was split into
replicates, spiked (if necessary), and stored in the refrigerator at less than 10°C and above freezing. After
6 and 24 hours from sample collection, unspiked replicates were analyzed by the appropriate procedure
(Table 1) and submitted for verification.
2.4 Technical Approach: Methods
The false positive and false negative confirmation rates were assessed for both Class A and Class B
unspiked matrices analyzed during the sewage sludge holding time study using Methods 1680 and 1681.
2.5 Technical Approach: Study Design and Analyses
For the verification of false positive and false negative confirmation rates, Methods 1680 and 1681 were
used to analyze unspiked samples at multiple laboratories. Table 1 summarizes the method/matrix
combinations that were evaluated during the holding time study and submitted to verification to meet the
objectives listed in Section 2.1.
Table 1. Study Analysis Summary
Matrix
Aerobically digested
Alkaline-stabilized
Anaerobically digested
Compost
Heat-Dried
Thermophilically digested
Method
1680/1681
1680/1681
1680/1681
1680/1681
1680
1680
No. of Tubes Submitted to Verification3
Positive
25
25
25
25
25
25
Negative
25
25
25
25
25
25
a Laboratories were instructed to submit 25 positive (turbidity and gas) and 25 negative (turbidity and no gas) tubes
to the verification laboratory, if available
2.5.1 Assessment of Methods 1680 and 1681 False Positive and False Negative
Confirmation Rates
Methods 1680 and 1681 false positive and false negative rates were assessed during the 2006 (Reference
8.2, EPA-821-R-07-003) holding time study by submitting tubes with growth from unspiked samples to
biochemical verification according to study instructions. For each matrix and method, the participant
laboratories were requested to submit growth from 25 positive (turbidity and gas) tubes and 25 negative
(turbidity and no gas) tubes. No attempt was made to verify the contents of those tubes which did not
exhibit visual evidence of growth (turbidity). All tubes (positive and negative) with growth were
submitted to the following verification procedure:
• For each positive and negative tube that was verified, growth from tubes were streaked for isolation
onto mEndo plates and incubated at 35.0°C ± 0.5°C for 24 ± 2 hours.
• Participant laboratories prepared mEndo plates for shipment to the verification laboratory by
wrapping the edges of the plates with parafilm and then wrapping the stack of plates associated with
each sample with bubble wrap. Wrapped plates were placed into a cooler lined with a trash bag and
-------
False Positive and False Negative Assessment Report
surrounded by blue ice. The cooler was sealed with shipping tape and shipped to the verification
laboratory using pre-addressed FedEx shipping labels.
• At the verification laboratory, the following number of isolates were verified using the Vitek®
automated identification system:
o Positive tube verification: Up to five isolated "typical" colonies from each mEndo plate (red with
characteristic metallic sheen) were submitted to biochemical verification. If any one of the five
colonies verified as a positive fecal coliform result, the tube was categorized as a true positive. If
none of the five colonies verified as a fecal coliform, the tube was categorized as a false positive,
and further colonies were not selected for verification due to cost constraints.
o Negative tube verification: Five isolated "atypical" colonies from each mEndo plate (colonies not
having typical color and morphology) were submitted to biochemical verification. If there were
less than five typical colonies on each mEndo plate, "atypical" colonies were also submitted to
ensure that five colonies per plate were submitted to verification. If none of the five colonies
verified as a fecal coliform, the tube was categorized as a true negative. If any of the five colonies
verified as a positive fecal coliform result, the tube was categorized as a false negative.
o No growth/unidentified: If no growth was observed on mEndo, or blood agar, the tube was not
included in the total number submitted to verification. In addition, any isolates that were not
identified by the Vitek® were not included in the total number submitted to verification.
Note: The Vitek® automated identification system is a fully automated system that performs bacterial
identification of isolates using fluorescent technology. After primary isolation, an isolated colony is
prepared at a known optical density in saline and inoculated into the Vitek® system. The gram
negative card contains 41 fluorescent biochemical tests that are read every 15 minutes. Algorithms
are used for organism identification.
2.6 Quality Control (QC) Analyses
Participating laboratories completed the following QC requirements: media sterility checks, dilution water
sterility checks, blender jar sterility checks, method blanks, positive controls, and negative controls.
Table 2 summarizes the positive and negative control cultures used during the study.
Table 2. Positive and Negative Control Cultures
Method
1680/1681
1680
1680/1681
Medium or Test
EC, A-1
LIB
mEndo
Positive Control
Escherichia coli
Escherichia coli
Escherichia coli
Negative Control
Enterobacter aerogenes
Pseudomonas aeruginosa
Enterobacter aerogenes
-------
False Positive and False Negative Assessment Report
SECTION 3.0 STUDY IMPLEMENTATION
3.1 Study Management
This study was designed under the direction of the Office of Science and Technology, Engineering and
Analysis Division within the U.S. Environmental Protection Agency's (EPA's) Office of Water (OW).
The EPA technical lead was Robin K. Oshiro. Coordination of activities for the study was performed by
the CSC Microbiology and Biochemistry Studies Group.
3.2 Study Schedule
Fecal coliform analyses were conducted between February 2006 and August 2006. Prior to analyzing
holding time study samples and submitting to verification, each laboratory was required to analyze four
initial precision and recovery (IPR) samples, one unspiked reference matrix sample, and one matrix spike
(MS) sample to demonstrate method proficiency.
3.3 Participant Laboratories
The 23 laboratories and one verification laboratory that participated in the assessment of false positive
and false negative confirmation rates are provided in Table 3, below. No endorsement of these
laboratories is implied, nor should any be inferred. Participant laboratories were randomly assigned
numbers for purposes of presenting data in this report.
-------
False Positive and False Negative Assessment Report
Table 3. Laboratories Participating in the Assessment of Methods 1680 and 1681 False Positive
and False Negative Confirmation Rates
Albuquerque Water Utility Authority
Water Quality Laboratory
Steve Glass, Bill Lindberg, Lauren Tapps
4201 2nd Street, S.W., Albuquerque, NM 87105-0511
The Industrial Laboratories Company, Incorporated
Tania Vogel, Geoff Henderson, Lenka Teodorovic
4046 Youngfield Street, Wheat Ridge, CO 80033
Alexandria Sanitation Authority
John Benard, Leulu M. Gebremedhin, Lisa Racey,
Paul Carbary, George Floyd
1500 Eisenhower Avenue, Alexandria, VA22314
Katahdin Analytical Services, Incorporated
Deborah Nadeau, Shelly Brown, Greg Lull, Amy Broadbent,
Jane April, Mandi Greenleaf
340 County Road #5, Westbrook, ME 04092
Amtest Laboratories
Kathy Fugiel, Neila Glidden, Melinda Woomer
14603 NE 87th Street, Redmond, WA 98052
King County Environmental Laboratory
Kate Leone, Despina Strong, Colin Elliot, Joe Calk, Tami Alley,
Bobbie Anderson, Karl Bruun, Eyob Mazengia, Robin Revelle,
Debbie Turner, Jodeen Wieser
322 West Ewing Street, Seattle, WA 98119-1507
Analytical Laboratories, Incorporated
Sandy Koch, Robert Voermans, Lynn Murray
1804 North 33rd Street, Boise, ID 83703-5814
Madison Metropolitan Sewerage District
Kurt Knuth, Montgomery Baker, Kris Farrar, Carol Mielke
1610 Moorland Road, Madison, Wl 53713
Barton Laboratory, Jefferson County Commission
J. Lynn King, Henry Word, Ronstead Claughton, Don
Lovell, Bob Spigner
1290 Oak Grove Road, Homewood, AL 35209-6961
Monroe County Environmental Laboratory at the Frank E.
VanLare Waste Treatment Plant
Drew Smith, Mary Merner, Dave Spanganberg, Stephen Bland
1574 Lakeshore Blvd, Admin Bldg #7, Rochester, NY 14617
Bay County Laboratory Services Division
Carol Monti and Anna Wright
3420 Transmitter Road, Panama City, FL 32404
Nova Biologicals, Incorporated
Paul Pearce, Brenda Bates, Donna Reioux, Amber Sutton
1775 East Loop 336, Suite 4, Conroe, TX 77301
City of Everett Water Pollution Control Facility
Jeff Wright and Tim Rickman
4027 4th Street S.E., Everett, WA 98205
Orange County Utilities Central Laboratory
Terri Slifko, Shelley Patterson, Vanessa Perez, Scott
Rampenthal, Theresa Slifko
9124 Curry Ford Road, Orlando, FL 32825
County Sanitation Districts of Los Angeles County
(JWPCP - WQL)
Kathy Walker, Debra Leachman, Mark Patterson
24501 S. Figueroa Street, Carson, CA 90745
St. George Regional Water Reclamation Facility Laboratory
Leslie Wentland and Amy Howe
3780 South 1550 West, St. George, UT 84790
Edge Analytical
Larry Henderson, Kent Oostra, Shannon Kizer
805 West Orchard Drive, Suite 4, Bellingham, WA 98225
SVL Analytical, Incorporated - Microbiology Laboratory
Linda Johann
2195 Ironwood Court, Suite C, Coeur d'Alene, ID 83814
Energy Laboratories, Incorporated - Casper Branch
Sheryl Garling, N. Lou Miller, Sherri L. Boatman, Randy
Ogden
2393 Salt Creek Highway, Casper, WY 82601
Universal Laboratories
Carol Kleemeier, Stacie Splinter, Linda McFarland
20 Research Drive, Hampton, VA 23666
Environmental Protection Agency, National Risk
Management Research Laboratory
Mark Meckes, Laura Boczek, Cliff Johnson
26 West Martin Luther King Drive, Cincinnati, OH 45268
Wichita Water and Sewer Wastewater Laboratory
Becky Gagnon and Karen Roberts
2305 East 57th Street South, Wichita, KS 67216
Environmental Science Corporation
Rodney Shinbaum, Kim Johnson, Rachel Freeman
12065 Lebanon Road, Mt. Juliet, TN 37122
Verification laboratory:
Orange County Sanitation District, Environmental
Sciences Laboratory
Charles McGee and Kim Patton
10844 Ellis Avenue, Fountain Valley, CA 92708
-------
False Positive and False Negative Assessment Report
SECTION 4.0 DATA REPORTING AND VALIDATION
4.1 Data Reporting
Laboratories submitted the following data to CSC for review and validation:
• Completed cover sheet with sample collection and QC information
• Completed sample-specific data reporting forms
• Documentation of any additional information that would assist in evaluating the data
4.2 Data Validation
Data review checklists were used to ensure that each data package was complete and to ensure that each
sample result met the study-specific and method-specific requirements. Items reviewed for each sample
included the following:
• Confirmation that original forms were submitted
• Confirmation that all QC checks were performed and exhibited the appropriate response
• Confirmation that all method-specific incubation times and temperatures were met
• Confirmation that all media and reagents were used within expiration dates
• Confirmation that all calculations were correct
This process was performed independently by two data reviewers, each of whom entered the results into
separate spreadsheets designed for data review and validation for this study. The results were compared
to verify consistency and identify potential data entry errors.
The following issues were encountered during laboratory analyses:
Laboratory 3 (Methods 1680 and 1681, Class A, composted and Class B, aerobically digested sewage
sludge): Eighteen of the isolates submitted were from spiked samples. As a result data were considered
invalid and not included in subsequent data analyses.
Laboratory 14 (Method 1680, Class B, anaerobically digested sewage sludge): One hundred of the
isolates submitted were from spiked samples. As a result data were considered invalid and not included
in subsequent data analyses.
Laboratory 20 (Method 1680, Class B, anaerobically digested sewage sludge): Three of the isolates
submitted were from spiked samples. As a result data were considered invalid and not included in
subsequent data analyses.
Laboratory 25 (Method 1681, Class A, anaerobically digested sewage sludge): All mEndo plates
submitted to verification were not sealed prior to shipment; during transit plates opened resulting in
contamination. None of the plates were submitted to verification.
Laboratory 29 (Method 1681, Class A, composted sewage sludge): Only spiked samples were submitted
to verification. As a result, data were considered invalid and not included in subsequent data analyses.
Although growth from 996 tubes (positive and negative) were submitted by the participating laboratories,
only 731 were actually verified due to issues noted above, and/or because no growth was observed on
submitted mEndo plates.
-------
False Positive and False Negative Assessment Report
SECTION 5.0 RESULTS
This section includes verification results of unspiked sewage sludge samples which were analyzed for
fecal coliforms using Methods 1680 and 1681 and submitted to verification. Only valid results are
included in this section; a detailed description of data invalidation information is included in Section 4.0.
The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted
for confirmation)] because it can be assumed that all clear tubes did not include the target organism.
However, the false positive rate could not be adjusted in a similar manner, because no assumption can be
made about whether the target organism was in tubes that appeared to be positive.
5.1 Method 1680 Class A Sewage Sludge Results
False positive and false negative confirmation rates for Class A matrices analyzed for fecal coliforms
using Method 1680 are summarized in Table 4.
Table 4. Laboratory-Specific False Positive and False Negative Rates for Method 1680 Class A
Matrices
Lab
3
15
7
Matrix
(digested
sewage
sludge)
Compost
Heat-Dried
False Positive Assessment
Typicals
Submitted
8
25
Number of
False
Positives
0
0
False
Positive
Rate (%)
0
0
False Negative Assessment
%
Atypicals
Submitted3
0/221
11/320
0/80
Number of
False
Negatives
0
2
0
False
Negative
Rate (%)b
0
0.6
0
a The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted for
confirmation)]
b The false negative confirmation rate = (false negatives)/(total number of atypicals); e.g., 2/320 = 0.63%
5.2 Method 1680 Class B Sewage Sludge Results
False positive and false negative confirmation rates for Class B matrices analyzed for fecal coliforms
using Method 1680 are summarized in Table 5.
10
-------
False Positive and False Negative Assessment Report
Table 5. Laboratory-Specific False Positive and False Negative Rates for Method 1680 Class B
Matrices
Lab
3
12
24
27
1
9
14
18
20
28
22
Matrix
Aerobic
Anaerobic
Thermophilic
False Positive Assessment
Typicals
Submitted
7
24
24
22
21
25
49
24
15
21
50
Number of
False
Positives
3
5
4
1
0
3
6
4
1
0
0
False
Positive
Rate (%)
42.9
21
16.7
4.6
0
12
12
17
6.7
0
0
False Negative Assessment
%
Atypicals
Submitted3
0/233
25/81
0/149
12/238
0/172
6/212
0/131
7/117
12/179
15/118
0/175
Number of
False
Negatives
0
22
0
10
0
6
0
6
12
0
0
False
Negative
Rate (%)b
0
27.2
0
4.2
0
2.8
0
5.1
6.7
0
0
a The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted for
confirmation)]
b The false negative confirmation rate = (false negatives)/(total number of atypicals); e.g., 22/81 = 27.16%
5.3 Method 1681 Class A Sewage Sludge Results
False positive and false negative confirmation rates for Class A matrices analyzed for fecal coliforms
using Method 1681 are summarized in Table 6.
Table 6. Laboratory-Specific False Positive and False Negative Rates for Method 1681 Class A
Matrices
Lab
3
13
23
Matrix
Compost
False Positive Assessment
Typicals
Submitted
8
25
10
Number of
False
Positives
0
6
10
False
Positive
Rate (%)
0
24
100
False Negative Assessment
%
Atypicals
Submitted3
0/233
4/152
0/58
Number of
False
Negatives
0
2
0
False
Negative
Rate (%)b
0
1.3
0
a The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted for
confirmation)]
b The false negative confirmation rate = (false negatives)/(total number of atypicals); e.g., 2/152 = 1.32%
11
-------
False Positive and False Negative Assessment Report
5.4 Method 1681 Class B Sewage Sludge Results
False positive and false negative confirmation rates for Class B matrices analyzed for fecal coliforms
using Method 1681 are summarized in Table 7.
Table 7. Laboratory-Specific False Positive and False Negative Rates for Method 1681 Class B
Matrices
Lab
3
8
1
5
11
19
26
Matrix
(digested
sewage
sludge)
Aerobic
Anaerobic
False Positive Assessment
Typicals
Submitted
8
25
22
25
24
23
24
Number of
False
Positives
1
0
2
1
3
0
0
False
Positive
Rate (%)
12.5
0
9.1
4
12.5
0
0
False Negative Assessment
%
Atypicals
Submitted3
0/239
0/144
5/191
11/103
20/184
5/183
14/53
Number of
False
Negatives
0
0
0
10
2
3
4
False
Negative
Rate (%)b
0
0
0
9.7
1.1
1.6
7.6
a The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted for
confirmation)]
b The false negative confirmation rate = (false negatives)/(total number of atypicals); e.g., 10/103 = 9.71%
5.5 Alkaline-Stabilized Results
False positive and false negative confirmation rates for alkaline-stabilized matrices analyzed for fecal
coliforms using both Methods 1680 and 1681 are summarized in Table 8. Due to the variability of
treatment within alkaline-stabilized matrices, results are not categorized by Class (A or B).
Table 8. Laboratory-Specific False Positive and False Negative Rates for Alkaline-Stabilized
Matrices
Lab
2
22
2
Method
1680
1681
False Positive Assessment
Typicals
Submitted
9
50
11
Number of
False
Positives
1
0
1
False
Positive
Rate (%)
11.1
0
9.1
False Negative Assessment
%
Atypicals
Submitted3
3/25
0/23
2/52
Number of
False
Negatives
2
0
1
False
Negative
Rate (%)b
8
0
1.9
a The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted for
confirmation)]
b The false negative confirmation rate = (false negatives)/(total number of atypicals); e.g., 2/25 = 8.00%
12
-------
False Positive and False Negative Assessment Report
5.6 Overall Results by Matrix and Method
Overall false positive and false negative confirmation rates for fecal coliforms using both Methods 1680
and 1681 are summarized in Table 9.
Table 9. Overall False Positive and False Negative Rates for Methods 1680 and 1681 by Matrix
Matrix3
Aerobic
Alkaline-
Stabilized
Anaerobic
Compost
Heat-Dried
Thermophilic
Method
1680
1681
1680
1681
1680
1681
1680
1681
1680
1680
False Positive Assessment
Typicals
Submitted
77
33
59
11
155
118
8
43
25
50
Number of
False
Positives
13
1
1
1
14
6
0
16
0
0
False
Positive
Rate (%)
16.9
3
1.7
9.1
9
5.1
0
37.2
0
0
False Negative Assessment
%
Atypicals
Submitted13
37/701
0/383
3/48
2/52
40/929
55/714
11/541
4/443
0/80
0/175
Number of
False
Negatives
32
0
2
1
24
19
2
2
0
0
False
Negative
Rate (%)c
4.6
0
4.2
1.9
2.6
2.7
0.4
0.5
0
0
a Aerobic, anaerobic and thermophilic matrices refer to specific digester process products
b The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted for
confirmation)]
c The false negative confirmation rate = (false negatives)/(total number of atypicals); e.g., 32/701 = 4.56%
5.7 Overall Results by Method
Overall false positive and false negative confirmation rates for fecal coliforms using both Methods 1680
and 1681 are summarized in Table 10.
13
-------
False Positive and False Negative Assessment Report
Table 10. Overall False Positive and False Negative Rates for Methods 1680 and 1681 by Method
Method
1680
1681
Matrix3
Aerobic
Alkaline-
Stabilized
Anaerobic
Compost
Heat-Dried
Thermophilic
Overall
Aerobic
Alkaline-
Stabilized
Anaerobic
Compost
Overall
False Positive Assessment
Typicals
Submitted
77
59
155
8
25
50
374
33
11
118
43
205
Number of
False
Positives
13
1
14
0
0
0
28
1
1
6
16
24
False
Positive
Rate (%)
16.9
1.7
9.
0
0
0
7.5
3
9.1
5.1
37.2
11.7
False Negative Assessment
%
Atypicals
Submitted13
37/701
3/48
40/929
11/541
0/80
0/175
91/2474
0/383
2/52
55/714
4/443
61/1592
Number of
False
Negatives
32
2
24
2
0
0
60
0
1
19
2
22
False
Negative
Rate (%)c
4.6
4.2
2.5
0.4
0
0
2.4
0
1.9
2.7
0.5
1.4
a Aerobic, anaerobic and thermophilic matrices refer to specific digester process products
b The false negative rate was adjusted based on the number of "true" negatives [clear tubes (not submitted for
confirmation)]
c The false negative confirmation rate = (false negatives)/(total number of atypicals); e.g., 32/701 = 4.56%
14
-------
False Positive and False Negative Assessment Report
SECTION 6.0 ASSESSMENT OF FALSE POSITIVE AND FALSE NEGATIVE
CONFIRMATION RATES: DISCUSSION
Participating laboratories streaked growth from 996 (767 typical and 229 atypical) tubes for isolation on
mEndo plates and submitted to verification. Of the 996 mEndo plates submitted, 731 (579 typical and
152 atypical) were actually verified; 265 plates submitted were not confirmed through biochemical
verification (see discussion in Section 4.2). Since "true" negative (clear/no growth) tubes observed
during the study could not be submitted to confirmation, the false negative confirmation rate was adjusted
to include the number of "true" negatives [clear tubes (not submitted for confirmation)] because it can be
assumed that all clear tubes did not include the target organism. However, the false positive rate could
not be adjusted in a similar manner, because no assumption can be made about whether the target
organism was in tubes that appeared to be positive.
Although the false negative rates were adjusted to include the number of "true" negatives [clear tubes (not
submitted for confirmation)], a significant number of negative tubes (turbidity and no gas) submitted to
verification verified as false negatives (82/152). This indicates that laboratories may be incorrectly
reading the tubes. In some instances, gas production indicating a positive result may be very weak
(especially as sample volume analyzed is decreased) and may be recorded incorrectly as a negative. To
improve future performance, analysts should verify ten percent of positive and negative tubes for each
matrix type at a minimum of once per month to increase proficiency.
Results of the analyses and summary observations for each individual method and matrix are provided
below.
6.1 Method 1680 Confirmation Rates for Class A Matrices
False positive and false negative rates for Class A matrices (Compost and Heat-Dried) using Method
1680 were very low. False positive rates were 0% for both matrices and false negative rates ranged from
0% to less than 1%.
Composted Matrices
• The false positive rate for Laboratory 3 was 0%; the false negative rates for Laboratories 3 and 15
were 0% and less than 1%, respectively. Two laboratories analyzed composted Class A matrices
resulting in two valid data sets.
Heat-Dried Matrix
• The false positive and false negative rate for Laboratory 7 was 0%. One laboratory analyzed a Class
A heat-dried matrix resulting in a single valid data set.
6.2 Method 1680 Confirmation Rates for Class B Matrices
False positive rates for Class B matrices using Method 1680, ranged from 0% - 43%. False negative rates
were relatively low across all matrices for 10 of 11 laboratories, ranging from 0% - 7%. However,
Laboratory 12 had a false negative rate of 27%.
Aerobically Digested Matrices
• False positive rates for Laboratories 3,12, 24, and 27 were 43%, 21%, 17%, and 5%, respectively,
with false negative rates of 0%, 27%, 0%, and 4%, respectively. Although Laboratory 3 observed a
false positive rate of 43% compared to the other laboratories, only seven plates/isolates were
15
-------
False Positive and False Negative Assessment Report
submitted to verification. Therefore the rate occurred based on only three false positives. Four
laboratories analyzed aerobically digested Class B matrices resulting in four valid data sets.
Anaerobically Digested Matrices
• False positive rates for Laboratories 1, 9, 14, 18, 20, and 28 were 0%, 12%, 12%, 17%, 7%, and 0%,
respectively, with false negative rates of 0%, 3%, 0%, 5%, 7%, and 0%, respectively. Six
laboratories analyzed anaerobically digested Class B matrices resulting in six valid data sets.
Thermophilically Digested Matrices
• The false positive and false negative rate for Laboratory 22 was 0%. One laboratory analyzed a
Class B thermophilically digested matrix resulting in a single valid data set.
6.3 Method 1681 Confirmation Rates for Class A Matrices
False positive rates for Class A matrices using Method 1681 varied from a low of 0% to a high of 100%.
In contrast, the false negative rates were consistently low with rates ranging from 0% - 1%.
Composted Matrices
• False positive rates for Laboratories 3,13, and 23 were 0%, 24%, and 100%, respectively, with false
negative rates of 0%, 1%, and 0%, respectively. The high false positive confirmation rates observed
for Laboratories 13 (24%) and 23 (100%) may be due to the composted matrices evaluated.
Materials used in the composting process may cause tubes to appear turbid and produce air bubbles
when agitated. Three laboratories analyzed composted Class A matrices resulting in three valid data
sets.
6.4 Method 1681 Confirmation Rates for Class B Matrices
False positive rates for Class B matrices using Method 1681 ranged from 0% - 13%. False negative rates
ranged from 0% - 10%.
Aerobically Digested Matrices
• False positive rates for Laboratories 3 and 8 were 13% and 0%, respectively, with false negative rates
of 0% and 0%, respectively. Two laboratories analyzed aerobically digested Class B matrices
resulting in two valid data sets.
Anaerobically Digested Matrices
• False positive rates for Laboratories 1, 5, 11, 19, and 26 were 9%, 4%, 13%, 0%, and 0%,
respectively, with false negative rates of 0%, 10%, 1%, 2%, and 8%, respectively. Five laboratories
analyzed anaerobically digested Class B matrices resulting in five valid data sets.
6.5 Methods 1680 and 1681 Confirmation Rates for Alkaline-Stabilized Matrices
For alkaline-stabilized matrices using Method 1680, false positive and false negative rates were relatively
low ranging from 0% - 11% and 0% - 8%, respectively. False positive and false negative rates for
Method 1681 were also relatively low at 9% and 2%, respectively.
16
-------
False Positive and False Negative Assessment Report
Alkaline-Stabilized Matrices Analyzed Using Method 1680
• False positive and negative rates for Laboratories 2 and 22 were 11% and 0% and 8% and 0%,
respectively. Two laboratories analyzed alkaline-stabilized matrices resulting in two valid data sets.
Alkaline-Stabilized Matrices Analyzed Using Method 1681
• The false positive and negative rate for Laboratory 2 was 9% and 2%, respectively. One laboratory
analyzed an alkaline-stabilized matrix resulting in a single valid data set.
17
-------
False Positive and False Negative Assessment Report
SECTION 7.0 CONCLUSIONS
The objective of this study was to assess false positive and false negative confirmation rates for fecal
coliform Methods 1680 and 1681 as they were not characterized based on biochemical verification during
the validation study for these methods in sewage sludge.
In this study EPA has characterized the false positive and false negative rates for Methods 1680 and 1681
for fecal coliforms in Class A and B sewage sludge across multiple laboratories and sewage sludge
matrices. During the study, 23 participating laboratories streaked growth from 996 (767 typical and 229
atypical) tubes for isolation on mEndo plates and submitted the plates to the verification laboratory for
confirmation between February 2006 and August 2006.
False Positive Rates
Overall false positive rates for Method 1680 ranged from 0% - 17% for all six matrices (aerobically
digested, alkaline-stabilized, anaerobically digested, composted, heat-dried, and thermophilically
digested).
Overall false positive rates for Method 1681 were relatively low for three of the matrices (aerobically
digested, alkaline-stabilized, and anaerobically digested) ranging from 3% - 9%. The false positive rate
for composted matrices was considerably higher, at 37%. Since materials used in the composting process
may cause tubes to appear turbid (not due to growth) and may also produce air bubbles (not gas) when
agitated, it is recommended that laboratories characterize each composted matrix evaluated in-house to
reduce the number of false positives and increase analyst proficiency.
False Negative Rates
Overall false negative rates were very low for Method 1680, ranging from 0% - 5% for all six matrices.
In addition, overall false negative rates for Method 1681 were also very low, ranging from 0% - 3% for
aerobically digested, alkaline-stabilized, anaerobically digested, and composted matrices.
Although the false negative rates were adjusted to include the number of "true" negatives [clear tubes (not
submitted for confirmation)], a significant number of negative tubes (turbidity and no gas) submitted to
verification verified as false negatives (82/152). This indicates that laboratories may be incorrectly
reading the tubes. In some instances gas production may be very weak (especially as sample volume
analyzed is decreased) and may be recorded incorrectly as a negative.
Analyst Proficiency Recommendations
The high false positive and negative rates observed for some method/matrix combinations may be due to
lack of familiarity with the matrices. To ensure that analysts are familiar with the different matrices
evaluated for fecal coliforms using Method 1680 and/or Method 1681, analysts should verify a minimum
often percent of all positive and negative tubes for each matrix evaluated to increase proficiency.
18
-------
False Positive and False Negative Assessment Report
SECTION 8.0 REFERENCES
8.1 USEPA. 2005. Assessment of the Effects of Holding Time on Fecal Coliform and Salmonella
Concentrations in Biosolids. EPA-821-R-04-029.
8.2 USEPA. 2006. Assessment of the Effects of Holding Time on Fecal Coliform and Salmonella
Concentrations in Biosolids. EPA-821-R-07-003.
8.3 USEPA. 2005. EPA Method 1680: Fecal Coliforms in Sewage Sludge (Biosolids) by Multiple-
Tube Fermentation using Lauryl Tryptose Broth (LTB) and EC Medium
8.4 USEPA. 2005. EPA Method 1681: Fecal Coliforms in Sewage Sludge (Biosolids) by Multiple
Tube Fermentation using A-1 Medium
8.5 Gilbert, R.O. 1987. Statistical Methods for Environmental Pollution Monitoring. VanNostrand
Reinhold, New York.
19
-------
False Positive and False Negative Assessment Report
SECTION 9.0 ACRONYMS
LTB Lauryl tryptose broth
LIA Lysine iron agar
MPN Most probable number
MSRV Modified semisolid Rappaport-Vassiliadis medium
QC Quality control
RSD Relative standard deviation
SAS Statistical analysis software
SD Standard deviation
TSA Tryptic soy agar
TSB Tryptic soy broth
TSI Triple sugar iron agar
XLD Xylose-lysine desoxycholate agar
20
-------
False Positive and False Negative Assessment Report
Appendix A:
Verification Instructions
21
-------
False Positive and False Negative Assessment Report
Sewage Sludge Holding Time Study
Verification Instructions for False Positive/False Negative Rates
Method 1680 and 1681: Fecal Conforms (Unspiked samples only)
Note: Verification of positive tubes and negative tubes with growth from unspiked samples will only be
performed during the holding time study - not during preliminary analyses.
Remove 6 hour and 24 hour EC and A-l tubes from refrigerator.
• For each matrix and method, submit growth from 25 positive tubes and 25 negative tubes with growth
to verification.
For each positive and negative tube being verified, streak growth from tubes for isolation onto m-
Endo plates and incubate at 35.0°C ± 0.5°C for 24 ± 2 hours.
• Label each m-Endo plate with corresponding sample ID and tube dilution and number (e.g.
Rep. #1, tube #3, 0.1 dilution).
• After 24 ± 2 hours, remove m-Endo plates from incubator.
• To prepare the m-Endo plates for shipping to the verification laboratory, wrap the edges of the plates
with parafilm and wrap the stack of plates associated with each sample with bubble wrap.
Place the plates in a cooler lined with a trash bag and surround the plates with blue ice. Seal the
cooler with shipping tape.
NOTE: Please ship the plates on the same day that they are removed from the incubator and contact
us on the day of shipment via phone/email/or fax:
Darcv Gibbons: 703-461-2308, dgibbons3@csc.com
Mary Smith: 703-461-2058, msmith247@csc.com
Fax: 703-461-8056
• Use provided FedEx mailing documents to ship the coolers to the verification laboratory
(Orange County Sanitation District, Attn: Charlie McGee).
22
------- |