U.S. Environmental Protection Agency
Environmental Technology Verification Program
Advanced Monitoring Systems Center
and
Environment Canada
Environmental Technology Verification
Program
Joint Verification Protocol for
Technologies for Rapid Detection of
Whole Soil and Soil Extract Toxicity
Battelle
Tlie Business of Innovation
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 2 of 42
Version 1.0
Date: 7/29/2008
Joint Verification Protocol
for
Verification of
Technologies for Rapid Detection of Whole Soil and Soil Extract
Toxicity
July 29, 2008
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
and
Environment Canada
Environmental Science and Technology Centre (ESTC)
335 River Road
Ottawa, ON K1A OH3
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 3 of 42
Version 1.0
Date: 7/29/2008
AKNOWLEDGMENTS
This joint verification protocol was developed through a collaboration between the
Environmental Technology Verification (ETV) programs of the United States and Canada and is
intended for use by both Canada and the United States either individually or together in jointly
verifying technologies which will be recognized by both countries. This joint protocol can also
be used by a vendor who would like to receive verification from either or both countries. This
joint verification protocol expands upon and replaces the generic verification protocol previously
prepared by the U.S. Environmental Protection Agency's (EPA's) ETV Advanced Monitoring
Systems (AMS) Center (Generic Verification Protocol for Technologies for Rapid Detection of
Soil Toxicity, April 2007). The AMS Center's generic protocol was prepared in coordination
with the EPA's Office of Solid Waste and Emergency Response and National Exposure
Research Laboratory and included input from vendors and stakeholders. Peer reviewers for the
protocol were Robert Seyfarth, Kenneth Hill, and Amy Juchatz of the Suffolk County (New
York) Department of Health Services' Division of Environmental Quality; Dr. John Hayse, I.
Hlohoskyj, and Leroy Walston of Argonne National Laboratory's Environmental Science
Division; and Dr. Karen Bradham of EPA's National Exposure Research Laboratory. Dr. Lisa
Taylor of Environment Canada's Environmental Science and Technology Centre provided
significant contribution to the revision of the AMS Center's generic protocol to create the joint
protocol. The contributions to this protocol from John Neate of ETV Canada are also gratefully
acknowledged. The joint protocol was reviewed by Robert Seyfarth and Leroy Walston, as well
as by Deana Crumbling of EPA's Office of Solid Waste and Emergency Response.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 4 of 42
Version 1.0
Date: 7/29/2008
TABLE OF CONTENTS
Section Page
SECTION A PROJECT MANAGEMENT 9
Al VERIFICATION TEST ORGANIZATION 9
A2 BACKGROUND 10
A3 VERIFICATION TEST DESCRIPTION AND SCHEDULE 11
A4 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA 13
A5 SPECIAL TRAINING/CERTIFICATION 16
A6 DOCUMENTATION AND RECORDS 16
SECTION B MEASUREMENT AND DATA ACQUISITION 19
Bl EXPERIMENTAL DESIGN 19
B2 SAMPLING REQUIREMENTS 30
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS 31
B4 LABORATORY REFERENCE METHODS 32
B5 QUALITY CONTROL 32
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE. 33
B7 CALIBRATION/VERIFICATION OF TEST PROCEDURES 33
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES 34
B9 NON-DIRECT MEASUREMENTS 34
BIO DAT A MANAGEMENT 34
SECTION C ASSESSMENT AND OVERSIGHT 35
Cl ASSESSMENTS AND RESPONSE ACTIONS 35
C2 REPORTS TO MANAGEMENT 38
SECTION D DATA VALIDATION AND USABILITY 39
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS 39
D2 VALIDATION AND VERIFICATION METHODS 39
D3 RECONCILIATION WITH USER REQUIREMENTS 40
SECTION E REFERENCES 42
El REFERENCES 42
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 5 of 42
Version 1.0
Date: 7/29/2008
List of Tables Page
Table 1. General Verification Test Schedule 13
Table 2. DQIs and Criteria for Critical Measurements for Reference Method 15
Table 3. Summary of Data Recording Process 17
Table 4. Categories and Example Contaminants 20
Table 5. Example Contaminant Compound Confirmatory Methods 32
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 6 of 42
Version 1.0
Date: 7/29/2008
DISTRIBUTION LIST FOR U.S. EPA ETV AND ENVIRONMENT CANADA ETV
JOINT VERIFICATION PROTOCOL
U.S. Environmental Protection Agency
Douglas Grosse, ETV Advanced Monitoring Systems (AMS) Center Project Officer
Lauren Drees, Director of Quality Assurance, National Risk Management Research Laboratory
Teresa Harten, EPA ETV Program Director
Battelle
Amy Dindal, AMS Center Manager
Mary Schrock, Verification Test Coordinator
Zachary Willenberg, AMS Center Quality Manager
Environment Canada
Abe Finkelstein, Chief, Innovative Solutions Section, Science & Technology Branch
Raymond Klicius, Head, Incentive Programs, Science & Technology Branch
Lisa Taylor, Ph.D., Manager, Method Development and Applications Section, Biological
Methods Division, Environmental Science and Technology Centre
ETV Canada
Steve Guerin, General Manager
Mona El Hallak, Senior Technical Director
John Neate, Senior Associate
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 7 of 42
Version 1.0
Date: 7/29/2008
DISTRIBUTION LIST FOR TEST/QUALITY ASSURANCE PLANS
Note: Test/Quality Assurance Plans developed from this Joint Verification Protocol for
verifications which will be recognized by both the U.S. and Canada should be distributed to the
Verification Program and Verification Organization associated with both countries.
Verifications which apply to one country require distribution only to the Verification Program
and Verification Organization pertinent to the verifying country.
Verification Program (United States: U.S. EPA ETV, Canada: Environment Canada ETV)
Verification Organization (United States: Battelle AMS Center, Canada: ETV Canada,
Verification entity to be determined)
Peer Reviewers
Vendors of Technologies for Rapid Detection of Soil Toxicity
Reference Laboratory
Test Facility, if applicable
Test Collaborators, if applicable
Subcontractors, if applicable
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 8 of 42
Version 1.0
Date: 7/29/2008
LIST OF ABBREVIATIONS/ACRONYMS
AMS Advanced Monitoring Systems
COA certificate of analysis
COC chain-of-custody
DQI data quality indicator
ECso median effective concentration causing 50% inhibition
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
LOEC lowest observed effect concentration
LRB laboratory record book
MSD minimum significant difference
NIST National Institute of Standards and Technology
NOEC no observed effect concentration
PCB polychlorinated biphenyl
PD percent difference
pdf Adobe portable document format
PE performance evaluation
QA quality assurance
QC quality control
QCS quality control samples
QMP quality management plan
RSD relative standard deviation
SOP standard operating procedure
TCDD 2,3,7,8-tetrachlorodibenzodioxin
TQAP test/quality assurance plan
TSA technical systems audit
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 9 of 42
Version 1.0
Date: 7/29/2008
SECTION A
PROJECT MANAGEMENT
Al VERIFICATION TEST ORGANIZATION
This protocol provides generic procedures for implementing a verification test for
technologies that rapidly detect soil toxicity in whole soil or soil extracts. This protocol outlines
a testing approach which is acceptable under both the U.S. EPA ETV and ETV Canada
verification programs. However, acceptability of the testing approach as outlined in this protocol
does not imply automatic verification by both the U.S. EPA ETV and ETV Canada verification
programs. Each specific round of testing will require preparation of a Test/Quality Assurance
Plan (TQAP). The TQAP will specify which verification programs and verification
organizations are involved in testing and verifying the technologies involved. This could be
jointly with both the U.S. EPA ETV and ETV Canada programs conducting testing and
verification together, in which the verification would be recognized by both countries' programs,
or individually where either U.S. EPA ETV or ETV Canada conducts testing and verification and
the verification is recognized by only one country's program.
Because the organizations involved in testing may vary, specific roles and responsibilities
will not be defined here, but must be defined in the TQAP prepared for each round of testing.
Information on roles and responsibilities defined in each TQAP should include the following
groups or individuals involved in each test:
. Verification program(s)
Verification organization(s)
. Key testing staff (verification organization program manager, testing leaders,
verification coordinators, technical staff, etc.)
Technology vendors
. Reference laboratories
. Test facilities
. Quality Managers for both the verification program and the verification organization.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 10 of 42
Version 1.0
Date: 7/29/2008
A2 BACKGROUND
The purpose of verification programs such as the U.S. EPA and Canadian Environmental
Technology Verification (ETV) programs is to provide objective and quality-assured
performance data on environmental technologies, so that users, developers, regulators, and
consultants can make informed decisions about purchasing and applying these technologies.
Stakeholder committees of buyers and users of such technologies recommend technology
categories, and technologies within those categories, as priorities for testing. As documented in
meeting minutes, technologies for rapidly detecting soil toxicity were identified as a priority
technology category through the U.S. EPA ETV Advanced Monitoring System (AMS) Center
stakeholder process since these technologies have the potential to make the evaluation of soil
toxicity more efficient and timely.
Soil toxicity testing can be used at hazardous waste sites to screen for particular areas of
concern or to assist in monitoring the effectiveness of cleanup. Soil toxicity tests do not require
knowing the contaminants present at the site; they are typically used as a broad range screen of
all potentially toxic compounds that may be present. Traditional soil toxicity tests include
evaluations such as seed germination and root elongation, as well as organism-based tests such
as earthworm survival.(1) Tests such as these can take several weeks to achieve results. This
protocol provides procedures for a verification test of rapid analysis technologies that detect
toxicity in whole soil and soil extracts. The objective of this soil toxicity technology verification
test is to evaluate the technology's ability to detect certain analytes that are particularly toxic to
humans by adding them, individually, to a controlled experimental matrix, as well as by testing
various "real-world" soil samples where the toxins may be present alone or with various other
toxins. This joint protocol outlines testing for a number of contaminants that are common to site
cleanups and known to be toxic, but does not include all toxic compounds or testing in all
situations which may be encountered in a site cleanup or evaluation situation. Data generated
from verification tests based on this joint protocol are intended to provide one set of objective
and quality assured performance data on soil rapid toxicity technologies, to assist users,
developers, regulators, and consultants in making informed decisions about purchasing and
properly applying these technologies.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 11 of 42
Version 1.0
Date: 7/29/2008
This verification test will determine the performance characteristics of commercially
available technologies that can provide results that make quicker, more efficient soil toxicity
determinations than the traditional tests which may take several weeks. Critical characteristics of
the soil toxicity technologies that will be assessed during this testing include the following:
• Endpoint
• Precision
• False negative rate
• False positive rate
• Sensitivity
• Matrix effects
• Data completeness
• Operational factors such as ease of use and maintenance
• Field portability.
A3 VERIFICATION TEST DESCRIPTION AND SCHEDULE
A3.1 Summary of Technology Category
Technologies applicable to this technology category can be those designed to directly test the
soil, or just the soil extract. These technologies are not intended to be a substitute for chemical
analyses for contaminants of interest but rather can be used as a complement that provides an
assessment of the biological response of toxicity. Conventional soil toxicity methods often can
take weeks to achieve results. Technologies to be evaluated in the verification of rapid soil
toxicity technologies include those that produce results within a substantially reduced time. This
may be within 24 hours for technologies that test soil extracts to within several days for those
involving whole soil. Such rapid soil toxicity technologies have the potential to expedite the
decision-making process for regulators. Rapid soil toxicity technologies do not provide a
measured concentration of specific toxins; rather, they provide a broad range screen of the toxic
nature of the soil. Specific procedures for the operation of each technology will be supplied as
part of the verification test. For technologies which operate by extracting an aliquot of soil and
then testing the extract, an extract may be added to bacteria, bioluminescent plankton, or other
such organisms or compounds which produce a measurable response that varies based on the
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 12 of 42
Version 1.0
Date: 7/29/2008
toxicity of contaminants in the soil. The specific response may vary by technology, but could
include a change in color or light intensity, respiration rate, or other response that is related to the
concentration level of the contaminant(s). Similarly, whole soil tests may involve germinating
seeds in both contaminated soil and reference soil and comparing the decrease or absence of
germination or root growth in the contaminated soil to that which occurs in the reference soil. In
this case, the inhibition in germination or root growth is related to contaminant concentration
levels.
A3.2 Verification Test Schedule
A verification test following this protocol should take approximately nine months to
complete. Test planning and preparation may take place over a period of several months once
vendors are committed to the test. Actual testing should be completed within two months. Data
review and reporting should be completed within four to five months. Table 1 shows a general
schedule of testing and data analysis/reporting activities to be conducted in a verification test that
follows this protocol. The test procedures are described in Section B of this protocol.
Subsequent to testing, a separate verification report will be drafted for each technology. Each
draft report will be peer-reviewed, revised, and submitted for final approval. Technologies for
detecting soil toxicity and associated equipment (but not consumables) will be returned to the
vendors at the completion of report writing.
A3.3 Test Facility
The test facility should be a location that can accommodate laboratory testing of
technologies for detecting toxicity in soil. This could be laboratory facilities at Battelle,
Environment Canada, or other such laboratory facilities that routinely test soil. Field portability
testing, if applicable, will be conducted by transporting the technology from a laboratory to a
non-laboratory area. In addition to a traditional field setting, non-laboratory areas could include
warehouses, shipping/receiving areas, storerooms, courtyards, and/or parking lots.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 13 of 42
Version 1.0
Date: 7/29/2008
A3.4 Health and Safety
All reference analyses and verification testing will follow the safety and health protocols
in place for the test facility. This includes maintaining a safe work environment and a current
awareness of handling potentially toxic chemicals. Exposure to potentially toxic chemicals will
be minimized, personal protective equipment will be worn, and safe laboratory practices will be
followed.
Table 1. General Verification Test Schedule3
Month
1-2
2-3
4-5
6
7
8
9
Testing Activities
• Prepare draft TQAP and submit for
vendor and peer reviews
• Revise draft TQAP
• Finalize and obtain vendor
approval of TQAP
• Procure necessary standards and
reagents
• Vendor to set up technology and
train technical staff on technology
use
• Conduct verification tests
• Conduct reference tests and
performance evaluation audit of
reference methods
• Conduct technical systems audit
• Return equipment to vendors
Data Analysis and Reporting
• Review and compile test data and records as
they become available
• Review and summarize verification testing staff
observations
• Begin preparation of report template
• Evaluate and analyze data generated during
testing
• Conduct data quality audits
• Complete report template
• Complete draft reports and submit for vendor
and peer review
• Revise draft reports and submit final reports for
approval by the verification program
• Distribute finalized, approved reports
• Post reports and verification statements on
verification program and verification
organization web sites
a Verification schedule begins once vendor(s) and collaborators(s) are committed to the verification test.
A4 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA
In performing the verification test, the verification organization will follow the technical
and QA procedures specified in this protocol and will comply with the data quality requirements
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 14 of 42
Version 1.0
Date: 7/29/2008
in the verification organization's quality management plan. The objective of this verification test
is to evaluate the performance of soil toxicity detecting technologies in their ability to measure
the presence of toxins in whole soil or soil extracts under controlled laboratory conditions. This
evaluation will assess the capabilities of the soil toxicity technologies to detect toxins added to a
controlled experimental matrix, as well as their ability to detect toxins in "real-world"
environmental samples. The evaluation will include a comparison of the soil toxicity technology
results to known concentrations of toxins in the test samples that will be confirmed as described
in Section B4. Additionally, this verification test will rely upon verification testing staff
observations to assess other performance characteristics of the technologies. Below is a
discussion of the quality objectives and the criteria for measurement data that have been
established to ensure that the test objectives are met.
A4.1 Quality Objectives
Data quality objectives assure that the data quality, quantity, and type are appropriate to
meet the verification test objectives and specify the minimum acceptance criteria for these
parameters. Data quality objectives for this verification test include those related to the reference
method performance and those related to the soil toxicity detecting technology performance, as
well as those related to documenting verification testing staff observations. Data quality
objectives for the reference methods (see Section B4) are presented in terms of data quality
indicator (DQI) criteria for the critical measurements associated with the reference methods. The
DQI criteria are listed in Table 2 and discussed in Section A4.2. The reference method data
quality relies, in part, on proper sample preparation, proper application of the reference method,
and proper maintenance of reference method instrumentation. The verification organization will
rely on the vendor's data quality objectives for each technology in order to ensure that the
technology is performing properly during testing. This will include adhering to each vendor's
criteria for calibration and performance of positive and negative control samples. The
technology data quality relies on proper operation and maintenance of the technologies and
proper sample preparation, as instructed by the vendor. Quantitative data quality objectives for
the operator observations have not been defined but are incorporated into documentation
requirements and data review, verification, and validation requirements for this verification test.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 15 of 42
Version 1.0
Date: 7/29/2008
Table 2. DQIs and Criteria for Critical Measurements for Reference Method
DQI
Bias and Accuracy
of Sample
Measurements
Completeness
Method
Representativeness
Method of
Assessment
Initial Calibration —
various levels as
specified in
reference method
Calibration Check
Sample — single-
level continuing
check of calibration
as specified in
reference method
Method Blank
Spiked Samples
Amount of valid
data obtained
Performance Test
Sample
Frequency
As required in
reference
method
As required in
reference
method
As required in
reference
method
As required in
reference
method
Overall
number of
data points
collected for
reference
method
Once, prior to
verification
testing
Minimum
Acceptance
Criteria
Refer to reference
method criteria
Refer to reference
method criteria
Refer to reference
method criteria
Refer to reference
method criteria
90% of overall data
points collected should
be valid.
Results within ± 10%
of expected value for
standard solutions,
results within certified
limits for standard
reference materials
Corrective Action
Investigate sources of
contamination or changes
in instrument parameters;
perform instrument
maintenance as needed;
reanalyze fresh standard or
sample, or repeat initial
calibration.
If feasible, analyze
additional samples to meet
the acceptance criterion.
Evaluate reference method
performance; perform
maintenance or
recalibration as required,
repeat performance test. If
performance test criteria
cannot be met, consider
alternative reference
laboratories.
A4.2 Criteria for Measurement Data
Table 2 presents the DQIs and general criteria for the reference method critical
measurements. Specific criteria should be added to the TQAP once the reference method is
known. The reference method measurement quality will be ensured by adhering to these DQI
criteria and monitored by following the calibration procedures and frequency recommended in
each respective reference method and by including method blank or spiked samples as indicated
in each reference method. Additionally, performance test samples will be sent to each laboratory
providing reference method analyses prior to analysis of verification test samples. Performance
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 16 of 42
Version 1.0
Date: 7/29/2008
test samples will be standard solutions or standard reference materials containing known
quantities of the analytes of interest. Each vendor will provide criteria for the soil toxicity
technologies for critical measurements related to calibration standards and recommendations for
appropriate positive and negative controls and their critical measurements. The verification
organization's Quality Manager or designee will perform a ISA at least once during this
verification test to review these QA/quality control (QC) requirements. The ETV verification
program's Quality Manager (US and/or Canada) also may conduct an independent TSA if
desired.
A5 SPECIAL TRAINING/CERTIFICATION
Documentation of training related to technology testing, field testing, data analysis, and
reporting should be maintained for all technical staff involved in verification testing. Location of
these training records should be documented in the TQAP. Documentation of the expertise and
experience of collaborators and/or subcontractors must be similarly available. Any minimum
education or experience requirements for testing staff should be specified in the TQAP. The
verification organization Quality Manager may verify the presence of appropriate training
records prior to the start of testing. If technical staff operate and/or maintain a technology during
the verification test, the technology vendor will be required to train those staff prior to the start of
testing. The verification organization will document this training with a consent form, signed by
the vendor, that states which specific technical staff have been trained on their technology.
A6 DOCUMENTATION AND RECORDS
The records for this verification test will include the TQAP based on this protocol, chain-
of-custody (COC) forms, laboratory record books (LRBs), data collection forms, electronic files
(both raw data and spreadsheets), and the final verification reports and verification statements.
The storage location for these records should be specified the TQAP. The verification program(s)
should be notified before disposal of any files. The QA/QC documentation and results of the
reference measurements made by the reference laboratory should be submitted to the verification
organization immediately upon completion of all sample analyses and maintained with the
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 17 of 42
Version 1.0
Date: 7/29/2008
records for this test. Table 3 has further details regarding the data recording practices and
responsibilities.
Table 3. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test events,
technology
maintenance,
downtime, etc.
Technology
calibration
information
Technology readings
Sample preparation
and reference
method analysis
procedures,
calibrations, QA, etc.
Reference method
results
Where Recorded
ETV LRBs or data
recording forms
ETV LRBs, data
recording forms, or
electronically
Either recorded
electronically by the
technology and
downloaded to an
independent
computer or storage
medium, hard copy
data printed by the
technology and taped
into an ETV LRB, or
handwritten records
into an ETV LRB or
on data sheets
LRBs, COC, or other
data recording forms
Electronically from
analytical method or
documented in
handwritten records
How Often
Recorded
Start/end of test
procedure, and at
each change of a test
parameter or change
of technology status
At technology
calibration or
recalibration
Every sample
analysis.
Throughout sampling
and analysis
processes
Every sample
analysis
By Whom
Technical staff
Technical staff or
vendor performing
the calibration
Technical staff
Technical staff and
Reference laboratory
Reference laboratory
Disposition of
Data
Used to organize and
check test results;
manually
incorporated in data
spreadsheets as
necessary
Incorporated in
verification report as
necessary
Transferred to or
manually entered
into spreadsheet for
statistical analysis
and comparisons
Retained as
documentation of
reference method
performance
Transferred to or
manually entered
into spreadsheets for
calculation of results,
and statistical
analysis and
comparisons as
needed
All written records must be in ink. Any corrections to notebook entries, or changes in
recorded data, must be made with a single line through the original entry. The correction is then
to be entered, initialed, and dated by the person making the correction. In all cases, strict
confidentiality of data from each vendor's technology, and strict separation of data from different
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 18 of 42
Version 1.0
Date: 7/29/2008
vendors' technologies, will be maintained. Separate files (including manual records, printouts,
and/or electronic data files) will be kept for each technology.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 19 of 42
Version 1.0
Date: 7/29/2008
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
This joint protocol outlines testing for a number of contaminants that are common to site
cleanups and known to be toxic to humans, but does not include all toxic compounds or testing in
all situations which may be encountered in a site cleanup or monitoring situation. Data
generated from verification tests based on this joint protocol are intended to provide one set of
objective and quality assured performance data on rapid soil toxicity technologies, to assist users,
developers, regulators, and consultants in making informed decisions about purchasing and
properly applying these technologies that would be acceptable for consideration under both the
US EPA ETV and ETV Canada verification programs. These technologies do not provide
identification or concentration of specific contaminants, but serve as a rapid screening tool to
determine whether the soil being tested is toxic. As part of this verification test, the technologies
will be subjected to various concentrations of chemicals representing several categories of
common contaminants such as commercial solvents, pesticides, persistent pollutants, and metals.
At a minimum, the categories listed in Table 4 should be evaluated during verification testing.
The specific compounds to be tested may be added or replace compounds in Table 4 (as
described in the TQAP) depending upon the capabilities of the technologies being tested. For
technologies evaluating soil extracts, each contaminant will be added individually to separate
aliquots of sand, and the spiked sand will be analyzed by the technologies. Sand is
recommended as the matrix for the spiking experiments because it is inert and it is anticipated
will minimally retain the contaminants of interest thereby providing an estimate of technology
performance in the case where nearly 100% of the contaminant would be extractable. For
technologies evaluating whole soil, each contaminant will be added individually to separate
aliquots of artificial soil [e.g., Organization of Economic Cooperation and Development (OECD)
artificial soil], and the spiked soil will be analyzed by the technologies. Artificial soil is
recommended for the whole soil technologies to optimize the biological response and more
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 20 of 42
Version 1.0
Date: 7/29/2008
accurately simulate interactions of the chemical with soil properties. The physical properties of
the artificial soil may vary depending on the desired characteristics (e.g., water content may be
manipulated to optimize the biological response). The exact procedures for sample preparation
will be detailed in each TQAP prepared for testing. Additionally, both soil extract and whole soil
technologies will be challenged with "real-world" environmental samples of various soil types
containing a variety of the contaminants in Table 4 to evaluate the technology performance on
samples more representative of those found in practical application of the technologies. These
samples are described in Section Bl.l.
All of the technologies will be tested in a laboratory. The technologies designed for use in
a field location will also be tested at a non-laboratory venue.
The analyses will be performed according to the vendor's recommended procedures as
described in the user's instructions or manual, or during training provided to the technical staff.
Similarly, calibration and maintenance of the technologies will be performed as specified by the
vendor. Results from the technologies being verified will be recorded manually by the operator
on appropriate data sheets or captured in an electronic data system and then transferred manually
or electronically for further data workup. Qualitative operational characteristics of each
technology such as ease of use will be assessed through observations made by the technical staff
throughout the verification test. The results from each technology will be reported individually.
According to ETV policy, no direct comparison will be made between technologies, but each
technology will undergo similar testing and will be reported in a similar manner.
Table 4. Categories and Example Contaminants
Category
Commercial solvents
Carbamate pesticide
Organophosphate pesticide
Metals
Persistent pollutants
Example Contaminant
Trichloroethylene
Toluene
Aldicarb
Dicrotophos
Arsenic
Lead
Mercury
Cadmium
Polychlorinated biphenyls (PCBs) (as Aroclor 1254)
2,3,7,8- tetrachlorodibenzodioxin (TCDD)
Benzo[a]pyrene
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 21 of 42
Version 1.0
Date: 7/29/2008
Bl.l Test Procedures
The verification test for technologies that detect toxicity in whole soil and/or soil extracts
will focus on a broad range of samples to provide a variety of toxin concentrations. This
verification will focus on evaluating dose/response relationships to specific contaminants known
to be toxic to humans as well as assessing the technology's ability to provide a toxic response to
real-world environmental samples for which known contaminants have been well characterized
using standard analytical laboratory methods.
The first sample type will be performance test samples where individual toxins will be
added to a clean sand or artificial soil. Sand is recommended for soil extract testing as an inert
matrix which will minimally retain the toxins. Use of an inert matrix will eliminate the matrix
itself from influencing the lowest detectable concentration of each contaminant and will evaluate
technology performance under conditions where the toxin is anticipated to be nearly 100%
extractable. For whole soil toxicity testing, an artificial soil is recommended to optimize the
biological response and more accurately simulate interactions of the chemical with soil
properties. Because the types of technologies anticipated to be tested will provide a broad range
screen of all potentially toxic compounds that may be present, the sand/artificial soil selected
should be free of any compounds which would cause a toxic response and not just free of the
contaminants of interest for the verification test. The sand/artificial soil will be spiked with each
contaminant at concentrations ten times screening or remediation goal levels (e.g., EPA Region 9
Superfund Preliminary Remediation Goals) as the highest concentration and will be analyzed in
replicate (minimum of three). Subsequent tenfold dilutions (i.e., spiking the sand/artificial soil
with a contaminant solution which is tenfold dilute from the starting level) will be prepared and
analyzed in replicate (minimum of three) until there is no longer a measurable response
indicating toxicity (i.e., inhibition as measured by each technology such as a reduction in light
output, change in respiration rate, etc.), up to a maximum of five dilutions below the highest
concentration. From these data, the lowest concentration at which the toxicity can be detected
can be estimated for each technology with respect to each contaminant. The second sample type
will be "real-world" environmental samples and will consist of 5 to 10 soils collected from
various cleanup sites or standard reference soils with well documented soil characteristics.
These samples will reflect a variety of soil types and will include soils known to contain the
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 22 of 42
Version 1.0
Date: 7/29/2008
contaminants of interest in this test (individually or in combination with other contaminants) as
well as some soils which are known to be free of contaminants, such as American Society for
Testing Materials artificial soil or Environmental Resource Associates Semivolatile Blank Soil
(Catalog Number 056). The environmental samples will be dried and homogenized (e.g., oven
dried using low heat with specifics to be detailed in the TQAP) prior to use in testing to ensure
that sample homogeneity is not a significant factor in technology performance. Because the
drying and homogenization process has the potential to affect the concentration of contaminants
in the samples, the concentration of contaminants will be measured after the drying and
homogenization process has taken place to ensure that the measured concentrations of
contaminants in the environmental samples accurately reflect the material used in testing.
Contaminants in the environmental samples will be measured using the same reference methods
that will be used to confirm the concentration of spiked contaminants in the performance test
samples. Appropriate soil characteristics such as total organic carbon, grain size distribution,
and pH should also be measured once the environmental samples have been dried and
homogenized. Information about the soil characteristics may aid in understanding differences in
the various environmental samples that will be tested. To the extent possible, each time
verification testing is conducted following this joint protocol, attempts should be made to use the
same environmental sites, or at a minimum sites with comparable types of soil and types and
quantities of contaminants. As with the performance test samples, environmental sample
selection should be made with consideration of the fact that the rapid toxicity tests may respond
to all toxic compounds and not just the contaminants included in verification testing. Therefore,
environmental samples should be as well characterized as possible prior to use in testing. Test
results, and in particular assessments of false positive/negatives and matrix effects, should take
into consideration whether the technology could be responding to unknown toxic compounds in
the samples. The third type of sample will be quality control samples. Quality control samples
are discussed further in Section B5.
It should be noted that the technologies covered by this protocol may include test
organisms (e.g., microorganisms or invertebrates) which may involve handling, culture or
preparatory work and will likely vary with each type of technology. It is expected that each
technology vendor will include instructions on proper preparation and handling of any test
organisms and that each vendor will outline in detail any procedures necessary to ensure healthy
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 23 of 42
Version 1.0
Date: 7/29/2008
test organisms (i.e., storage conditions, light, temperature, feeding etc.) and any procedures
necessary to ensure an accurate measurement of the biological response (i.e., UV light,
spectrophotometer, microscopes etc.). Additionally, each vendor will specify any QA aspects
that should be monitored to ensure proper preparation and handling of organisms (i.e. fridge
temperature records, calibration records, organism suppliers, taxonomic verification records,
health records etc.). Any preparation or QA aspects which will be uniformly applied across all
participating technologies will be detailed in the individual TQAPs prepared for tests performed
using this protocol.
Also note that inter-unit reproducibility (e.g., test kits from different lots, multiple
detectors, etc.) and inter-operator reproducibility are not addressed in this protocol. Should these
parameters be desired, the procedures for evaluating them will need to be added to TQAPs
prepared for specific tests. In general the same technician will be used to perform all testing
where possible. At a minimum, only technicians with equivalent training and experience
operating the technology will be used to perform verification testing.
The technologies will be evaluated for the parameters listed in sections B1.1.1 to B1.1.9.
If modification of these parameters is required due to the nature of the technologies being tested,
any changes will be described in the test-specific TQAP.
B 1.1.1 Endpoint
Each technology produces its own unique biological or biochemical endpoint derived
from the inhibition data gathered when analyzing various concentrations of contaminants in soil
[e.g., median effective concentration causing 50% inhibition (ECso) values]. For each
technology, the endpoint used for verification testing will be recommended by the vendor. The
endpoint will be used to assess whether or not there was a response to a test sample.
Bl.1.2 Precision
Inhibition results (endpoints) specific to each technology from replicates (minimum of
three) of each test sample will be evaluated. The average measurement, standard deviation (S),
and relative standard deviation (RSD) of the replicate measurements will be calculated and
reported in order to evaluate the precision of the technologies. To the extent possible and
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 24 of 42
Version 1.0
Date: 7/29/2008
appropriate, precision values will be reported with average data values so that measurement
uncertainty is understood.
Bl. 1.3 False Negative Rate
The false negative rate, or frequency of performance test sample inhibitions which are
similar to the negative control reported when a contaminant is present in the performance test
sample at toxic concentrations, will be calculated. Note that real-world environmental samples
could be used for the assessment of false negatives but this would make the evaluation much
more complicated due to the possibility of matrix effects.
BL 1.4 False Positive Rate
The false positive rate, or frequency of performance test sample detectable inhibitions
which are reported for unspiked samples, will be calculated. Note that real-world environmental
samples could be used for the assessment of false positives but this would make the evaluation
much more complicated due to the possibility of matrix effects.
Bl.1.5 Sensitivity
Various contaminants will be added individually to a controlled experimental matrix at
multiple concentration levels and analyzed by the participating technologies to assess their
ability to detect the toxicity of these contaminants (performance test samples). After analyzing
several concentrations of each contaminant (i.e., ten times the screening or remediation goal
level specified in the TQAP and subsequent tenfold dilutions up to a maximum of five dilutions
below the highest concentration), a sensitivity assessment will be made. The sensitivity
assessment to be used will be described in the TQAP because it may depend on the technologies
being tested. Examples of sensitivity assessments that could be used are an evaluation of the
lowest tested concentration which gives a response significantly different from the negative
control or a calculation of the lowest observed effect concentration (LOEC) and no observed
effect concentration (NOEC) along with an associated minimum significant difference (MSD)
value.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 25 of 42
Version 1.0
Date: 7/29/2008
Bl.1.6 Matrix Effects
Five to ten environmental samples representing a variety of soil types and contaminants
or mixtures of contaminants will be analyzed. The concentrations of contaminants present in the
environmental samples will be measured according to reference methods. The technology's
ability to detect contaminants in the environmental samples will be compared with the lowest
detectable level of contaminant determined for each technology to assess whether the
environmental sample matrix influenced the ability of the technology to detect toxicity.
Bl.1.7 Data Completeness
Data completeness will be determined as the number of valid measurements (i.e., useable
endpoint measurements with the technology) out of the total number of measurements taken. The
cause of any substantial loss of data will be established from technical staff observations or
technology records and noted in the discussion of the data completeness results.
Bl.1.8 Operational Factors
Operational and sustainability factors such as maintenance needs, calibration frequency,
data output, consumables used, ease of use, repair requirements, waste production, and sample
throughput will be evaluated based on technical staff observations. An LRB or data sheets will
be used to document observations. Examples of information to be recorded include the daily
status of diagnostic indicators for the technology, use or replacement of any consumables, the
effort or cost associated with maintenance or repair, vendor effort (e.g., time on-site) for repair or
maintenance, the duration and causes of any technology downtime or data acquisition failure,
quantity and hazardous nature of any waste generated, how to safely dispose of such waste,
operator observations about technology ease of use, clarity of the vendor's instruction manual,
user-friendliness of any needed software, overall convenience of the technologies and
accessories/consumables, and the number of samples that could be processed per hour or per day.
These observations will be summarized to aid in describing the technology performance in the
verification report on each technology.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 26 of 42
Version 1.0
Date: 7/29/2008
Bl.1.9 Field Portability
Testing the operation of the technologies in a field setting is a key component of the
verification test. Evaluating the performance of each field-portable technology while being used
outside the laboratory without the availability of miscellaneous laboratory supplies is important
to the buyers and users of these technologies. Technologies will be evaluated in a field setting
only if the vendor states that the technology has that capability. For those technologies that are
meant to be field-portable, this parameter will be assessed by transporting the technology to a
non-laboratory location. In addition to traditional field settings, non-laboratory areas could
include warehouses, shipping/receiving areas, storerooms, courtyards, and/or parking lots
provided the location meets the criteria that the area is absent of laboratory amenities such as
laboratory bench space, power, lighting, temperature control, storage and refrigeration, etc. as
would be the case in a traditional field setting. Ideally all of the samples included in the lab-
based tests would be repeated in the field; however, at a minimum one performance test sample
or environmental sample that had a strong response in the lab-based tests will be analyzed in
triplicate in the field. Results obtained in the field will be compared with the results for the same
sample obtained in the laboratory by the same technician where possible, or at a minimum by a
technician with equivalent training and experience operating the technology to the technician
who performed the laboratory analysis. Technical staff will also record observations related to
field portability such as requirements for power, space, and ease of use in and transport to a non-
laboratory setting.
B1.2 Statistical Analysis
The statistical methods and calculations used for evaluation of the quantitative
performance parameters are described in the following sections.
B.I.2.1 Endpoint
Each technology produces its own unique endpoint derived from the inhibition data
gathered when analyzing various concentrations of contaminants in soil (e.g., ECso values). For
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 27 of 42
Version 1.0
Date: 7/29/2008
each technology, these data will be documented and presented with respect to each contaminant
and concentration level using the appropriate endpoint for the technology.
Bl.2.2 Precision
The standard deviation (S) of the results for the replicate analyses of the same sample will
be calculated as follows.
£=,„_,
" ~ k=\
" (
I
1/2
(1)
where n is the number of replicate samples, A-4 is the endpoint measurement for the kih sample,
and Mis the average endpoint measurement of the replicate samples. The technology precision
for each sample will be reported in terms of RSD, which will be calculated as follows.
RSD(%) = =
^ } M
x 100 (2)
The average (M), standard deviation (S) and relative standard deviation (RSD) values for
each analyte at each concentration will be listed in data tables in the verification report; however,
verification statements and performance summary tables in the verification report will list the
range of RSDs obtained for all concentrations of each contaminant tested.
B1.2.3 False Negative Rate
Results will be considered false negative only when a technology is exposed to a
contaminant concentration greater than the desired remediation or screening level and the
technology does not indicate inhibition greater than the negative control. The rate of false
negatives, expressed as a percentage of total samples analyzed for each contaminant, will be
calculated by dividing the number of false negative measurements (Mfn) by the total number of
measurements included in verification testing (Mtotai).
Mfn ,-ON
FalseNegative(%) = x 100 (3)
Mtotai
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 28 of 42
Version 1.0
Date: 7/29/2008
B1.2.4 False Positive Rate
Results will be considered false positive only when an unspiked sample produces
inhibition greater than that of the negative control. The rate of false positives, expressed as a
percentage of total samples analyzed for each contaminant, will be calculated by dividing the
number of false positive measurements (Mfr) by the total number of measurements included in
verification testing (Mfoto/).
P fA\
FalsePositive(%) = x 100 (4)
Mtotal
Bl.2.5 Sensitivity
The sensitivity of the technology for detecting various contaminants in performance test
samples (i.e., sand spiked with contaminant) will be assessed. The exact procedures used to
assess sensitivity will be detailed in each TQAP developed from this protocol. Sensitivity
assessments that could be used are an evaluation of the lowest tested concentration of each
contaminant where the average inhibition plus or minus the standard deviation does not overlap
with the average inhibition plus or minus the standard deviation of the negative control or
determination of the LOEC and NOEC values along with an associated MSD value calculated
using an appropriate parametric multiple-comparison method (e.g., Dunnett's test). The
sensitivity of the reference method will not be assessed other than it must meet the DQI
requirements (Table 2).
Bl.2.6 Matrix Effects
The technology's ability to detect each contaminant in the environmental samples will be
compared with the technology's lowest detectable contaminant level determined by spiking the
contaminant into an inert matrix (i.e., sand) as described in Section Bl.2.5. If the contaminant
concentration in the environmental sample (measured using reference methods described in
Section B4) is above the lowest detectable level in an inert matrix (as determined in Section
Bl.2.5), but the technology result for the environmental sample is negative, matrix effects will be
considered to have contributed to this false negative response. It should be noted that rapid
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 29 of 42
Version 1.0
Date: 7/29/2008
toxicity technologies are intended to respond to all toxic compounds and not just contaminants of
interest in this verification test. As broad screens of toxicity, these technologies may also be
susceptible to toxicity potentiation and antagonism. Therefore, the matrix effect results reported
should consider the possibility of such toxicity effects and include discussion of such effects in
the verification report so that readers can understand matrices where the toxic response may be
affected. The number of environmental samples where matrix effects affected results (Mmatrix)
out of the total number of environmental samples tested (Mfoto/) will be reported as a percentage
using Equation 5.
MatrixEffect(%) = """" x 100 (5)
Mtotal
Bl.2.7 Data Completeness
Data completeness will be calculated as the percentage of the total possible data by
dividing the number of valid data measurements generated by each technology (Mvaiid) by the
total number of data measurements included in verification testing (Mtotai).
IVlvalid t/-\
Completeness^/^) = x 100 W
Mtotal
The cause of any substantial loss of data will be established from operator observations or
technology records and noted in the discussion of the data completeness results.
Bl.2.8 Operational Factors
There are no statistical calculations applicable to operational factors. Operational factors
will be determined based on documented observations of the technical staff.
Bl.2.9 Field Portability
The results obtained from the measurements made on samples in the laboratory and field
setting will be compiled independently for each technology and compared to assess the accuracy
of the measurements under the different analysis conditions. Means and standard deviations of
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 30 of 42
Version 1.0
Date: 7/29/2008
the endpoints generated in both locations will be compared and assessed for whether they are
statistically different.
B1.3 Reporting
The data obtained in the verification test will be compiled separately for each vendor's
technology, and the data evaluations will be applied to each technology's data set without
reference to any other. At no time will data from different vendors' technologies be
intercompared or ranked. Following completion of the data evaluations, a draft verification
report and verification statement will be prepared for each vendor's technology, stating the
verification test procedures and documenting the performance observed. For example,
descriptions of the data acquisition procedures, use of vendor-supplied proprietary software,
consumables used, repairs and maintenance needed, and the nature of any problems will be
presented in the draft report. Each report will briefly describe the verification program(s), the
verification organization(s), and the procedures used in verification testing. The results of the
verification test will be stated quantitatively, without comparison to any other technology tested
or comment on the acceptability of the technology's performance. Each draft verification report
will be submitted for review by the respective technology vendor, by the verification program(s),
and peer reviewers. Comments on the draft report will be addressed in revisions of the report.
The peer review comments and responses will be tabulated to document the peer review process.
The reporting and review process will be conducted according to the quality procedures set forth
by the verification program(s) and the verification organization(s).
B2 SAMPLING REQUIREMENTS
B2.1 Sample Collection, Storage, and Shipment
Environmental samples will be collected for use in a verification test following the
TQAP. As much as possible, samples will be obtained from known contaminated sites using the
same sampling techniques that are in place at the site for the site evaluation process. Samples
may be collected in bulk and shipped to the test facility in plastic buckets or other suitable
containers. Shipments will be via a trackable overnight delivery service to the test facility
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 31 of 42
Version 1.0
Date: 7/29/2008
sample custodian. Samples will be stored refrigerated or frozen as is appropriate for the
contaminants expected to be contained in the soil. Environmental samples will be dried (e.g.,
oven dried using low heat with specifics to be detailed in the TQAP) and homogenized prior to
use in testing to ensure that sample heterogeneity is a minimal factor in testing multiple
technologies. Because of the sample handling involved, the environmental samples will be
homogenized before concentrations of contaminants are measured using the reference methods.
Appropriate soil characteristics such as total organic carbon, grain size distribution, and pH
should be measured once the environmental samples have been dried and homogenized.
Information about the soil characteristics may aid in understanding differences in the various
environmental samples that will be tested.
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Sample custody will be documented throughout collection, transport, shipping (if
necessary), and analysis using standard COC forms provided by the verification organization or
supplied by the reference laboratory, as appropriate. Samples transferred within the verification
organization may be documented with internal COC forms. Each COC form will summarize the
samples collected and analyses requested. The COC forms will track sample release from the
sampling location to the test facility and/or reference laboratory; or release directly from the test
facility to the reference laboratory. Each COC form will be signed by the person relinquishing
the samples once that person has verified that the COC form is accurate. The original sample
COC forms will accompany the samples; the shipper will keep a copy. Upon receipt at the test
facility and/or reference laboratory, COC forms will be signed by the person receiving the
samples once that person has verified that all samples identified on the COC forms are present.
Any discrepancies will be noted on the form; and the sample receiver will immediately contact
the verification organization to report missing, broken, or compromised samples. Copies of all
COC forms will be delivered to the verification organization and maintained with the test
records.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 32 of 42
Version 1.0
Date: 7/29/2008
B4 LABORATORY REFERENCE METHODS
Table 5 lists the analytical methods that can be used to determine or measure the
concentration of contaminants analyzed during verification tests performed following this
protocol. Additional methods may be used provided they are appropriate for the contaminant
and matrix and are documented in the TQAP for the verification test.
B5 QUALITY CONTROL
Steps will be taken to maintain the quality of data collected during verification tests
conducted under this protocol. This will include analyzing specific quality control samples
(QCS) at a regular frequency by the technologies undergoing verification. The QCSs will
Table 5. Example Contaminant Compound Confirmatory Methods
Example Contaminant
Trichloroethylene, toluene
Aldicarb
Dicrotophos
Arsenic, lead, mercury, cadmium
PCBs (as Aroclor 1254)
2,3,7,8-TCDD
Benzo[a]pyrene
Method
SW-846 8260B(2)
EPA531.1(3)
SW-846 8141A(4)
EPA 200.8(5)
SW-846 8270C(6)
EPA 1613B(7)
SW-846 8270C(6)
include negative controls, positive controls, and calibration checks. Negative control samples,
consisting of unspiked experimental matrix, will help ensure that no sources of contamination are
introduced in the sample handling and analysis procedures. The positive control and calibration
check samples, specified by each vendor, will indicate to the technical staff whether or not the
technology is functioning properly. The vendor will provide the approximate endpoint that
should result with their technology upon analysis of the positive control and calibration check.
QCSs producing results that do not meet the anticipated results specified by the vendor will be
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 3 3 of 42
Version 1.0
Date: 7/29/2008
reanalyzed and corrective action taken if needed to ensure that test sample results are not
affected. Corrective actions may include reanalyzing samples to verify that the technology has
been operated properly, conducting maintenance, or recalibrating. Positive and negative controls
will be analyzed at a frequency of approximately 5% based on the total number of test samples.
Calibration checks will be analyzed according to guidance provided by each technology vendor.
As described in Section B4, the reference laboratory will follow standard reference
methods for determining the toxins evaluated during verification tests conducted under this
protocol. All reference measurements will be expected to meet the reference method QC
requirements (such as those listed in Table 2) or, in absence of specific requirements in the
reference method, the reference laboratory's standard requirements for QC samples.
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The equipment used by the test facility and/or reference laboratory will be tested,
inspected, and maintained as per the SOPs of the test facility and/or reference laboratory and/or
the manufacturer's recommendations so as to meet the performance requirements established in
the TQAP. When technical staff operate and maintain technologies undergoing testing, they will
follow directions provided by the technology vendor. Otherwise, operation and maintenance of
the technologies will be the responsibility of the technology vendor.
B7 CALIBRATION/VERIFICATION OF TEST PROCEDURES
Systems used for reference analyses will be calibrated as appropriate before any reference
samples are analyzed and recalibrated as needed based on the reference methods and/or reference
laboratory SOPs.
Technologies undergoing testing will be calibrated initially by the respective technology
vendor prior to shipping the technology to the test facility, or during training, and will be
recalibrated according to direction from the vendor. Calibration checks will be performed upon
direction of the vendor. In the event that recalibration is necessary, the recalibration will be
carried out by the technology vendor or by technical staff under the direction of the vendor. All
calibrations will be documented as appropriate by the technical staff or vendor.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 34 of 42
Version 1.0
Date: 7/29/2008
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
All materials, supplies, and consumables will be ordered by the verification organization,
unless otherwise donated from test collaborators. Where possible, the verification organization
will rely on sources of materials and consumables that have been used previously as part of
verification testing without problems. The verification organization will also rely on previous
experience or recommendations from the verification program(s), stakeholders, test
collaborators, subcontractors, or technology vendors. Where possible, materials or supplies will
be traceable to the National Institute of Standards and Technology (NIST). Upon receipt of any
supplies or consumables, the verification organization will visually inspect and ensure that the
materials received are those that were ordered and that there are no visual signs of damage that
could compromise the suitability of the materials. Certificates of analysis (CO A) or other
documentation of analytical purity will be checked for all reagents and standards to ensure
suitability for the verification test and will be included with the test files. If damaged,
unsuitable, or inappropriate goods are received, they will be returned or disposed of, and
arrangements will be made to receive replacement materials.
B9 NON-DIRECT MEASUREMENTS
No non-direct measurements will be used during this verification test.
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by the
verification organization, vendor, test collaborator, and/or subcontractor staff during the
verification test. Table 3 summarizes the types of data to be recorded. All maintenance activities,
repairs, calibrations, and operator observations relevant to the operation of the technologies will
be documented by technical staff in LRBs or on data sheets. Results from the reference methods,
including raw data, analyses, and final results, will be compiled by the reference laboratory,
preferably in electronic format, and submitted to the verification organization at the conclusion
of reference method testing.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 35 of 42
Version 1.0
Date: 7/29/2008
Records received or generated by any technical staff during the verification test will be
reviewed by a verification organization staff member within two weeks of generation or receipt,
before the records are used to calculate, evaluate, or report verification results. This review will
be performed by a verification organization technical staff member involved in the verification
test, but not the staff member who originally generated the record. The review will be
documented by the person performing the review by adding his/her initials and date to the hard
copy of the record being reviewed. In addition, any calculations performed by technical staff will
be spot-checked by the verification organization QA and/or technical staff to ensure that
calculations are performed correctly. Calculations to be checked include any statistical
calculations described in this protocol. The data obtained from this verification test will be
compiled and reported independently for each technology. Results for technologies from
different vendors will not be compared with each other.
Among the QA activities conducted by verification organization QA staff will be an audit
of data quality. This audit will consist of a review by the verification organization Quality
Manager of at least 10% of the test data. The results of this audit will be compiled in an
assessment report. During the course of any such audit, the verification organization Quality
Manager will inform the technical staff of any findings and any need for immediate corrective
action. If serious data quality problems exist, the verification organization Quality Manager will
request that the verification organization Program Manager issue a stop work order. Once the
assessment report has been prepared, the verification organization will ensure that a response is
provided for each adverse finding or potential problem, and will implement any necessary
follow-up corrective action. The verification organization Quality Manager will ensure that
follow-up corrective action has been taken.
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in verification tests conducted under this protocol to anticipate
and resolve potential problems before the quality of performance is compromised. One of the
major objectives of this protocol is to establish mechanisms necessary to ensure this. Internal QC
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 36 of 42
Version 1.0
Date: 7/29/2008
measures described in this protocol, which is peer reviewed by a panel of outside experts, will be
implemented by the technical staff; these QC measures will give information on data quality on a
day-to-day basis. The responsibility for interpreting the results of these checks and resolving any
potential problems resides with the verification organization. Technical staff have the
responsibility to identify problems that could affect data quality or the ability to use the data.
Technical staff will work with the verification organization Quality Manager to resolve any
problems that are identified. Action will be taken to control the problem, identify a solution to
the problem, minimize losses and correct data, where possible. Independent of any verification
program QA activities, the verification organization will be responsible for ensuring that the
audits described in the following sections are conducted as part of this verification test.
Cl.l Performance Evaluation Audits
A performance evaluation (PE) audit will be conducted to assess the quality of the
reference method measurements made in this verification test. The PE audit of the reference
methods will be performed by supplying each reference method a blind sample or standard
reference material containing the toxins of interest. The PE audit samples will be analyzed in the
same manner as all other samples, and the analytical results for the PE audit samples will be
compared with the nominal concentration or certified value. The target criterion for this PE audit
is agreement of the analytical result within 25% of the nominal concentration [by percent
difference (PD)] or within 25% of the certified value (by PD). If the PE audit results do not meet
the tolerances shown, they will be repeated. If the outlying results persist, a change in reference
instrument and a repeat of the PE audit may be considered. This audit will be performed once
prior to the start of the test and will be the responsibility of the verification organization Quality
Manager or designee.
C1.2 Technical Systems Audits
The verification organization Quality Manager or designee will perform a TSA at least
once during verification tests conducted under this protocol. The purpose of this audit is to
ensure that the verification test is being performed in accordance with the quality plans in place
at the verification organization, this protocol, published reference methods, and any SOPs used
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 37 of 42
Version 1.0
Date: 7/29/2008
by the reference laboratory. In the TSA, the verification organization Quality Manager, or a
designee, may review the reference methods used, compare actual test procedures to those
specified or referenced in this protocol, and review data acquisition and handling procedures. In
the TSA, the verification organization Quality Manager will tour the test facility, observe sample
collection if appropriate, inspect documentation of sample COC, and review technology-specific
records. He or she will also check standard certifications and technology data acquisition
procedures and may confer with the technology vendors, reference laboratory, and technical
staff. The verification organization Quality Manager may also visit the reference laboratory to
review procedures and adherence to this plan and applicable SOPs. A TSA report will be
prepared, including a statement of findings and the actions taken to address any adverse findings.
The verification program Quality Manager will receive a copy of the verification organization's
TSA report. At the verification program's discretion, verification program QA staff may also
conduct an independent on-site TSA during the verification test. The TSA findings will be
communicated to technical staff at the time of the audit and documented in a TSA report.
C1.3 Data Quality Audits
The verification organization Quality Manager or designee will audit at least 10% of the
verification data acquired in the verification test. The verification organization Quality Manager
will trace the data from initial acquisition, through reduction and statistical comparisons, to final
reporting. All calculations performed on the data undergoing the audit will be checked.
C1.4 QA/QC Reporting
Each assessment and audit will be documented and submitted in accordance with the
verification organization's quality management plan. The results of the TSA will be submitted to
the verification program. Assessment reports will include the following:
• Identification of any adverse findings or potential problems
• Response to adverse findings or potential problems
• Recommendations for resolving problems
• Confirmation that solutions have been implemented and are effective
• Citation of any noteworthy practices that may be of use to others.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 3 8 of 42
Version 1.0
Date: 7/29/2008
C2 REPORTS TO MANAGEMENT
The verification organization Quality Manager, during the course of any assessment or
audit, will identify to the technical staff performing experimental activities any immediate
corrective action that should be taken. If serious quality problems exist, the verification
organization Quality Manager is authorized to request that the verification organization Program
Manager issue a stop work order. Once the assessment report has been prepared, the verification
organization will ensure that a response is provided for each adverse finding or potential problem
and will implement any necessary follow-up corrective action. The verification organization's
Quality Manager will ensure that follow-up corrective action has been taken. This protocol, any
TQAPs based on this protocol, and final verification reports are reviewed by the verification
organization's QA staff and the verification organization program management staff. Upon final
review and approval, both documents may be posted on the verification organization's and
verification program's web site, if applicable.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 3 9 of 42
Version 1.0
Date: 7/29/2008
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are stated in Section BIO of
this protocol. In general, the data review requirements specify that the data generated during this
test will be reviewed by a verification organization technical staff member within two weeks of
generation of the data. The reviewer will be familiar with the technical aspects of the
verification test, but will not be the person who generated the data. This process will serve both
as the data review and the data verification and will ensure that the data have been recorded,
transmitted, and processed properly. Furthermore, this process will ensure that the soil toxicity
detecting technology data and the reference method data were collected under appropriate testing
conditions and that the reference method data meet the specifications of the reference method.
The data validation requirements for this test involve an assessment of the data quality
relative to the DQIs and audit acceptance criteria specified for this test. The DQIs listed in
Section B5 will be used to validate the quality of the data. The QA audits described within
Section C of this document, including the PE audit and audit of data quality, are designed to
validate the quality of the data.
D2 VALIDATION AND VERIFICATION METHODS
Data verification is conducted as part of the data review, as described in Section BIO of
this protocol. A visual inspection of handwritten data will be conducted to ensure that all entries
were properly recorded or transcribed and that any erroneous entries were properly noted (i.e.,
single line through the entry with an explanation of the error and the initials of the recorder and
date of entry). Electronic data from the technologies and other instruments used during the test
will be inspected to ensure proper transfer from the datalogging system. Data manually
incorporated into spreadsheets for use in calculations will be checked against handwritten data to
ensure that transcription errors have not occurred. All calculations used to transform the data will
be reviewed to ensure the accuracy and the appropriateness of the calculations. Calculations
performed manually will be reviewed and repeated using a handheld calculator or commercial
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 40 of 42
Version 1.0
Date: 7/29/2008
software (e.g., Excel). Calculations performed using standard commercial office software (e.g.,
Excel) will be reviewed by inspecting the equations used in calculations and verifying selected
calculations by handheld calculator. Calculations performed using specialized commercial
software (i.e., for analytical instrumentation) will be reviewed by inspecting and, when feasible,
verifying by handheld calculator or standard commercial office software.
To ensure that the data generated from this test meet the goals of the test, a number of
data validation procedures will be performed. Section C of this protocol describes the validation
safeguards employed for this verification test. Data validation and verification efforts include the
completion of QC activities and the performance of TSA and PE audits as described in
Section C. The data from this test will be evaluated relative to the measurement DQIs described
in Section B5, and the PE audit acceptance criteria given in Section Cl.l of this protocol. Data
failing to meet these criteria will be flagged in the data set and not used for evaluation of the
technologies, unless these deviations are accompanied by descriptions that adequately
demonstrate that data quality was not compromised.
An audit of data quality will be conducted by the verification organization's Quality
Manager to ensure that data review, verification, and validation procedures were completed and
to assure the overall data quality. The schedule for completing TSA, PE and audits of data
quality are included in Table 1.
D3 RECONCILIATION WITH USER REQUIREMENTS
The purpose of a verification test performed following this protocol is to evaluate the
performance of commercial technologies for detecting toxicity in soil. In part, this evaluation
will include comparisons of the results from the technologies to results from established
analytical reference methods. To meet the requirements of the user community, the data
obtained in such a verification test should include thorough documentation of the performance of
the technologies during the verification test. The data review, verification, and validation
procedures described above will ensure that verification test data meet these requirements and
are accurately presented in the verification reports generated from the test and that data not
meeting these requirements are appropriately flagged and discussed in the verification reports.
Additionally, all data generated using reference methods that are used to evaluate technology
results during the verification test should meet the QA requirements of the reference methods.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 41 of 42
Version 1.0
Date: 7/29/2008
This joint verification protocol and any resulting verification report(s) generated
following procedures described in this protocol will be reviewed by participating technology
vendors, verification organization staff, test collaborators, the verification program(s), and
external expert peer reviewers. These reviews will ensure that this protocol, verification test(s) of
technologies for detecting toxicity in soil, and the resulting report(s) meet the needs of potential
users and regulators. The final report(s) will be submitted to the verification program(s) in
Microsoft Word and in 508 compliant Adobe Portable Document Format (pdf). If applicable, the
final report subsequently may be posted on the verification organization's and verification
program's web site.
-------
Technologies for Rapid Detection of Whole Soil and Soil Extract Toxicity
U.S. EPA ETV and Environment Canada ETV Joint Verification Protocol
Page 42 of 42
Version 1.0
Date: 7/29/2008
SECTION E
REFERENCES
El REFERENCES
1. "ECO Update: Catalogue of Standard Toxicity Tests for Ecological Risk Assessment," EPA
Office of Solid Waste and Emergency Response, March 1994, Publication 9345.0-051.
2. SW-846 Method 8260B, "Volatile Organic Compounds by Gas Chromatography/Mass
Spectrometery (GC/MS)," Revision 2, 1996.
3. EPA Method 531.1, "Measurement of n-Methylcarbamoyloximes and n-Methylcarbamates in
Water by Direct Aqueous Injection HPLC with Post Column Derivatization," Revision 3.1,
1995.
4. SW-846 Method 8141A, "Organophosphorous Compounds by Gas Chromatography:
Capillary Column Technique," Revision 1, September 1994.
5. EPA Method 200.8, "Determination of Trace Elements in Waters and Wastes by Inductively-
Coupled Plasma Mass Spectrometry," Revision 5.4, 1994.
6. SW-846 Method 8270C, "Semi-volatile Organic Compounds by Gas Chromatography/Mass
Spectrometry (GC/MS)," Revision 3, 1996.
7. EPA Method 1613, "Tetra- through Octa-Chlorinated Dioxins and Furans by Isotope
Dilution HRGC/HRMS," Revision B, 1994.
------- |