Balteiie
7/?L' Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Generic Verification Protocol for
Technologies for Rapid Detection of
Soil Toxicity
-------
Generic Verification Protocol
for
Verification of
Technologies for Rapid Detection of Soil Toxicity
April 26, 2007
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 3 of 45
Version 1.0
Date: 4/26/2007
AKNOWLEDGMENTS
This generic verification protocol was developed through a collaboration with the U.S.
Environmental Protection Agency's (EPA's) Office of Solid Waste and Emergency Response and
National Exposure Research Laboratory and includes input from vendors and stakeholders. Peer
reviewers for the protocol were Robert Seyfarth, Kenneth Hill, and Amy Juchatz of the Suffolk
County Department of Health Services' Division of Environmental Quality; Dr. John Hayse, I.
Hlohoskyj, and Leroy Walston of Argonne National Laboratory's Environmental Science
Division; and Dr. Karen Bradham of EPA's National Exposure Research Laboratory. The
contributions to this protocol from the EPA Project Officer Robert Fuerst and Quality Assurance
Manager Elizabeth Betz are also gratefully acknowledged.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 4 of 45
Version 1.0
Date: 4/26/2007
TABLE OF CONTENTS
Section Page
SECTION A PROJECT MANAGEMENT 8
Al VERIFICATION TEST ORGANIZATION 8
A2 BACKGROUND 15
A3 VERIFICATION TEST DESCRIPTION AND SCHEDULE 16
A4 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA 19
A5 SPECIAL TRAINING/CERTIFICATION 20
A6 DOCUMENTATION AND RECORDS 21
SECTION B MEASUREMENT AND DATA ACQUISITION 24
Bl EXPERIMENTAL DESIGN 24
B2 SAMPLING REQUIREMENTS 34
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS 34
B4 LABORATORY REFERENCE METHODS 35
B5 QUALITY CONTROL 35
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE. 36
B7 CALIBRATION/VERIFICATION OF TEST PROCEDURES 36
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES 37
B9 NON-DIRECT MEASUREMENTS 37
BIO DAT A MANAGEMENT 37
SECTION C ASSESSMENT AND OVERSIGHT 39
Cl ASSESSMENTS AND RESPONSE ACTIONS 39
C2 REPORTS TO MANAGEMENT 41
SECTION D DATA VALIDATION AND USABILITY 42
D1 DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS 42
D2 VALIDATION AND VERIFICATION METHODS 42
D3 RECONCILIATION WITH USER REQUIREMENTS 43
SECTION E REFERENCES 45
El REFERENCES 45
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 5 of 45
Version 1.0
Date: 4/26/2007
List of Figures Page
Figure 1. Organization Chart for the Verification Test 9
List of Tables
Table 1. General Verification Test Schedule 18
Table 2. DQIs and Criteria for Critical Measurements for Reference Method 21
TableS. Summary of Data Recording Process 23
Table 4. Categories and Contaminants 25
Table 5. Contaminant Compound Confirmatory Methods 35
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 6 of 45
Version 1.0
Date: 4/26/2007
DISTRIBUTION LIST
U.S. Environmental Protection Agency
Advanced Monitoring Systems (AMS) Center Project Officer
AMS Quality Manager
Battelle AMS Center
AMS Center Manager
Verification Testing Leader
Verification Test Coordinator
Quality Manager
Technical Staff
Peer Reviewers
Vendors of Technologies for Rapid Detection of Soil Toxicity
Reference Laboratory
Test Facility, if applicable
Test Collaborators, if applicable
Subcontractors, if applicable
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 7 of 45
Version 1.0
Date: 4/26/2007
LIST OF ABBREVIATIONS/ACRONYMS
AMS Advanced Monitoring Systems
COA certificate of analysis
COC chain-of-custody
DQI data quality indicator
ECso median effective concentration causing 50% inhibition
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification
LRB laboratory record book
NIST National Institute of Standards and Technology
PCB polychlorinated biphenyl
PD percent difference
pdf Adobe portable document format
PE performance evaluation
QA quality assurance
QC quality control
QCS quality control samples
QMP quality management plan
RSD relative standard deviation
SOP standard operating procedure
TCDD 2,3,7,8-tetrachlorodibenzodioxin
TQAP test/quality assurance plan
TSA technical systems audit
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 8 of 45
Version 1.0
Date: 4/26/2007
SECTION A
PROJECT MANAGEMENT
Al VERIFICATION TEST ORGANIZATION
This protocol provides generic procedures for implementing a verification test for
technologies that rapidly detect soil toxicity. Verification tests are conducted under the auspices
of the U.S. Environmental Protection Agency (EPA) through the Environmental Technology
Verification (ETV) Program. Verification tests of monitoring technologies are coordinated by
Battelle, which manages the ETV Advanced Monitoring Systems (AMS) Center through a
cooperative agreement with EPA. The scope of the AMS Center covers verification of
monitoring technologies for contaminants and natural species in air, water, and soil. In
performing verification tests, Battelle follows the procedures specified in this protocol and
complies with quality requirements in the "Quality Management Plan (QMP) for the ETV
Advanced Monitoring Systems Center" (AMS Center QMP).(1)
Verification tests are performed by Battelle in cooperation with EPA and the vendors
whose technologies for rapid detection of soil toxicity are being verified. These test procedures
may be performed by Battelle, test facility staff, a qualified collaborator, and/or a qualified
subcontractor. The specific staff members who will perform the test procedures are referred to
as "technical staff in this protocol. Each technology vendor is expected to provide Battelle with
their respective technology and will train the technical staff in their technology use.
Quality assurance (QA) oversight will be provided by the Battelle Quality Manager and
also by the EPA AMS Center Quality Manager, at his or her discretion. The organization chart in
Figure 1 identifies the responsibilities of the organizations and individuals associated with the
verification test. Solid lines in the figure indicate direct reporting relationships, dashed lines
indicate indirect reporting relationships. Roles and responsibilities are defined further below.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 9 of 45
Version 1.0
Date: 4/26/2007
Battelle
Management
Battelle AMS
Center Manager
EPA AMS Center
Project Officer
Battelle
Quality Manager
EPA AMS Center
Quality Manager
Battelle
Verification
Testing Leader
Battelle
Verification
Test Coordinator
Technology
Vendors
Reference
Laboratory
Technical Staff
Test Facility
Figure 1. Organization Chart for the Verification Test
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 10 of 45
Version 1.0
Date: 4/26/2007
Al.l Battelle
The AMS Center's Verification Test Coordinator has overall responsibility for ensuring
that the technical, schedule, and cost goals established for the verification test are met.
Specifically, the Verification Test Coordinator will:
• Prepare a draft test/QA plan (TQAP) based on this protocol.
• Prepare draft verification reports and verification statements.
• Establish a budget for the verification test and manage staff to ensure the budget is
not exceeded.
• Revise the draft TQAP, verification reports, and verification statements in response to
reviewer comments.
• Assemble a team of qualified technical staff to conduct the verification test.
• Direct the team in performing the verification test in accordance with this protocol
and any TQAP prepared based on this protocol.
• Hold a kick-off meeting approximately one week prior to the start of the verification
test to review the critical logistical, technical, and administrative aspects of the
verification test. Responsibility for each aspect of the verification test will be
confirmed.
• Ensure that all quality procedures specified in this protocol and in the AMS Center
QMP(1) are followed.
• Serve as the primary point of contact for technology vendors.
• Ensure that confidentiality of sensitive vendor information is maintained.
• Assist vendors as needed during verification testing.
• Become familiar with the operation and maintenance of the technologies through
instruction by the vendors, if needed.
• Respond to any issues raised in assessment reports, audits, or from test staff
observations, and institute corrective action as necessary.
• Coordinate distribution of the final TQAP, verification reports, and verification
statements.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 11 of 45
Version 1.0
Date: 4/26/2007
The Battelle Verification Testing Leader will provide technical guidance and will:
• Support the Verification Test Coordinator in preparing the TQAP and organizing the
testing.
• Review the draft and final TQAP.
• Attend the verification test kick-off meeting.
• Review the draft and final verification reports and verification statements.
The Battelle AMS Center Manager will:
• Review the draft and final TQAP.
• Review the draft and final verification reports and verification statements.
• Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test.
• Ensure that confidentiality of sensitive vendor information is maintained.
• Support the Verification Test Coordinator in responding to any issues raised in
assessment reports and audits.
• Maintain communication with EPA's technical and quality managers.
• Issue a stop work order if Battelle or EPA QA staff discovers adverse findings that
will compromise test results.
Battelle's Quality Manager for the AMS Center will:
• Review the draft and final TQAP.
• Attend the verification test kick-off meeting.
• Conduct a technical systems audit (TSA) once during the verification test, or
designate other QA staff to conduct the audit.
• Audit at least 10% of the verification data or designate other QA staff to conduct the
data audit.
• Prepare and distribute an assessment report for each audit.
• Verify implementation of any necessary corrective action.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 12 of 45
Version 1.0
Date: 4/26/2007
• Request that Battelle's AMS Center Manager issue a stop work order if audits
indicate that data quality is being compromised.
• Provide a summary of the QA/QC activities and results for the verification reports.
• Review the draft and final verification reports and verification statements.
A1.2 Technology Vendors
The responsibilities of the technology vendors are as follows:
• Review and provide comments on the draft TQAP.
• Accept (by signature of a company representative) the final TQAP prior to test
initiation.
• Provide their technology for evaluation during the verification test.
• Provide all other equipment/supplies/reagents/consumables needed to operate their
technology for the duration of the verification test.
• Supply a representative to provide training on their technology use, and provide
written consent and instructions for test staff to carry out verification testing,
including written instructions for routine operation of their technology.
• Provide maintenance and repair support for their technology, on-site if necessary,
throughout the duration of the verification test.
• Review and provide comments on the draft verification report and statement for their
respective technology.
A1.3 EPA
EPA's responsibilities in the AMS Center are based on the requirements stated in the
"Environmental Technology Verification Program Quality Management Plan" (EPA ETV
QMP).(2) The roles of specific EPA staff are as follows:
The EPA's AMS Center Quality Manager will:
• Review the draft TQAP.
• Perform at his or her option one external TSA during the verification test.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 13 of 45
Version 1.0
Date: 4/26/2007
• Notify the EPA AMS Center Project Officer of the need for a stop work order if the
external audit indicates that data quality is being compromised.
• Prepare and distribute an assessment report summarizing results of the external audit.
• Review draft verification reports and verification statements.
The EPA's Project Officer for the AMS Center will:
• Review the draft TQAP.
• Approve the final TQAP.
• Notify the Battelle AMS Center Manager of the need for a stop work order if it is
discovered that data quality is being compromised.
• Review the draft verification reports and verification statements.
• Oversee the EPA review process for the TQAP, verification reports, and verification
statements.
• Coordinate the submission of verification reports and verification statements for final
EPA approval.
A1.4 Technical Staff
The technical staff will support the Verification Test Coordinator in planning and
conducting the verification test. The responsibilities of the technical staff will be to:
• Assist in planning for the test, and making arrangements for the receipt of and
training on the technologies.
• Attend the verification test kick-off meeting.
• Assist vendor staff as needed during technology receipt and training.
• Conduct verification testing using the vendor's technology, if necessary.
• Conduct reference testing where applicable or help coordinate with laboratories
providing reference testing.
• Perform statistical calculations specified in this protocol on the technology data as
needed.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 14 of 45
Version 1.0
Date: 4/26/2007
• Provide results of statistical calculations and associated discussion for the verification
reports as needed.
• Support the Verification Test Coordinator in responding to any issues raised in
assessment reports and audits as needed.
A1.5 Reference Laboratory
When valid reference methods are available, the contaminant concentration of the soil
test samples will be confirmed. When the analyses are within Battelle's capabilities, Battelle
will perform the analyses; however, if it is more cost effective, Battelle may collaborate with
another laboratory or establish a subcontract with a commercial laboratory to perform the
analyses.
In order to be selected to perform the reference analyses during the verification test, a
commercial laboratory will need to provide Battelle documentation that demonstrates its
competence to perform the needed analysis; such documentation may include copies of its
method/standard operating procedure (SOP), QA manual, state government
certifications/approvals for analysis of the appropriate contaminant, and/or staff training records,
where available. If the prospective laboratory does not demonstrate its capability adequately,
another laboratory will be selected and its competence verified in a similar manner.
A1.6 Test Facility
Either Battelle or another appropriate facility such as a laboratory that conducts soil
testing will serve as the test facility. The test facility personnel are expected to:
• Identify a point of contact for the test who will serve as the primary interface with the
Verification Test Coordinator.
• Attend the verification test kick-off meeting.
• Ensure that test facility staff and facilities are ready for the verification test.
• Assist Battelle's Verification Test Coordinator in ensuring that the verification testing
is conducted in accordance with this protocol and any TQAPs that are generated.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 15 of 45
Version 1.0
Date: 4/26/2007
• Assist Battelle and technology vendor staff as needed during technology receipt,
training, testing, and return of equipment to technology vendors.
• Ensure that necessary test facility resources (e.g., space and power) are committed to
the verification test.
A2 BACKGROUND
The ETV Program's AMS Center conducts third-party performance testing of
commercially available technologies that detect or monitor natural species or contaminants in air,
water, and soil. The purpose of ETV is to provide objective and quality-assured performance
data on environmental technologies, so that users, developers, regulators, and consultants can
make informed decisions about purchasing and applying these technologies. Stakeholder
committees of buyers and users of such technologies recommend technology categories, and
technologies within those categories, as priorities for testing. Technologies for rapidly detecting
soil toxicity were identified as a priority technology category through the AMS Center
stakeholder process since they have the potential to make the evaluation of soil toxicity more
efficient and timely.
Soil toxicity testing can be used at hazardous waste sites to screen for particular areas of
concern or to assist in monitoring the effectiveness of cleanup. Soil toxicity tests do not require
knowing the contaminants present at the site; they are typically used as a broad range screen of
all potentially toxic compounds that may be present. Traditional soil toxicity tests include
evaluations such as seed germination and root elongation, as well as organism-based tests such
as earthworm survival.(3) Tests such as these can take several weeks to achieve results. This
protocol provides procedures for a verification test of rapid analysis technologies that detect
toxicity in soil. The objective of this soil toxicity technology verification test is to evaluate the
technology's ability to detect certain analytes that are particularly toxic to humans by adding
them, individually, to a controlled experimental matrix, as well as by testing various "real-world"
soil samples where the toxins may be present alone or with various other toxins. This generic
protocol outlines testing for a number of contaminants that are common to site cleanups and
known to be toxic, but does not include all toxic compounds or testing in all situations which
may be encountered in a site cleanup or evaluation situation. Data generated from verification
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 16 of 45
Version 1.0
Date: 4/26/2007
tests based on this generic protocol are intended to provide one set of objective and quality
assured performance data on environmental technologies, to assist users, developers, regulators,
and consultants in making informed decisions about purchasing and properly applying these
technologies.
This verification test will determine the performance characteristics of commercially
available technologies that can provide results within a 24-hour period to make soil toxicity
determinations more efficient. Critical characteristics of the soil toxicity technologies that will
be assessed during this testing include the following:
• Endpoint
• Precision
• False negative rate
• False positive rate
• Lowest detectable level
• Matrix effects
• Data completeness
• Operational factors such as ease of use and maintenance
• Field portability.
A3 VERIFICATION TEST DESCRIPTION AND SCHEDULE
A3.1 Summary of Technology Category
Conventional soil toxicity methods often can take weeks to achieve results. Technologies
to be evaluated in the verification of rapid soil toxicity technologies include those that produce
results within a 24-hour period. Such rapid soil toxicity technologies have the potential to
expedite the decision-making process for regulators. Technologies applicable to this technology
category can be those designed to directly test the soil, or just the soil extract. Rapid soil toxicity
technologies do not provide a measured concentration of specific toxins; rather, they provide a
broad range screen of the toxic nature of the soil. For example, a soil extract may be added to a
bacteria, bioluminescent plankton, or other such organism or compound producing a measurable
response that varies based on the toxicity of contaminants in the soil. The specific response may
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 17 of 45
Version 1.0
Date: 4/26/2007
vary by technology, but could include a change in color or light intensity, respiration rate, or
other response that is proportional to the concentration level of the contaminant.
A3.2 Verification Test Schedule
A verification test following this protocol should take between ten months and one year
to complete. Test planning and preparation may take place over a period of four to five months.
Actual testing should be completed within one month. Data review and reporting should be
completed within four to five months. Table 1 shows a general schedule of testing and data
analysis/reporting activities to be conducted in a verification test that follows this protocol. The
test procedures are described in Section B of this protocol. Subsequent to testing, a separate
verification report will be drafted for each technology. Each draft report will be peer-reviewed,
revised, and submitted to EPA for final approval. Technologies for detecting soil toxicity and
associated equipment (but not consumables) will be returned to the vendors at the completion of
report writing.
A3.3 Test Facility
The test facility should be a location that can accommodate laboratory testing of
technologies for detecting toxicity in soil. This could be laboratory facilities at Battelle or other
such laboratory facilities that routinely test soil. Field portability testing, if applicable, will be
conducted by transporting the technology from a laboratory to a non-laboratory area. In addition
to a traditional field setting, non-laboratory areas could include warehouses, shipping/receiving
areas, storerooms, courtyards, and/or parking lots.
A3.4 Health and Safety
All reference analyses and verification testing will follow the safety and health protocols
in place for the test facility. This includes maintaining a safe work environment and a current
awareness of handling potentially toxic chemicals. Exposure to potentially toxic chemicals will
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 18 of 45
Version 1.0
Date: 4/26/2007
be minimized, personal protective equipment will be worn, and safe laboratory practices will be
followed.
Table 1. General Verification Test Schedule
Month
1-3
3-4
4-5
6
7
8
9
10
Testing Activities
• Identify test collaborators/
subcontractors and test facility
• Recruit vendors
• Prepare draft TQAP and submit for
vendor and peer reviews
• Revise draft TQAP
• Finalize and obtain vendor
approval of TQAP
• Procure necessary standards and
reagents
• Vendor to set up technology and
train technical staff on technology
use
• Conduct verification tests
• Conduct reference tests and
performance evaluation audit of
reference methods
• Conduct TSA
• Return equipment to vendors
Data Analysis and Reporting
• Review and compile test data and records as
they become available
• Review and summarize verification testing staff
observations
• Begin preparation of report template
• Evaluate and analyze data generated during
testing
• Conduct data quality audits
• Complete report template
• Complete draft reports and submit for vendor
and peer review
• Revise draft reports and submit final reports for
EPA approval
• Distribute finalized, EPA-approved reports
• Post ETV reports and verification statements on
ETV Web site
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 19 of 45
Version 1.0
Date: 4/26/2007
A4 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA
In performing the verification test, Battelle will follow the technical and QA procedures
specified in this protocol and will comply with the data quality requirements in the AMS Center
QMP.(1) The objective of this verification test is to evaluate the performance of soil toxicity
detecting technologies in their ability to measure the presence of toxins in soil under controlled
laboratory conditions. This evaluation will assess the capabilities of the soil toxicity
technologies to detect toxins added to a controlled experimental matrix, as well as their ability to
detect toxins in "real-world" environmental samples. The evaluation will include a comparison
of the soil toxicity technology results to known concentrations of toxins in the test samples that
will be confirmed as described in Section B4. Additionally, this verification test will rely upon
verification testing staff observations to assess other performance characteristics of the
technologies. Below is a discussion of the quality objectives and the criteria for measurement
data that have been established to ensure that the test objectives are met.
A4.1 Quality Objectives
Data quality objectives assure that the data quality, quantity, and type are appropriate to
meet the verification test objectives and specify the minimum acceptance criteria for these
parameters. Data quality objectives for this verification test include those related to the reference
method performance and those related to the soil toxicity detecting technology performance, as
well as those related to documenting verification testing staff observations. Data quality
objectives for the reference methods (see Section B4) are presented in terms of data quality
indicator (DQI) criteria for the critical measurements associated with the reference methods and
are listed in Table 2 and discussed in Section A4.2. The reference method data quality relies, in
part, on proper sample preparation, proper application of the reference method, and proper
maintenance of reference method instrumentation. Battelle will rely on the vendor's data quality
objectives for each technology in order to ensure that the technology is performing properly
during testing. This will include adhering to each vendor's criteria for calibration and
performance of positive and negative control samples. The technology data quality relies on
proper operation and maintenance of the technologies and proper sample preparation, as
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 20 of 45
Version 1.0
Date: 4/26/2007
instructed by the vendor. Quantitative data quality objectives for the operator observations have
not been defined but are incorporated into documentation requirements and data review,
verification, and validation requirements for this verification test.
A4.2 Criteria for Measurement Data
Table 2 presents the DQIs and criteria for the reference method critical measurements.
The reference method measurement quality will be ensured by adhering to these DQI criteria and
monitored by following the calibration procedures and frequency recommended in each
respective reference method and by including method blank or spiked samples as indicated in
each reference method. Additionally, performance test samples will be sent to each laboratory
providing reference method analyses prior to analysis of verification test samples. Performance
test samples will be standard solutions or standard reference materials containing known
quantities of the analytes of interest. Each vendor will provide criteria for the soil toxicity
technologies for critical measurements related to calibration standards and recommendations for
appropriate positive and negative controls and their critical measurements. The Battelle Quality
Manager or his or her designee will perform a TSA at least once during this verification test to
review these QA/quality control (QC) requirements. The EPA AMS Center Quality Manager
also may conduct an independent TSA, at his or her discretion.
A5 SPECIAL TRAINING/CERTIFICATION
Documentation of training related to technology testing, field testing, data analysis, and
reporting is maintained for all Battelle technical staff in training files at their respective Battelle
location. Documentation of the expertise and experience of collaborators and/or subcontractors
must be similarly available. The Battelle Quality Manager may verify the presence of
appropriate training records prior to the start of testing. If technical staff operate and/or maintain
a technology during the verification test, the technology vendor will be required to train those
staff prior to the start of testing. Battelle will document this training with a consent form, signed
by the vendor, that states which specific technical staff have been trained on their technology.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 21 of 45
Version 1.0
Date: 4/26/2007
Battelle technical staff will have a minimum of a bachelor's degree in science/engineering or
equivalent work experience.
A6 DOCUMENTATION AND RECORDS
The records for this verification test will include the TQAP based on this protocol, chain-of-
custody (COC) forms, laboratory record books (LRBs), data collection forms, electronic files
(both raw data and spreadsheets), and the final verification reports and verification statements.
All of these records will be maintained in the Verification Test Coordinator's office or at the test
Table 2. DQIs and Criteria for Critical Measurements for Reference Method
DQI
Bias and Accuracy
of Sample
Measurements
Completeness
Method
Representativeness
Method of
Assessment
Initial Calibration —
various levels as
specified in
reference method
Calibration Check
Sample — single-
level continuing
check of calibration
as specified in
reference method
Method Blank
Spiked Samples
Amount of valid
data obtained
Performance Test
Sample
Frequency
As required in
reference
method
As required in
reference
method
As required in
reference
method
As required in
reference
method
Overall
number of
data points
collected for
reference
method
Once, prior to
verification
testing
Minimum
Acceptance
Criteria
Refer to reference
method criteria
Refer to reference
method criteria
Refer to reference
method criteria
Refer to reference
method criteria
90% of overall data
points collected should
be valid.
Results within ± 10%
of expected value for
standard solutions,
results within certified
limits for standard
reference materials
Corrective Action
Investigate sources of
contamination or changes
in instrument parameters;
perform instrument
maintenance as needed;
reanalyze fresh standard or
sample, or repeat initial
calibration.
If feasible, analyze
additional samples to meet
the acceptance criterion.
Evaluate reference method
performance; perform
maintenance or
recalibration as required,
repeat performance test. If
performance test criteria
cannot be met, consider
alternative reference
laboratories.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 22 of 45
Version 1.0
Date: 4/26/2007
facility during the test and will be transferred to the Verification Test Coordinator's office at the
conclusion of the verification test. The location (e.g., specific personal computer, server, or
media type and storage location) of final versions of the electronic files will be noted in the test
records. All Battelle LRBs are stored indefinitely, either by the Verification Test Coordinator or
Battelle's Records Management Office. EPA will be notified before disposal of any files. The
QA/QC documentation and results of the reference measurements made by the reference
laboratory will be submitted to Battelle immediately upon completion of all sample analyses and
maintained with the records for this test. Table 3 has further details regarding the data recording
practices and responsibilities.
All written records must be in ink. Any corrections to notebook entries, or changes in
recorded data, must be made with a single line through the original entry. The correction is then
to be entered, initialed, and dated by the person making the correction. In all cases, strict
confidentiality of data from each vendor's technology, and strict separation of data from different
vendors' technologies, will be maintained. Separate files (including manual records, printouts,
and/or electronic data files) will be kept for each technology.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 23 of 45
Version 1.0
Date: 4/26/2007
Table 3. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test events,
technology
maintenance,
downtime, etc.
Technology
calibration
information
Technology readings
Sample preparation
and reference
method analysis
procedures,
calibrations, QA, etc.
Reference method
results
Where Recorded
ETV LRBs or data
recording forms
ETV LRBs, data
recording forms, or
electronically
Either recorded
electronically by the
technology and
downloaded to an
independent
computer or storage
medium, hard copy
data printed by the
technology and taped
into an ETV LRB, or
handwritten records
into an ETV LRB or
on data sheets
LRBs, COC, or other
data recording forms
Electronically from
analytical method or
documented in
handwritten records
How Often
Recorded
Start/end of test
procedure, and at
each change of a test
parameter or change
of technology status
At technology
calibration or
recalibration
Every sample
analysis.
Throughout sampling
and analysis
processes
Every sample
analysis
By Whom
Technical staff
Technical staff or
vendor performing
the calibration
Technical staff
Technical staff and
Reference laboratory
Reference laboratory
Disposition of
Data
Used to organize and
check test results;
manually
incorporated in data
spreadsheets as
necessary
Incorporated in
verification report as
necessary
Transferred to or
manually entered
into spreadsheet for
statistical analysis
and comparisons
Retained as
documentation of
reference method
performance
Transferred to or
manually entered
into spreadsheets for
calculation of results,
and statistical
analysis and
comparisons as
needed
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 24 of 45
Version 1.0
Date: 4/26/2007
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
This generic protocol outlines testing for a number of contaminants that are common to
site cleanups and known to be toxic to humans, but does not include all toxic compounds or
testing in all situations which may be encountered in a site cleanup or evaluation situation. Data
generated from verification tests based on this generic protocol are intended to provide one set of
objective and quality assured performance data on environmental technologies, to assist users,
developers, regulators, and consultants in making informed decisions about purchasing and
properly applying these technologies. These technologies do not provide identification or
concentration of specific contaminants, but serve as a rapid screening tool to determine whether
the soil being tested is toxic. As part of this verification test, the technologies will be subjected
to various concentrations of chemicals representing several categories of common contaminants
such as commercial solvents, pesticides, persistent pollutants, and metals. At a minimum, the
contaminants listed in Table 4 should be evaluated during verification testing. Each contaminant
will be added individually to separate aliquots of sand, and the spiked sand will be analyzed by
the technologies. Sand is recommended as the matrix for the spiking experiments because it is
inert and will minimally retain the contaminants of interest thereby providing an estimate of
technology performance in the case where nearly 100% of the contaminant would be
bioavailable. Additionally, "real-world" environmental samples of various soil types containing a
variety of the contaminants in Table 4 will be analyzed to evaluate the technology performance
on samples more representative of those used in practical application of the technologies. These
samples are described in Section Bl.l.
All of the technologies will be tested in a laboratory. The technologies designed for use in
a field location will also be tested at a non-laboratory venue.
The analyses will be performed according to the vendor's recommended procedures as
described in the user's instructions or manual, or during training provided to the technical staff.
Similarly, calibration and maintenance of the technologies will be performed as specified by the
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 25 of 45
Version 1.0
Date: 4/26/2007
vendor. Results from the technologies being verified will be recorded manually by the operator
on appropriate data sheets or captured in an electronic data system and then transferred manually
or electronically for further data workup. Qualitative operational characteristics of each
technology such as ease of use will be assessed through observations made by the Verification
Test Coordinator and technical staff throughout the verification test. The results from each
technology will be reported individually. No direct comparison will be made between
technologies, but each technology will undergo similar testing.
Table 4. Categories and Contaminants
Category
Commercial solvents
Carbamate pesticide
Organophosphate pesticide
Metals
Persistent pollutants
Contaminant
Trichloroethylene
Toluene
Aldicarb
Dicrotophos
Arsenic
Lead
Mercury
Cadmium
Polychlorinated biphenyls (PCBs) (as Aroclor 1254)
2,3,7,8- tetrachlorodibenzodioxin (TCDD)
Benzo[a]pyrene
Bl.l Test Procedures
The verification test for technologies that detect toxicity in soil will focus on a broad
range of samples to provide a variety of toxin concentrations. ETV verifications usually include
a comparison of the results generated by the technologies being verified with the results of
analysis of the same samples using a standard reference method that measures the same
endpoint, usually concentration. In the case of this verification test, the standard methods for soil
toxicity, such as seed germination and root elongation evaluations, as well as organism-based
tests such as earthworm survival, are quite different from those used in rapid soil toxicity
technologies. Sensitivity to contaminants will be different among different test species.
Therefore, the standard soil toxicity tests would not provide an appropriate benchmark for direct
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 26 of 45
Version 1.0
Date: 4/26/2007
comparison with the results produced by the rapid soil toxicity technologies because of the
differing test species. It is well documented that the contaminants used during this verification
test are toxic to humans; the objective of the test is to determine each technology's ability to
detect this toxicity if the contaminant is present. In lieu of a traditional reference measurement of
toxicity, the concentration of each contaminant in the test samples will be confirmed
independently by standard laboratory methods.
The first sample type will be performance test samples where individual toxins will be
added to sand. Sand is recommended as an inert matrix which will minimally retain the toxins.
Use of an inert matrix will eliminate the matrix itself from influencing the lowest detectable
concentration of each contaminant and will evaluate technology performance under conditions
where the toxin is nearly 100% bioavailable. The sand will be spiked with each contaminant at
concentrations ten times screening or remediation goal levels (e.g., EPA Region 9 Superfund
Preliminary Remediation Goals) as the highest concentration and will be analyzed in replicate
(minimum of three). Subsequent tenfold dilutions will be prepared and analyzed in replicate
(minimum of three) until there is no longer a measurable response indicating toxicity (i.e.,
inhibition as measured by each technology such as a reduction in light output, change in
respiration rate, etc.), up to a maximum of five dilutions below the highest concentration. From
these data, the lowest concentration at which the toxicity can be detected, or toxicity threshold,
will be estimated for each technology with respect to each contaminant. The second sample type
will be "real-world" environmental samples and will consist of 5 to 10 soils collected from
various cleanup sites or standard reference soils with well documented soil characteristics.
These samples will reflect a variety of soil types and will include soils known to contain the
contaminants evaluated in this study (individually or in combination with other contaminants) as
well as some soils which are known to be free of contaminants, such as American Society of
Testing Materials artificial soil or Environmental Resource Associates Semivolatile Blank Soil
(Catalog Number 056). The environmental samples will be dried and homogenized prior to use
in testing to ensure that sample homogeneity is not a significant factor in technology
performance. Because the drying and homogenization process has the potential to affect the
concentration of contaminants in the samples, the concentration of contaminants will be
measured after the drying and homogenization process has taken place to ensure that the
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 27 of 45
Version 1.0
Date: 4/26/2007
measured concentrations of contaminants in the environmental samples accurately reflect the
material used in testing. Contaminants in the environmental samples will be measured using the
same reference methods that will be used to confirm the concentration of spiked contaminants in
the performance test samples. Appropriate soil characteristics such as total organic carbon, grain
size distribution, and pH should also be measured once the environmental samples have been
dried and homogenized. Information about the soil characteristics may aid in understanding
differences in the various environmental samples that will be tested. To the extent possible, each
time verification testing is conducted following this generic protocol, attempts should be made to
use the same environmental sites, or at a minimum sites with comparable types of soil and types
and quantities of contaminants. The third type of sample will be quality control samples. Quality
control samples are discussed further in Section B5.
The technologies will be evaluated for the parameters listed in sections Bl.1.1 toBl.1.9.
B1.1.1 Endpoint
Each technology produces its own unique endpoint derived from the inhibition data
gathered when analyzing various concentrations of contaminants in soil [e.g, median effective
concentration causing 50% inhibition (ECso) values]. For each technology, the endpoint used for
verification testing will be recommended by the vendor.
Bl.1.2 Precision
Inhibition results (endpoints) specific to each technology from replicates (minimum of
three) of each test sample will be evaluated. Relative standard deviation (RSD) of the replicate
measurements will be calculated in order to evaluate the precision of the technologies.
Bl. 1.3 False Negative Rate
The false negative rate, or frequency of inhibition similar to the negative control reported
when a contaminant is present at toxic concentrations, will be calculated.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 28 of 45
Version 1.0
Date: 4/26/2007
Bl. 1.4 False Positive Rate
The false positive rate, or frequency of detectable inhibition reported in unspiked
samples, will be calculated.
Bl. 1.5 Lowest Detectable Level
Various contaminants will be added individually to a controlled experimental matrix at
multiple concentration levels and analyzed by the participating technologies to assess their
ability to detect the toxicity of these contaminants. After analyzing several concentrations of
each contaminant, the lowest concentration of each contaminant that causes inhibition
statistically greater than that of the negative control will be reported.
Bl.1.6 Matrix Effects
Various environmental samples representing a variety of soil types and contaminants or
mixtures of contaminants will be analyzed. The concentrations of contaminants present in the
environmental samples will be measured according to reference methods. The technology's
ability to detect contaminants in the environmental samples will be compared with the lowest
detectable level of contaminant determined for each technology to assess whether the
environmental sample matrix influenced the ability of the technology to detect toxicity.
Bl.1.7 Data Completeness
Data completeness will be determined as the number of valid measurements out of the
total number of measurements taken. The cause of any substantial loss of data will be established
from technical staff observations or technology records and noted in the discussion of the data
completeness results.
Bl.1.8 Operational Factors
Operational and sustainability factors such as maintenance needs, calibration frequency,
data output, consumables used, ease of use, repair requirements, waste production, and sample
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 29 of 45
Version 1.0
Date: 4/26/2007
throughput will be evaluated based on technical staff and Verification Test Coordinator
observations. An LRB or data sheets will be used to document observations. Examples of
information to be recorded include the daily status of diagnostic indicators for the technology,
use or replacement of any consumables, the effort or cost associated with maintenance or repair,
vendor effort (e.g., time on-site) for repair or maintenance, the duration and causes of any
technology downtime or data acquisition failure, quantity and hazardous nature of any waste
generated, operator observations about technology ease of use, clarity of the vendor's instruction
manual, user-friendliness of any needed software, overall convenience of the technologies and
accessories/consumables, and the number of samples that could be processed per hour or per day.
These observations will be summarized to aid in describing the technology performance in the
verification report on each technology.
Bl.1.9 Field Portability
Testing the operation of the technologies in a field setting is a key component of the
verification test. Evaluating the performance of each field-portable technology while being used
outside the laboratory without the availability of miscellaneous laboratory supplies is important
to the buyers and users of these technologies. Technologies will be evaluated in a field setting
only if the vendor states that the technology has that capability. For those technologies that are
meant to be field-portable, this parameter will be assessed by transporting the technology to a
non-laboratory location. In addition to traditional field settings, non-laboratory areas could
include warehouses, shipping/receiving areas, storerooms, courtyards, and/or parking lots
provided the location meets the criteria that the area is absent of laboratory amenities such as
laboratory bench space, power, lighting, storage and refrigeration, etc. as would be the case in a
traditional field setting. Ideally all of the samples included in the lab-based tests would be
repeated in the field; however, at a minimum one performance test sample or environmental
sample that had a strong response in the lab-based tests will be analyzed in triplicate in the field.
Results obtained in the field will be compared with the results for the same sample obtained in
the laboratory. Technical staff will also record observations related to field portability such as
requirements for power, space, and ease of use in and transport to a non-laboratory setting.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 30 of 45
Version 1.0
Date: 4/26/2007
B1.2 Statistical Analysis
The statistical methods and calculations used for evaluation of the quantitative
performance parameters are described in the following sections.
B.I.2.1 Endpoint
Each technology produces its own unique endpoint derived from the inhibition data
gathered when analyzing various concentrations of contaminants in soil (e.g., ECso values). For
each technology, these data will be documented and presented with respect to each contaminant
and concentration level using the appropriate endpoint for the technology.
Bl.2.2 Precision
The standard deviation (S) of the results for the replicate analyses of the same sample will
be calculated as follows.
S =
k=l
1/2
(1)
where n is the number of replicate samples, A-4 is the endpoint measurement for the kih sample,
and Mis the average endpoint measurement of the replicate samples. The technology precision
for each sample will be reported in terms of RSD, which will be calculated as follows.
RSD(%) =
M
x 100
(2)
The RSD values for each analyte at each concentration will be listed in data tables in the
verification report; however, verification statements and performance summary tables in the
verification report will list the range of RSDs obtained for all concentrations of each contaminant
tested.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 31 of 45
Version 1.0
Date: 4/26/2007
B1.2.3 False Negative Rate
Results will be considered false negative only when a technology is exposed to a
contaminant concentration greater than the desired remediation or screening level and the
technology does not indicate inhibition greater than the negative control. The rate of false
negatives, expressed as a percentage of total samples analyzed for each contaminant, will be
calculated by dividing the number of false negative measurements (Mfn) by the total number of
measurements included in verification testing (Mfoto/).
Mfn f~, \
FalseNegative(%) = —— x 100 (3)
Mtotal
B1.2.4 False Positive Rate
Results will be considered false positive only when an unspiked sample produces
inhibition greater than that of the negative control. The rate of false positives, expressed as a
percentage of total samples analyzed for each contaminant, will be calculated by dividing the
number of false positive measurements (Mfr) by the total number of measurements included in
verification testing (Mfoto/).
n fA\
FalsePositive(%) = x 100 (4)
Mtotal
Bl.2.5 Lowest Detectable Level
The lowest detectable level will be the concentration of contaminant in performance test
samples (i.e., sand spiked with contaminant) that causes an inhibition greater than the average
inhibition of the negative control, which will be calculated using all negative control
measurements made during the test. Performance test sample results that are above the average
negative control plus the standard deviation about the average will be considered detectable. The
lowest performance test sample that results in a detectable inhibition calculated in this manner
will be considered the lowest detectable concentration.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 32 of 45
Version 1.0
Date: 4/26/2007
Bl.2.6 Matrix Effects
The technology's ability to detect each contaminant in the environmental samples will be
compared with the technology's lowest detectable contaminant level determined by spiking
contaminant into an inert matrix (i.e., sand) as described in Section Bl.2.5. If the contaminant
concentration in the environmental sample (measured using reference methods described in
Section B4) is above the lowest detectable level in an inert matrix (as determined in Section
Bl.2.5), but the technology result for the environmental sample is negative, matrix effects will be
considered to have contributed to this false negative response. The number of environmental
samples where matrix effects affected results (Mmatrix) out of the total number of environmental
samples tested (Mtotaj) will be reported as a percentage using Equation 5.
matrix ic\
MatrixEffect(%) = x 100 (5)
Mtotal
Bl.2.7 Data Completeness
Data completeness will be calculated as the percentage of the total possible data by
dividing the number of valid data measurements generated by each technology (Mvaiid) by the
total number of data measurements included in verification testing (Mfoto/).
IVlvalid f£\
Completeness^} = x 100 (6)
Mtotal
The cause of any substantial loss of data will be established from operator observations or
technology records and noted in the discussion of the data completeness results.
Bl.2.8 Operational Factors
There are no statistical calculations applicable to operational factors. Operational factors
will be determined based on documented observations of the technical staff and the Verification
Test Coordinator.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 3 3 of 45
Version 1.0
Date: 4/26/2007
Bl.2.9 Field Portability
The results obtained from the measurements made on samples in the laboratory and field
setting will be compiled independently for each technology and compared to assess the accuracy
of the measurements under the different analysis conditions. Means and standard deviations of
the endpoints generated in both locations will be compared and assessed for whether they are
statistically different.
B1.3 Reporting
The data obtained in the verification test will be compiled separately for each vendor's
technology, and the data evaluations will be applied to each technology's data set without
reference to any other. At no time will data from different vendors' technologies be
intercompared or ranked. Following completion of the data evaluations, a draft verification
report and verification statement will be prepared for each vendor's technology, stating the
verification test procedures and documenting the performance observed. For example,
descriptions of the data acquisition procedures, use of vendor-supplied proprietary software,
consumables used, repairs and maintenance needed, and the nature of any problems will be
presented in the draft report. Each report will briefly describe the ETV program, the AMS
Center, and the procedures used in verification testing. The results of the verification test will be
stated quantitatively, without comparison to any other technology tested or comment on the
acceptability of the technology's performance. Each draft verification report will be submitted
for review by the respective technology vendor and by EPA and other peer reviewers.
Comments on the draft report will be addressed in revisions of the report. The peer review
comments and responses will be tabulated to document the peer review process. The reporting
and review process will be conducted according to the requirements of the AMS Center QMP.(1)
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 34 of 45
Version 1.0
Date: 4/26/2007
B2 SAMPLING REQUIREMENTS
B2.1 Sample Collection, Storage, and Shipment
Environmental samples will be collected for use in a verification test following this
protocol. As much as possible, samples will be obtained from known contaminated sites using
the same sampling techniques that are in place at the site for the site evaluation process. Samples
may be collected in bulk and shipped to the test facility in plastic buckets or other suitable
containers. Shipments will be via a trackable overnight delivery service to the test facility
sample custodian. Samples will be stored refrigerated or frozen as is appropriate for the
contaminants expected to be contained in the soil. Environmental samples will be dried and
homogenized prior to use in testing to ensure that sample heterogeneity is a minimal factor in
testing multiple technologies. Because of the sample handling involved, the environmental
samples will be homogenized before concentrations of contaminants are measured using the
reference methods. Appropriate soil characteristics such as total organic carbon, grain size
distribution, and pH should be measured once the environmental samples have been dried and
homogenized. Information about the soil characteristics may aid in understanding differences in
the various environmental samples that will be tested.
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Sample custody will be documented throughout collection, transport, shipping (if
necessary), and analysis using standard COC forms provided by Battelle or supplied by the
reference laboratory, as appropriate. Samples transferred within Battelle may be documented in
bound sample login LRBs. Each COC form will summarize the samples collected and analyses
requested. The COC forms will track sample release from the sampling location to the test
facility and/or reference laboratory; or release directly from the test facility to the reference
laboratory. Each COC form will be signed by the person relinquishing the samples once that
person has verified that the COC form is accurate. The original sample COC forms will
accompany the samples; the shipper will keep a copy. Upon receipt at the test facility and/or
reference laboratory, COC forms will be signed by the person receiving the samples once that
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 35 of 45
Version 1.0
Date: 4/26/2007
person has verified that all samples identified on the COC forms are present. Any discrepancies
will be noted on the form; and the sample receiver will immediately contact the Verification Test
Coordinator to report missing, broken, or compromised samples. Copies of all COC forms will
be delivered to the Verification Test Coordinator and maintained with the test records.
B4 LABORATORY REFERENCE METHODS
Table 5 lists the methods that can be used to verify the concentration of contaminants
analyzed during verification tests performed following this protocol. Additional methods may be
used provided they are appropriate for the contaminant and matrix and are documented in the
TQAP for the verification test.
Table 5. Contaminant Compound Confirmatory Methods
Chemical
Trichloroethylene, toluene
Aldicarb
Dicrotophos
Arsenic, lead, mercury, cadmium
PCBs (as Aroclor 1254)
2,3,7,8-TCDD
Benzo[a]pyrene
Method
SW-846 8260B(4)
EPA531.1(5)
SW-846 8141A(6)
EPA 200.8(7)
SW-846 8270C(8)
EPA 1613B(9)
SW-846 8270C(8)
B5 QUALITY CONTROL
Steps will be taken to maintain the quality of data collected during verification tests
conducted under this protocol. This will include analyzing specific quality control samples
(QCS) at a regular frequency by the technologies undergoing verification. The QCSs will
include negative controls, positive controls, and calibration checks. Negative control samples,
consisting of unspiked experimental matrix, will help ensure that no sources of contamination are
introduced in the sample handling and analysis procedures. The positive control and calibration
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 36 of 45
Version 1.0
Date: 4/26/2007
check samples, specified by each vendor, will indicate to the technical staff whether or not the
technology is functioning properly. The vendor will provide the approximate endpoint that
should result with their technology upon analysis of the positive control and calibration check.
QCSs producing results that do not meet the anticipated results specified by the vendor will be
reanalyzed and corrective action taken if needed to ensure that test sample results are not
affected. Corrective actions may include reanalyzing samples to verify that the technology has
been operated properly, conducting maintenance, or recalibrating. Positive and negative controls
will be analyzed at a frequency of approximately 5% based on the total number of test samples.
Calibration checks will be analyzed according to guidance provided by each technology vendor.
As described in Section B4, the reference laboratory will follow standard reference
methods for determining the toxins evaluated during verification tests conducted under this
protocol. All reference measurements will be expected to meet the reference method QC
requirements (such as those listed in Table 2) or, in absence of specific requirements in the
reference method, the reference laboratory's standard requirements for QC samples.
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The equipment used by the test facility and/or reference laboratory will be tested,
inspected, and maintained as per the SOPs of the test facility and/or reference laboratory and/or
the manufacturer's recommendations so as to meet the performance requirements established in
this document. When technical staff operate and maintain technologies undergoing testing, they
will follow directions provided by the technology vendor. Otherwise, operation and maintenance
of the technologies will be the responsibility of the technology vendor.
B7 CALIBRATION/VERIFICATION OF TEST PROCEDURES
Systems used for reference analyses will be calibrated as appropriate before any reference
samples are analyzed and recalibrated as needed based on the reference methods and/or reference
laboratory SOPs.
Technologies undergoing testing will be calibrated initially by the respective technology
vendor prior to shipping the technology to the test facility, or during training, and will be
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 37 of 45
Version 1.0
Date: 4/26/2007
recalibrated according to direction from the vendor. Calibration checks will be performed upon
direction of the vendor. In the event that recalibration is necessary, the recalibration will be
carried out by the technology vendor or by technical staff under the direction of the vendor. All
calibrations will be documented as appropriate by the technical staff or vendor.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
All materials, supplies, and consumables will be ordered by the Verification Test
Coordinator or designee. Where possible, Battelle will rely on sources of materials and
consumables that have been used previously as part of ETV testing without problems. Battelle
will also rely on previous experience or recommendations from EPA advisors, stakeholders, test
collaborators, subcontractors, or technology vendors. Where possible, materials or supplies will
be traceable to the National Institute of Standards and Technology (NIST). Upon receipt of any
supplies or consumables, the Verification Test Coordinator or designee will visually inspect and
ensure that the materials received are those that were ordered and that there are no visual signs of
damage that could compromise the suitability of the materials. Certificates of analysis (COA) or
other documentation of analytical purity will be checked for all reagents and standards to ensure
suitability for the verification test and will be included with the test files. If damaged,
unsuitable, or inappropriate goods are received, they will be returned or disposed of, and
arrangements will be made to receive replacement materials.
B9 NON-DIRECT MEASUREMENTS
No non-direct measurements will be used during this verification test.
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by
Battelle, vendor, collaborator, and/or subcontractor staff during the verification test. Table 3
summarizes the types of data to be recorded. All maintenance activities, repairs, calibrations, and
operator observations relevant to the operation of the technologies will be documented by
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 3 8 of 45
Version 1.0
Date: 4/26/2007
technical staff in LRBs or on data sheets. Results from the reference methods, including raw
data, analyses, and final results, will be compiled by the reference laboratory, preferably in
electronic format, and submitted to Battelle at the conclusion of reference method testing.
Records received or generated by any technical staff during the verification test will be
reviewed by a Battelle staff member within two weeks of generation or receipt, before the
records are used to calculate, evaluate, or report verification results. If a Battelle staff member
generated the record, this review will be performed by a Battelle technical staff member involved
in the verification test, but not the staff member who originally generated the record. The review
will be documented by the person performing the review by adding his/her initials and date to the
hard copy of the record being reviewed. In addition, any calculations performed by technical
staff will be spot-checked by Battelle QA and/or technical staff to ensure that calculations are
performed correctly. Calculations to be checked include any statistical calculations described in
this protocol. The data obtained from this verification test will be compiled and reported
independently for each technology. Results for technologies from different vendors will not be
compared with each other.
Among the QA activities conducted by Battelle QA staff will be an audit of data quality.
This audit will consist of a review by the Battelle Quality Manager of at least 10% of the test
data. The results of this audit will be compiled in an assessment report. During the course of any
such audit, the Battelle Quality Manager will inform the technical staff of any findings and any
need for immediate corrective action. If serious data quality problems exist, the Battelle Quality
Manager will request that Battelle's AMS Center Manager issue a stop work order. Once the
assessment report has been prepared, the Verification Test Coordinator will ensure that a
response is provided for each adverse finding or potential problem, and will implement any
necessary follow-up corrective action. The Battelle Quality Manager will ensure that follow-up
corrective action has been taken.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 3 9 of 45
Version 1.0
Date: 4/26/2007
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in verification tests conducted under this protocol to anticipate
and resolve potential problems before the quality of performance is compromised. One of the
major objectives of this protocol is to establish mechanisms necessary to ensure this. Internal QC
measures described in this protocol, which is peer reviewed by a panel of outside experts, will be
implemented by the technical staff and monitored by the Verification Test Coordinator; these QC
measures will give information on data quality on a day-to-day basis. The responsibility for
interpreting the results of these checks and resolving any potential problems resides with the
Verification Test Coordinator. Technical staff have the responsibility to identify problems that
could affect data quality or the ability to use the data. Any problems that are identified will be
reported to the Verification Test Coordinator, who will work with the Battelle Quality Manager
to resolve any issues. Action will be taken to control the problem, identify a solution to the
problem, minimize losses and correct data, where possible. Independent of any EPA QA
activities, Battelle will be responsible for ensuring that the audits described in the following
sections are conducted as part of this verification test.
Cl.l Performance Evaluation Audits
A performance evaluation (PE) audit will be conducted to assess the quality of the
reference method measurements made in this verification test. The PE audit of the reference
methods will be performed by supplying each reference method a blind sample or standard
reference material containing the toxins of interest. The PE audit samples will be analyzed in the
same manner as all other samples, and the analytical results for the PE audit samples will be
compared with the nominal concentration or certified value. The target criterion for this PE audit
is agreement of the analytical result within 25% of the nominal concentration [by percent
difference (PD)] or within 25% of the certified value (by PD). If the PE audit results do not meet
the tolerances shown, they will be repeated. If the outlying results persist, a change in reference
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 40 of 45
Version 1.0
Date: 4/26/2007
instrument and a repeat of the PE audit may be considered. This audit will be performed once
prior to the start of the test and will be the responsibility of the Verification Test Coordinator or
designee.
C1.2 Technical Systems Audits
The Battelle Quality Manager or designee will perform a TSA at least once during
verification tests conducted under this protocol. The purpose of this audit is to ensure that the
verification test is being performed in accordance with the AMS Center QMP(1), this protocol,
published reference methods, and any SOPs used by the reference laboratory. In the TSA, the
Battelle Quality Manager, or a designee, may review the reference methods used, compare actual
test procedures to those specified or referenced in this protocol, and review data acquisition and
handling procedures. In the TSA, the Battelle Quality Manager will tour the test facility, observe
sample collection if appropriate, inspect documentation of sample COC, and review technology-
specific records. He or she will also check standard certifications and technology data acquisition
procedures and may confer with the technology vendors, reference laboratory, and technical
staff. The Battelle Quality Manager may also visit the reference laboratory to review procedures
and adherence to this plan and applicable SOPs. A TSA report will be prepared, including a
statement of findings and the actions taken to address any adverse findings. The EPA AMS
Center Quality Manager will receive a copy of Battelle's TSA report. At EPA's discretion, EPA
QA staff may also conduct an independent on-site TSA during the verification test. The TSA
findings will be communicated to technical staff at the time of the audit and documented in a
TSA report.
C1.3 Data Quality Audits
The Battelle Quality Manager or designee will audit at least 10% of the verification data
acquired in the verification test. The Battelle Quality Manager will trace the data from initial
acquisition, through reduction and statistical comparisons, to final reporting. All calculations
performed on the data undergoing the audit will be checked.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 41 of 45
Version 1.0
Date: 4/26/2007
C1.4 QA/QC Reporting
Each assessment and audit will be documented and submitted in accordance with
Sections 3.3.4 and 3.3.5 of the AMS Center QMP.(1) The results of the ISA will be submitted to
EPA. Assessment reports will include the following:
• Identification of any adverse findings or potential problems
• Response to adverse findings or potential problems
• Recommendations for resolving problems
• Confirmation that solutions have been implemented and are effective
• Citation of any noteworthy practices that may be of use to others.
C2 REPORTS TO MANAGEMENT
The Battelle Quality Manager, during the course of any assessment or audit, will identify
to the technical staff performing experimental activities any immediate corrective action that
should be taken. If serious quality problems exist, the Battelle Quality Manager is authorized to
request that Battelle's AMS Center Manager issue a stop work order. Once the assessment report
has been prepared, the Verification Test Coordinator will ensure that a response is provided for
each adverse finding or potential problem and will implement any necessary follow-up corrective
action. The Battelle Quality Manager will ensure that follow-up corrective action has been taken.
This protocol, any TQAPs based on this protocol, and final verification reports are reviewed by
EPA AMS Center QA staff and EPA AMS Center program management staff. Upon final review
and approval, both documents will be posted on the ETV Web site (www.epa.gov/etv).
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 42 of 45
Version 1.0
Date: 4/26/2007
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are stated in Section BIO of
this protocol. In general, the data review requirements specify that the data generated during this
test will be reviewed by a Battelle technical staff member within two weeks of generation of the
data. The reviewer will be familiar with the technical aspects of the verification test, but will not
be the person who generated the data. This process will serve both as the data review and the
data verification and will ensure that the data have been recorded, transmitted, and processed
properly. Furthermore, this process will ensure that the soil toxicity detecting technology data
and the reference method data were collected under appropriate testing conditions and that the
reference method data meet the specifications of the reference method.
The data validation requirements for this test involve an assessment of the data quality
relative to the DQIs and audit acceptance criteria specified for this test. The DQIs listed in
Section B5 will be used to validate the quality of the data. The QA audits described within
Section C of this document, including the PE audit and audit of data quality, are designed to
validate the quality of the data.
D2 VALIDATION AND VERIFICATION METHODS
Data verification is conducted as part of the data review, as described in Section BIO of
this protocol. A visual inspection of handwritten data will be conducted to ensure that all entries
were properly recorded or transcribed and that any erroneous entries were properly noted (i.e.,
single line through the entry with an explanation of the error and the initials of the recorder and
date of entry). Electronic data from the technologies and other instruments used during the test
will be inspected to ensure proper transfer from the datalogging system. Data manually
incorporated into spreadsheets for use in calculations will be checked against handwritten data to
ensure that transcription errors have not occurred. All calculations used to transform the data will
be reviewed to ensure the accuracy and the appropriateness of the calculations. Calculations
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 43 of 45
Version 1.0
Date: 4/26/2007
performed manually will be reviewed and repeated using a handheld calculator or commercial
software (e.g., Excel). Calculations performed using standard commercial office software (e.g.,
Excel) will be reviewed by inspecting the equations used in calculations and verifying selected
calculations by handheld calculator. Calculations performed using specialized commercial
software (i.e., for analytical instrumentation) will be reviewed by inspecting and, when feasible,
verifying by handheld calculator or standard commercial office software.
To ensure that the data generated from this test meet the goals of the test, a number of
data validation procedures will be performed. Section C of this protocol describes the validation
safeguards employed for this verification test. Data validation and verification efforts include the
completion of QC activities and the performance of ISA and PE audits as described in
Section C. The data from this test will be evaluated relative to the measurement DQIs described
in Section B5, and the PE audit acceptance criteria given in Section Cl.l of this protocol. Data
failing to meet these criteria will be flagged in the data set and not used for evaluation of the
technologies, unless these deviations are accompanied by descriptions that adequately
demonstrate that data quality was not compromised.
An audit of data quality will be conducted by the Battelle Quality Manager to ensure that
data review, verification, and validation procedures were completed and to assure the overall
data quality. The schedule for completing TSA, PE and audits of data quality are included in
Table 1.
D3 RECONCILIATION WITH USER REQUIREMENTS
The purpose of a verification test performed following this protocol is to evaluate the
performance of commercial technologies for detecting toxicity in soil. In part, this evaluation
will include comparisons of the results from the technologies to results from established
reference methods. To meet the requirements of the user community, the data obtained in such a
verification test should include thorough documentation of the performance of the technologies
during the verification test. The data review, verification, and validation procedures described
above will ensure that verification test data meet these requirements and are accurately presented
in the verification reports generated from the test and that data not meeting these requirements
are appropriately flagged and discussed in the verification reports. Additionally, all data
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 44 of 45
Version 1.0
Date: 4/26/2007
generated using reference methods that are used to evaluate technology results during the
verification test should meet the QA requirements of the reference methods.
This generic verification protocol and any resulting ETV verification report(s) generated
following procedures described in this protocol will be reviewed by participating technology
vendors, ETV AMS Center staff, test collaborators, EPA, and external expert peer reviewers.
These reviews will ensure that this protocol, verification test(s) of technologies for detecting
toxicity in soil, and the resulting report(s) meet the needs of potential users and regulators. The
final report(s) will be submitted to EPA in Microsoft Word and in 508 compliant Adobe Portable
Document Format (pdf) and subsequently posted on the ETV Web site.
-------
Technologies for Rapid Detection of Soil Toxicity
Generic Verification Protocol
Page 45 of 45
Version 1.0
Date: 4/26/2007
SECTION E
REFERENCES
El REFERENCES
1. "Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center"
(AMS Center QMP), U.S. EPA Environmental Technology Verification Program, prepared
by Battelle, Columbus, Ohio, Version 6.0, November 2005.
2. "Environmental Technology Verification Program Quality Management Plan" (EPA ETV
QMP), December 2002, EPA/600/R-03/021.
3. "ECO Update: Catalogue of Standard Toxicity Tests for Ecological Risk Assessment," EPA
Office of Solid Waste and Emergency Response, March 1994, Publication 9345.0-051.
4. SW-846 Method 8260B, "Volatile Organic Compounds by Gas Chromatography/Mass
Spectrometery (GC/MS)," Revision 2, 1996.
5. EPA Method 531.1, "Measurement of n-Methylcarbamoyloximes and n-Methylcarbamates in
Water by Direct Aqueous Injection HPLC with Post Column Derivatization," Revision 3.1,
1995.
6. SW-846 Method 8141A, "Organophosphorous Compounds by Gas Chromatography:
Capillary Column Technique," Revision 1, September 1994.
7. EPA Method 200.8, "Determination of Trace Elements in Waters and Wastes by Inductively-
Coupled Plasma Mass Spectrometry," Revision 5.4, 1994.
8. SW-846 Method 8270C, "Semi-volatile Organic Compounds by Gas Chromatography/Mass
Spectrometry (GC/MS)," Revision 3, 1996.
9. EPA Method 1613, "Tetra- through Octa-Chlorinated Dioxins and Furans by Isotope
Dilution HRGC/HRMS," Revision B, 1994.
------- |