Batteiie
"Inc Business L?/ Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Test/QA Plan for Verification of
Ballast Water Exchange
Screening Tools
ET V ETV ET
-------
Verification of
Ballast Water Exchange Screening Tools
January 30, 2007
Prepared by
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 3 of 44
Version 1.0
Date: 1/30/2007
A2 TABLE OF CONTENTS
Section Page
A PROJECT MANAGEMENT
Al Title Page 1
A2 Table of Contents 3
A3 Distribution List 6
A4 Verification Test Organization 7
A5 Background 12
A6 Verification Test Description and Schedule 14
A7 Quality Objectives and Criteria for Measurement Data 15
A8 Special Training/Certification 17
A9 Documentation and Records 20
B MEASUREMENT AND DATA ACQUISITION
Bl Experimental Design 22
B2 Sampling Requirements 31
B3 Sample Handling and Custody Requirements 32
B4 Laboratory Reference Method 33
B5 Quality Control 34
B6 Instrument/Equipment Testing, Inspection, and Maintenance 35
B7 Calibration/Verification of Test Procedures 35
B8 Inspection/Acceptance of Supplies and Consumables 36
B9 Non-Direct Measurements 37
BIO Data Management 37
C AS SES SMENT AND OVERSIGHT
Cl Assessments and Response Actions 38
C2 Reports to Management 40
D DATA VALIDATION AND USABILITY
Dl Data Review, Validation, and Verification Requirements 41
D2 Validation and Verification Methods 41
D3 Reconciliation with User Requirements 42
E REFERENCES 44
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 4 of 44
Version 1.0
Date: 1/30/2007
List of Figures Page
Figure 1. Organization Chart for the Verification Test 8
List of Tables
Table 1. Planned Verification Test Schedule 15
Table 2. DQIs and Criteria for Critical Measurements for Reference Method 18
Table 3. Summary of Data Recording Process 21
Table 4. Proposed Environmental Samples 24
Table 5. Verification Test Samples 25
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 5 of 44
Version 1.0
Date: 1/30/2007
ETV Advanced Monitoring Systems Center
Test/QA Plan for Verification of
Ballast Water Exchange Screening Tools
January 30, 2007
APPROVAL:
Name
Company
Date
-------
A3 DISTRIBUTION LIST
Elizabeth A. Betz
U.S. Environmental Protection Agency -
HEASD
National Exposure Research Laboratory
E205-01 EPA Mailroom
Research Triangle Park, NC 27711
Robert Fuerst
U.S. Environmental Protection Agency -
HEASD
National Exposure Research Laboratory
D205-05 EPA Mailroom
Research Triangle Park, NC 27711
Gail Roderick
NTS Program
USCG R&D Center
1082 Shennecossett Road
Groton, CT 06340
Travis Martin
Dakota Technologies, Inc
2201 A12StN
Fargo, ND 58102
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 6 of 44
Version 1.0
Date: 1/30/2007
Jeanine Boyle
Rosanna Buhl
Mark Curran
Amy Dindal
Carlton Hunt
Clare Larson
Mary Schrock
Zachary Willenberg
Battelle
505 King Ave.
Columbus, OH 43201
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 7 of 44
Version 1.0
Date: 1/30/2007
SECTION A
PROJECT MANAGEMENT
A4 VERIFICATION TEST ORGANIZATION
The verification test will be conducted under the auspices of the U.S. Environmental
Protection Agency (EPA) through the Environmental Technology Verification (ETV) Program.
It will be performed by Battelle, which is managing the ETV Advanced Monitoring Systems
(AMS) Center through a cooperative agreement with EPA. The scope of the AMS Center covers
verification of monitoring technologies for contaminants and natural species in air, water, and
soil.
The day to day operations of this verification test will be coordinated and supervised by
Battelle, with the participation of the vendors who will be having the performance of their
technologies for screening ballast water exchange verified. Testing will be conducted at Battelle
in Columbus, Ohio. Each vendor will provide Battelle with their respective technology and will
train the Battelle staff in their technology use. Battelle staff will operate the technologies during
verification testing.
The organization chart in Figure 1 identifies the responsibilities of the organizations and
individuals associated with the verification test. Roles and responsibilities are defined further
below. Quality Assurance (QA) oversight will be provided by the Battelle Quality Manager and
also by the EPA AMS Center Quality Manager, at her discretion.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 8 of 44
Version 1.0
Date: 1/30/2007
Battelle
Management
Amy Dindal
Battelle AMS
Center Manager
Zachary Willenberg
Battelle
Quality Manager
Dr. Carlton Hunt
Verification
Testing Leader
Mary Schrock
Battelle
Verification
Test Coordinator
Battelle
Technical Staff
Robert Fuerst
EPA AMS Center
Project Officer
!
Elizabeth Betz
EPA AMS Center
Quality Manager
Ballast Water Exchange
Screening
Technology
Vendor
Representatives
Figure 1. Organization Chart for the Verification Test
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 9 of 44
Version 1.0
Date: 1/30/2007
A4.1 Battelle
Ms. Mary Schrock is the AMS Center's Verification Test Coordinator for this test. In this
role, Ms. Schrock will have overall responsibility for ensuring that the technical, schedule, and
cost goals established for the verification test are met. Specifically, Ms. Schrock will:
• Prepare the draft test/QA plan, verification reports, and verification statements.
• Establish a budget for the verification test and manage staff to ensure the budget is
not exceeded.
• Revise the draft test/QA plan, verification reports, and verification statements in
response to reviewers' comments.
• Assemble a team of qualified technical staff to conduct the verification test.
• Direct the team in performing the verification test in accordance with this test/QA
plan.
• Hold a kick-off meeting approximately one week prior to the start of the verification
test to review the critical logistical, technical, and administrative aspects of the
verification test. Responsibility for each aspect of the verification test will be
confirmed.
• Ensure that all quality procedures specified in this test/QA plan and in the AMS
Center Quality Management Plan1 (QMP) are followed.
• Serve as the primary point of contact for vendor representatives.
• Ensure that confidentiality of sensitive vendor information is maintained.
• Assist vendors as needed during verification testing.
• Become familiar with the operation and maintenance of the technologies through
instruction by the vendors, if needed.
• Respond to any issues raised in assessment reports, audits, or from test staff
observations, and institute corrective action as necessary.
• Coordinate distribution of the final test/QA plan, verification reports, and verification
statements.
Dr. Carlton Hunt will serve as Verification Testing Leader. Dr. Hunt will:
• Support Ms. Schrock in preparing the test/QA plan and designing the testing.
• Review the draft and final test/QA plan.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 10 of 44
Version 1.0
Date: 1/30/2007
• Attend the verification test kick-off meeting.
• Review the draft and final verification reports and verification statements.
Ms. Amy Dindal is Battelle's Manager for the AMS Center. As such, Ms. Dindal will
oversee the various stages of verification testing. Ms. Dindal will:
• Review the draft and final test/QA plan.
• Attend the verification test kick-off meeting.
• Review the draft and final verification reports and verification statements.
• Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test.
• Ensure that confidentiality of sensitive vendor information is maintained.
• Support Ms. Schrock in responding to any issues raised in assessment reports and
audits.
• Maintain communication with EPA's technical and quality managers.
• Issue a stop work order if Battelle or EPA QA staff discovers adverse findings that
will compromise test results.
Technical staff from Battelle will support Ms. Schrock in planning and conducting the
verification test. The responsibilities of the technical staff will be to:
• Assist in planning for the test, and making arrangements for the receipt of and
training on the technologies.
• Attend the verification test kick-off meeting.
• Assist vendor staff as needed during technology receipt and training.
• Conduct verification testing using the vendor's technology, if necessary.
• Conduct reference testing.
• Perform statistical calculations specified in this test/QA plan on the technology data
as needed.
• Provide results of statistical calculations and associated discussion for the verification
reports as needed.
• Support Ms. Schrock in responding to any issues raised in assessment reports and
audits related to statistics and data reduction as needed.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 11 of 44
Version 1.0
Date: 1/30/2007
Mr. Zachary Willenberg is Battelle's Quality Manager for the AMS Center.
Mr. Willenberg will:
• Review the draft and final test/QA plan.
• Attend the verification test kick-off meeting.
• Conduct a technical systems audit at least once during the verification test, or
designate other QA staff to conduct the audit.
• Audit at least 10% of the verification data or designate other QA staff to conduct the
data audit.
• Prepare and distribute an assessment report for each audit.
• Verify implementation of any necessary corrective action.
• Request that Battelle's AMS Center Manager issue a stop work order if audits
indicate that data quality is being compromised.
• Provide a summary of the QA/QC activities and results for the verification reports.
• Review the draft and final verification reports and verification statements.
A4.2 Technology Vendors
The responsibilities of the technology vendors are as follows:
• Review and provide comments on the draft test/QA plan.
• Accept (by signature of a company representative) the final test/QA plan prior to test
initiation.
• Provide two units of their technology for evaluation during the verification test.
• Provide all other equipment/supplies/reagents/consumables needed to operate their
technology for the duration of the verification test.
• Supply training on the use of the technology, and provide written consent and
instructions for test staff to carry out verification testing, including written
instructions for routine operation of their technology.
• Provide maintenance and repair support for their technology, on-site if necessary,
throughout the duration of the verification test.
• Review and provide comments on the draft verification report and statement for their
respective technology.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 12 of 44
Version 1.0
Date: 1/30/2007
A4.3 EPA
EPA's responsibilities in the AMS Center are based on the requirements stated in the
"Environmental Technology Verification Program Quality Management Plan" (EPA QMP).2 The
roles of specific EPA staff are as follows:
Ms. Elizabeth Betz is EPA's AMS Center Quality Manager. For the verification test,
Ms. Betz will:
• Review the draft test/QA plan.
• Perform at her option one external technical systems audit during the verification test.
• Notify the EPA AMS Center Project Officer of the need for a stop work order if the
external audit indicates that data quality is being compromised.
• Prepare and distribute an assessment report summarizing results of the external audit.
• Review draft verification reports and verification statements.
Mr. Robert Fuerst is EPA's Project Officer for the AMS Center. Mr. Fuerst will:
• Review the draft test/QA plan.
• Approve the final test/QA plan.
• Review the draft verification reports and verification statements.
• Oversee the EPA review process for the test/QA plan, verification reports, and
verification statements.
• Coordinate the submission of verification reports and verification statements for final
EPA approval.
A5 BACKGROUND
The ETV Program's AMS Center conducts third-party performance testing of
commercially available technologies that detect or monitor natural species or contaminants in air,
water, and soil. The purpose of ETV is to provide objective and quality assured performance data
on environmental technologies, so that users, developers, regulators, and consultants can make
informed decisions about purchasing and applying these technologies. Stakeholder committees
of buyers and users of such technologies recommend technology categories, and technologies
within those categories, as priorities for testing. Technologies for ballast water exchange
screening tools were identified as a priority technology category through the AMS Center
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 13 of 44
Version 1.0
Date: 1/30/2007
stakeholder process since they have the potential to make the evaluation of ballast water
exchange monitoring procedures efficient and timely.
Mid-ocean ballast water exchange (BWE) will become mandatory for all vessels entering
U.S. waters from outside the 200-mile exclusive economic zone. To support such regulation,
accurate and portable verification tools are needed for determining that ballast water exchange
has taken place. One parameter which has been proposed as a means of distinguishing between
coastal and open ocean water content in ballast water is colored dissolved organic matter
(CDOM).3'4'5 CDOM refers to the fraction of dissolved organic matter that absorbs light and
fluoresces in the ultra-violet (UV) and visible regions of the spectrum. This test/quality assurance
(QA) plan provides procedures for a verification test of BWE screening tools which use CDOM
fluorescence to evaluate ballast water exchange. The objective of this verification test is to
evaluate the performance of BWE screening tools in measuring CDOM relative to a standard
CDOM measurement approach using laboratory bench scale excitation-emission spectrometry
under controlled laboratory conditions. This test will not verify that the technologies
successfully quantify CDOM concentrations or detect ballast water exchange, but rather how
well the technologies measure fluorescence from CDOM compared to a standard technique for
measuring fluorescence. This test will also not represent all types of waters which may be
encountered in ballast water screening, but will attempt to represent a range of water (and
subsequently the range of fluorescence measurements generated by various types of water) that
may be expected in practical application. Critical characteristics of the BWE screening tool
technologies that will be assessed during this testing include the following:
• Accuracy
• Linearity
• Precision
• Method Detection Limit
• Inter-unit reproducibility
• Temperature effects
• Matrix effects
• Data completeness
• Operational factors such as ease of use and maintenance.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 14 of 44
Version 1.0
Date: 1/30/2007
A6 VERIFICATION TEST DESCRIPTION AND SCHEDULE
A6.1 Summary of Technology Category
Technologies to be evaluated in the verification of BWE screening tools include those
which provide a portable, quick measurement of CDOM. The technologies function by
measuring the amount of fluorescent CDOM in ballast water. CDOM has been proposed as a
means of distinguishing between coastal and open ocean water content in ballast water.3'4'5 This
verification test will not evaluate whether the CDOM measurement is capable of detecting
ballast water exchange, or even if the CDOM measurement accurately measures CDOM in
water, rather it will evaluate whether the screening tool CDOM measurements are comparable to
CDOM measurements obtained by a standard technique. The screening tool CDOM
measurements will be reported via a digital display or electronic output signal.
A6.2 Verification Test Schedule
Table 1 shows the planned schedule of testing and data analysis/reporting activities to be
conducted in this verification test. The planned dates for conducting verification tests of BWE
screening tools are January - March 2007 at Battelle's laboratories in Columbus, Ohio. It will be
necessary for participating vendors to provide their technologies to Battelle by a specified date so
testing staff may become familiar with operating the units before testing begins. Vendor staff
will provide training in operating the technologies either in person or by teleconference. The
period of operation for verification testing will be approximately two to four weeks. The test
procedures are described in Section B of this test/QA plan.
Subsequent to the verification test, a separate verification report will be drafted for each
participating technology. These reports will be reviewed by the respective vendor and by peer
reviewers, and submitted to EPA for final signature. Technologies and associated equipment
(but not consumables) will be returned to the vendors at the completion of report writing, unless
other arrangements have been made with Battelle.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 15 of 44
Version 1.0
Date: 1/30/2007
Table 1. Planned Verification Test Schedule
Dates
January-
February 2007
January -March
2007
February-
March 2007
March- April
2007
April-May 2007
Testing Activities
Set up of ballast water exchange
screening tools and training of
verification testing staff on technology
use.
Conduct verification testing.
Return equipment to vendors.
Data Analysis and Reporting
Review and compile test data and records as they
become available.
Review and summarize verification testing staff
observations.
Prepare report templates and complete common
sections of reports.
Evaluate and analyze data generated during testing.
Complete draft reports and submit for vendor and
peer reviews.
Revise draft reports and submit final reports for
EPA approval.
A6.3 Test Site
Laboratory analyses will be conducted in Battelle laboratories in Columbus, Ohio. There
will be no field portability testing during this technology verification.
A6.4 Health and Safety
Battelle will conduct all verification testing and reference CDOM measurements
following the safety and health protocols in place for the laboratory and facilities. This includes
maintaining a safe work environment and a current awareness of handling potentially toxic
chemicals. Exposure to potentially toxic chemicals will be minimized, personal protective
equipment will be worn, and safe laboratory practices will be followed.
A7 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA
In performing the verification test, Battelle will follow the technical and QA procedures
specified in this test/QA plan and will comply with the data quality requirements in the AMS
Center QMP. l The objective of this verification test is to evaluate the performance of BWE
screening tools in their ability to measure CDOM versus a standard CDOM measurement
technique under controlled laboratory conditions. This evaluation will assess the capabilities of
the BWE screening technologies in both laboratory prepared and real-world open ocean and
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 16 of 44
Version 1.0
Date: 1/30/2007
coastal environmental samples (no actual ballast water samples are anticipated), and will include
a comparison between the BWE screening tool results and those of a standard technique as
described in Section B4. Additionally, this verification test will rely upon verification testing
staff observations to assess other performance characteristics of the BWE screening tools.
Below is a discussion of the quality objectives and the criteria for measurement data that have
been established to assure that the test objectives are met.
A7.1 Quality Objectives
Data quality objectives indicate the minimum data quality required to meet the BWE
screening tool verification objectives. Data quality objectives for this verification test include
those related to the reference method performance, those related to the BWE screening tool
technology performance, as well as those related to documenting verification testing staff
observations. Data quality objectives for the reference method (see Section B4) are presented in
terms of data quality indicator (DQI) criteria for the critical measurements associated with the
reference method and are listed in Table 2 and discussed in Section A7.2. The reference method
data quality relies, in part, on proper sample preparation, proper application of the reference
method, and proper maintenance of reference method instrumentation. Battelle will rely on the
vendor's data quality objectives for each BWE screening tool in order to insure that the
screening technology is performing properly during testing. This will include adhering to each
vendor's criteria for calibration and performance of positive and negative control samples. The
screening technology data quality relies on proper operation and maintenance of the BWE
screening tools and proper sample preparation. Quantitative data quality objectives for the
operator observations have not been defined but are incorporated into documentation
requirements and data review, verification, and validation requirements for this verification test.
A7.2 Criteria for Measurement Data
Table 2 presents the DQIs and criteria for the reference method critical measurements.
The reference method measurement quality will be assured by adherence to these DQI criteria.
The reference method measurement quality will be monitored by comparing the Varian Gary
Eclipse spectrometer output at the wavelengths of interest with calibration standards
(naphthalene-anthracene) traceable to National Institute of Standards and Technology (NIST)
standards, observation of quinine sulfate at reference levels, and by inclusion of negative control
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 17 of 44
Version 1.0
Date: 1/30/2007
samples and positive control samples for stability monitoring. Criteria for the BWE screening
tool technologies for critical measurements related to calibration standards and recommendations
for appropriate positive and negative controls and their critical measurements will be provided
by each vendor. The Battelle Quality Manager or his designee will perform a technical systems
audit (TSA) at least once during this verification test to review these QA/QC requirements. The
EPA AMS Center Quality Manager also may conduct an independent TSA, at her discretion.
A8 SPECIAL TRAINING/CERTIFICATION
Documentation of training related to technology testing, data analysis, and reporting is
maintained for all Battelle technical staff in training files at their respective Battelle location.
The Battelle Quality Manager may verify the presence of appropriate training records prior to the
start of testing. If Battelle staff operate and/or maintain a technology during the verification test,
the technology vendor will be required to train those staff prior to the start of testing. Battelle
will document this training with a consent form, signed by the vendor, that states which specific
Battelle staff have been trained and determined by the vendor to be competent in operation of the
vendor's technology. Battelle technical staff will have a minimum of a bachelor's degree in
science/engineering or have equivalent work experience.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 18 of 44
Version 1.0
Date: 1/30/2007
Table 2. DQIs and Criteria for Critical Measurements for Reference Method
DQI
Precision
Bias and Accuracy of
Instrument --Zero
Bias and Accuracy of
Overall Instrument
Sensitivity
Bias and Accuracy of
Sample Measurements
Method of
Assessment
Triplicate analyses of
test samples
Zero Check-observe
signal with emission
beam blocked
Using conditions
simulating tests,
observe NIST
calibration standard
(naphthalene-
anthracene)
Initial Calibration-
various levels
quinine sulfate
prepared in water
(Burdick and Jackson
HPLC grade, or
equivalent). Start
with unspiked water,
then observe 4 or
more additional
concentrations of
quinine sulfate
prepared as per
ASTM E 579-046
Calibration Check
Sample-one
concentration of
quinine sulfate (i.e,
10 ppb quinine
sulfate).
Positive Control
Sample- 50 ppb SR
fulvic acid
Negative Control
Sample- (Burdick
and Jackson HPLC
grade, or equivalent)
Frequency
All test samples
Every 4 hours
Beginning of
sample set
analyzed each
day, at least
every 4 hours
during analysis
of the set, and
at conclusion of
the set.
Perform initial
calibration at
the outset of
verification
testing and after
any parameter
changes or
overall
instrument
sensitivity
changes of
>4%.
Minimum 5%
of all samples
tested.
Minimum 5%
of all samples
tested.
Minimum 10%
of all samples
tested.
Minimum
Acceptance Criteria
3% relative standard
deviation (RSD)
< 2 counts with Time
Constant = 5 seconds.
Target: constant
intensity throughout
the experiment.
If >1% variation of
signal, then adjust
voltage.
Check standard is
within ± 2% change in
response from initial
calibration after
adjustment of Overall
Instrument Sensitivity
Reanalyze affected
samples since last
successful check if
calibration check
change is >4%.
No more than ±3%
change in response
during the sample set
analyzed on the same
day.
Within 10% of counts
of original zero in
initial calibration
standard, which is near
zero counts.
Corrective Action
Data considered suspect
and reanalyzed or reported
with qualifiers if reanalysis
is not possible
Re-zero the instrument with
the emission beam blocked.
Recalibrate to target
fluorescence value with
NIST calibration standard.
Constant intensity is re-
established by varying the
photomultiplier tube (PMT)
operating voltage. If
variation is >3%, re-
analyze previous sample set
or report with qualifiers.
Investigate sources of
standard contamination or
changes in instrument
parameters; clean the
cuvette; reanalyze fresh
control or standard; run
Varian validation tests for
operation.
Adjust Overall Instrument
Sensitivity; check new
calibration check sample
(i.e., different concentration
of quinine sulfate or freshly
prepared standard); repeat
initial calibration; run
Varian validation tests for
operation.
Clean cell (multiple times if
necessary); check
instrument zero.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 19 of 44
Version 1.0
Date: 1/30/2007
DQI
Selectivity
Completeness
Representativeness
Method of
Assessment
Negative Control
Sample- - (Burdick
and Jackson HPLC
grade, or equivalent)
Positive Control
Sample- 50 ppb SR
fulvic acid
Environmental
Samples-coastal and
open ocean
Amount of valid data
obtained
Performance Test
and Environmental
Samples (see
Table 5)
Frequency
Minimum 10%
of all samples
tested.
Minimum 5%
of all samples
tested.
All
Environmental
Samples used
in verification
testing.
Overall number
of data points
collected for
test
Overall for test
Minimum
Acceptance Criteria
Within 10% of counts
of original zero in
initial calibration
standard.
No more than ± 3%
change in response
during the sample set
analyzed on the same
day.
Environmental
Samples will be
monitored for meeting
vendor criteria for
distinguishing between
coast and open ocean
samples.
90% of overall data
points collected should
be valid.
Target: samples with
fluorescence
measurements which
span the responses
obtained in the initial
calibration curve.
Corrective Action
Adjust Overall Instrument
Sensitivity; check new
control sample; repeat
initial calibration; run
Varian validation tests for
operation.
If an Environmental
Sample has a CDOM
measurement which does
not correspond to the
vendor criteria for the type
of sample being tested (i.e.,
a coastal sample does not
meet vendor's criteria for
identification as a coastal
sample) and the above
corrective actions have
been taken, flag the results
in the report. Full
excitation-emission matrix
(EEM) measurements may
also be obtained on the
sample in question in order
to further evaluate sample
properties.
If feasible, analyze
additional samples to meet
the acceptance criterion.
Additional samples or
samples fortified with
quinine sulfate or fulvic
acid may be considered to
replace some
Environmental Samples if
those initially tested do not
represent a range of
fluorescence measurements
that span the responses
obtained in the initial
calibration curve. Such
samples will only be used
to provide a range of
fluorescence measurements
for side-by-side comparison
of screening tool
measurements to reference
method measurements.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 20 of 44
Version 1.0
Date: 1/30/2007
A9 DOCUMENTATION AND RECORDS
The records for this verification test will include the test/QA plan, sample chain-of-
custody forms, laboratory record books (LRB), data collection forms, electronic files (both raw
data and spreadsheets of sample results or statistical calculations), and the final verification
reports and verification statements. All of these records will be maintained in the Verification
Test Coordinator's office (or at the test location during the test) and will be transferred to
permanent storage at Battelle's Records Management Office at the conclusion of the verification
test. The location (e.g., specific personal computer, server, or media type and storage location) of
final versions of the electronic files will be noted in the test records. All Battelle LRBs are stored
indefinitely, either by the Verification Test Coordinator or Battelle's Records Management
Office. EPA will be notified before disposal of any files. Table 3 has further details regarding the
data recording practices and responsibilities.
All written records must be in ink. Any corrections to notebook entries, or changes in
recorded data, must be made with a single line through the original entry. The correction is then
to be entered, initialed, and dated by the person making the correction. In all cases, strict
confidentiality of data from each vendor's technology, and strict separation of data from different
vendors' technology, will be maintained. Separate files (including manual records, printouts,
and/or electronic data files) will be kept for each technology.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 21 of 44
Version 1.0
Date: 1/30/2007
Table 3. Summary of Data Recording Process
Data to Be
Recorded
Dates, times, and
details of test events,
technology
maintenance, down
time, etc.
Technology
calibration
information
Technology readings
Sample collection
and reference
method analysis
procedures,
calibrations, QA, etc.
Reference method
results
Where Recorded
ETV LRBs or data
recording forms
ETV LRBs, data
recording forms, or
electronically
Either recorded
electronically by the
technology and
downloaded to an
independent
computer or hard
copy data printed by
the technology and
taped into the ETV
LRB or hand entered
into ETV LRBs or
data recording forms.
LRBs, chain-of-
custody, or other
data recording forms
Electronically or
manually into ETV
LRBs or data
recording forms.
Where possible at
least the same
number or a
maximum of one
number more
significant figures as
the BWE screening
technology result
will be reported for
the reference
method.
How Often
Recorded
Start/end of test
procedure, and at
each change of a test
parameter or change
of technology status
At technology
calibration or
recalibration
Recorded
continuously for
electronic data and
printed after each
measurement for
hard copy print-outs
or recorded manually
with each reading.
Throughout sampling
and analysis
processes
Every sample or QC
analysis
By Whom
Battelle
Battelle
Battelle
Battelle and others
assisting in sample
collection
Battelle
Disposition of
Data
Used to organize and
check test results;
manually
incorporated in data
spreadsheets as
necessary
Incorporated in
verification report as
necessary
Converted to or
manually entered
into spreadsheet for
statistical analysis
and comparisons
Retained as
documentation of
sample collection or
reference method
performance
Transferred to
spreadsheets for
calculation of results,
and statistical
analysis and
comparisons
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 22 of 44
Version 1.0
Date: 1/30/2007
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
BWE screening tool technologies will be tested only in a laboratory under controlled
laboratory conditions; no field testing will take place during this verification test. This will allow
comparison of the technology results to a reference method under a specified set of conditions.
Performance test and environmental samples will be analyzed in triplicate and evaluated for
accuracy compared to expected measurements based on the reference CDOM analyses,
instrument linearity across the range of concentrations tested, and precision among the replicate
measurements obtained. Two units of each technology will be used to measure the test samples.
Measurements with the two units will be concurrent with each other and with the reference
method measurement, so that inter-unit precision can be evaluated. All measurements made for
direct comparison with the reference method will be conducted at ambient room temperature.
However, because these technologies will be used in a wide range of temperatures in practical
application and because temperature can affect CDOM fluorescence, a subset of test samples will
be analyzed with the BWE screening tool units only at two additional temperatures, one below
ambient and one above ambient (i.e., approximately 4 °C and 34 °C, respectively) in order to
evaluate the screening tool's variability due to temperature effects. Note that while temperature
is one of several variables that might affect practical application (other possibilities include
humidity, ambient light, exposure to the elements, etc.), this verification test will only test the
affect of varying temperature on the screening technology performance.
The analyses will be performed according to the vendor's recommended procedures as
described in the user's instructions or manual, or during training provided to Battelle staff.
Similarly, calibration and maintenance of the technologies will be performed as specified by the
vendor. Results from the technologies being verified will be recorded manually by the operator
on appropriate data sheets or captured in an electronic data system and then transferred manually
or electronically for further data workup. Qualitative characteristics of each technology such as
ease of use will be assessed through observations made by the Test Coordinator and operators
throughout the verification test. The results from each technology will be reported individually.
No direct comparison will be made between technologies, but each technology will undergo
similar testing so it is convenient for end users to evaluate the ETV testing results.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 23 of 44
Version 1.0
Date: 1/30/2007
Bl.l Test Procedures
The BWE screening tool verification test will focus on analyzing a broad range of
samples which will provide a variety of CDOM concentrations. The first sample type will be
performance test samples where analytes or compounds known to cause fluorescence (i.e.,
quinine sulfate or Suwannee River fulvic acid) will be added at multiple concentration levels to
Burdick and Jackson HPLC grade (or equivalent) water and analyzed by the participating
technologies to assess their ability to detect the CDOM fluorescence in water. Analytes will be
added to Burdick and Jackson FtPLC grade water (or equivalent) at a range of concentrations that
span the technologies' linear range or at a minimum cover a range of fluorescence values
expected to be encountered between open ocean and coastal water samples. The second sample
type will be environmental samples and will consist of water collected from various coastal and
open ocean areas. The environmental samples, which are expected to represent a range of
CDOM concentrations that may be encountered in real-world water samples, will be collected
from a maximum of 15 locations and will be measured by the reference method in order to
screen their CDOM response before use in testing. Table 4 lists proposed locations for collecting
environmental samples. A minimum of 10 samples will be selected from those collected to
challenge the technologies with real-world samples of varying CDOM concentrations. If
necessary to achieve a good range of CDOM concentrations, some open ocean or coastal
samples may be fortified in the laboratory (i.e., with quinine sulfate or Suwanee JAiver fulvic
acid) to adjust their CDOM concentration, and used in place of coastal samples. Note that any
fortified samples will be used for side-by-side comparison of the screening tool results to those
obtained with the reference method only and are only for the purposes of providing a range of
fluorescence measurements. Full excitation-emission matrix (EEM) measurements using the
reference method instrumentation will be obtained on these 10 environmental samples used in
verification testing in order to provide additional spectroscopic information on these samples.
For full EEM measurements, the Varian Gary Eclipse spectrometer automatically corrects
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 24 of 44
Version 1.0
Date: 1/30/2007
Table 4. Proposed Environmental Samples
Location
Duxbury Bay, MA
Boston Harbor, MA
Massachusetts Bay - NF7,
MA
Massachusetts Bay - NF10,
MA
Sequim Bay, WA
Puget Sound, WA
East Coast, FL -1
East Coast, FL - 2
Open Ocean - 1
Open Ocean - 2
Long Island Sound, NY
New York Harbor, NY
New York Bight, NY
San Diego Harbor, CA
Narragansett Bay, RI
Description
Off the dock at Battelle Duxbury
Operations in Duxbury, MA
Inside of Neponset Estuary in Boston
Harbor, MA
Nine miles east of Deer Island, MA
Nine miles east of Deer Island, MA
Off the dock at Battelle Marine Sciences
Lab in Sequim, WA
Outside of Ediz Hook in Port Angeles, WA
Inter-coastal water way in West Palm
Beach, FL
Atlantic Ocean beach off Palm Beach, FL
NASS-5 Open Ocean Seawater Reference
Material for Trace Metals (available from
National Research Council Canada)
MOOS-1 Seawater Certified Reference
Material for Nutrients
(available from National Research Council
Canada)
Dock in Port Jefferson, NY
East River, NY
Atlantic ocean sample from a beach in
South Hampton, NY
San Diego Harbor, CA
Off 2-14 Great Island Rd, Narragansett, RI
Sample Type
Coastal seawater
A mixture of freshwater and coastal
seawater with expected high CDOM
fluorescence
Coastal seawater
Coastal seawater
Coastal seawater
Coastal seawater
A mixture of freshwater and coastal
seawater with expected high CDOM
fluorescence
Coastal seawater
Open ocean seawater with expected
low CDOM fluorescence
Open ocean seawater with expected
low CDOM fluorescence
Coastal seawater
A mixture of freshwater and coastal
seawater with expected high CDOM
fluorescence
Coastal seawater
Coastal seawater
Coastal seawater
excitation, and emission will be calibrated so as to ensure the highest removal of instrumental
scaling effects. The full EEM measurements are for informational purposes only. Reference
method measurements used for comparing the technology results to reference method results will
be obtained using only the wavelengths and technique that the BWE screening tool uses and not
the full EEM measurements. The third sample type will be quality control samples. Quality
control samples are discussed further in Section B5. Table 5 shows the samples that will be
evaluated during this verification test.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 25 of 44
Version 1.0
Date: 1/30/2007
Table 5. Verification Test Samples
Performance
Factor
Performance Test
Quinine sulfate prepared
in Burdick and Jackson
HPLC grade water (or
equivalent) per ASTM
E579-046
Fulvic acid prepared in
Burdick and Jackson
HPLC grade water (or
equivalent). The pH of
the fulvic acid solutions
will not be adjusted, but
will be checked and
recorded prior to their
use.
Accuracy, linearity,
precision,
temperature effects
Method detection
limit
Accuracy, linearity,
precision,
temperature effects
Method detection
limit
Sample Description
unspiked
1 ppb quinine sulfate
5 ppb quinine sulfate
10 ppb quinine surf ate
50 ppb quinine sulfate
100 ppb quinine sulfate
Quinine sulfate at 5 x vendor
detection limit
unspiked
1 ppb Suwanee River (SR)
fulvic acid
5 ppb SR fulvic acid
10 ppb SR fulvic acid
50 ppb SR fulvic acid
100 ppb SR fulvic acid
SR fulvic acid at 5 x vendor
detection limit
Replicates per BWE
Screening Tool Unit
~4°C
3
3
3
3
3
3
-
3
3
3
3
3
3
-
~24°C
3
3
3
3
3
3
7
3
3
3
3
3
3
7
~34°C
3
3
3
3
3
3
-
3
3
3
3
3
3
-
Environmental Samples
Location 1 -open ocean
Location 2-open ocean
Location 3 -coastal
Location 4-coastal
Location 5 -coastal
Location 6-coastal
Location 7-coastal
Location 8-coastal
Location 9-coastal
Location 10-coastal
Matrix Effects
unspiked
-
-
-
-
-
-
-
-
-
-
3
3
3
3
3
3
3
3
3
3
-
-
-
-
-
-
-
-
-
-
Quality Control
Negative control
Positive control
Calibration Check
Burdick and Jackson HPLC
grade water (or equivalent)
50 ppb SR fulvic acid
10 ppb quinine surf ate
TOTAL
8 (minimum)
8 (minimum)
Single measurement per every
9 verification sample
measurements
185 (minimum)
Shading indicates samples to be tested and number of replicates using the reference method.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 26 of 44
Version 1.0
Date: 1/30/2007
ETV verifications usually include a comparison of the results generated by the
technologies being verified with the results of analysis of the same samples using a standard
reference method that measures the same endpoint. Reference methods are discussed in
Section B4.
The technologies will be evaluated for the following parameters:
B1.1.1 Accuracy
Accuracy will be determined by comparing the BWE screening tool technology CDOM
measurement to the CDOM measurement generated by a Varian Gary Eclipse spectrometer at a
single temperature (i.e., data from ambient temperature will be used) for both performance test
samples and environmental samples. See Section B4 for complete details of how the reference
method measurements will be made in comparison to how the BWE screening tool
measurements are made. Percent difference (PD) between the technology result and the Varian
Gary Eclipse spectrometer result will be determined.
B 1.1.2 Linearity
Linearity will be determined by plotting the CDOM measurements taken while analyzing
varying concentrations of analytes which are known to fluoresce against the analyte
concentration and performing linear curve fitting to determine the slope and regression statistics.
Bl. 1.3 Precision
Precision will be measured among replicate measurements of the same sample by
determining relative standard deviation (RSD).
B.I. 1.4 Method Detection Limit
Method detection limit will be determined by analyzing 7 replicates of known fluorescing
analytes at a concentration five times the vendor expected detection limit for the analyte.
Bl. 1.5 Inter-unit Reproducibility
Inter-unit reproducibility will be determined by calculating the relative percent difference
(RPD) between the average of replicate CDOM measurements of the same sample taken at the
same temperature for all PT and environmental samples made using two different units of the
BWE screening tool technology.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 27 of 44
Version 1.0
Date: 1/30/2007
Bl. 1.6 Temperature Effects
The BWE screening tool CDOM measurements made at ~ 4 °C and ~ 34 °C will be
compared to the BWE screening tool CDOM measurements made at ambient temperature
(~ 24 °C) by determining RPD for each concentration of the curve analyzed at multiple
temperatures. Note that reference method measurements will only be taken at ambient
temperature and so there will not be any direct comparison of the BWE screening tool results to
reference method measurements at ~ 4 °C and ~ 34 °C. Comparisons will only be between the
BWE screening tool results at ~ 4 °C and ~ 34 °C to the BWE screening tool results at ambient
temperature. During the measurements at ~ 4 °C and ~ 34 °C, the testing solutions will be
equilibrated at the testing temperature for at least 30 minutes prior to the start of testing. BWE
screening tool units will be equilibrated per vendor's recommendations for when there is a
difference in instrument versus sample temperature, if available. If this information is not
available from the vendor, then a minimum 5 minute equilibration time will be used for
instrumentation.
B.I. 1.7Matrix Effects
Matrix effects will be evaluated by comparing the environmental sample PD values
obtained for determining accuracy in Section B. 1.1.1 with those that were obtained for the PT
samples to determine if the environmental sample PD values are statistically different from the
PT sample PD values. Data will be evaluated to determine if there are trends in PD with the
environmental samples that could be due to matrix effects. Full EEM data may be used to help
assess differences in environmental sample matrices.
B. 1.1.8 Data Completeness
Data completeness will be determined as the number of valid measurements out of the
total number of measurements taken.
Bl.1.9 Operational Factors
Operational factors such as maintenance needs, calibration frequency, data output,
consumables used, ease of use, repair requirements, waste production, and sample throughput
will be evaluated based on operator and Verification Test Coordinator observations. An LRB or
data sheets will be used to document observations. Examples of information to be recorded
include the daily status of diagnostic indicators for the technology, use or replacement of any
consumables, the effort or cost associated with maintenance or repair, vendor effort (e.g., time on
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 28 of 44
Version 1.0
Date: 1/30/2007
site) for repair or maintenance, the duration and causes of any technology down time or data
acquisition failure, operator observations about technology ease of use, clarity of the vendor's
instruction manual, user-friendliness of any needed software, overall convenience of the
technologies and accessories/consumables, or the number of samples that could be processed per
hour or per day. These observations will be summarized to aid in describing the technology
performance in the verification report on each technology.
B1.2 Statistical Analysis
The statistical methods and calculations used for evaluating quantitative performance
parameters are described in the following sections.
B.I. 2.1 Accuracy
Accuracy will be determined by calculating the PD between the BWE screening tool
technology CDOM measurement (Mi) and the CDOM measurement generated by a Varian Gary
Eclipse spectrometer (M2) at a single temperature (i.e., data from ambient temperature will be
used) for both performance test samples and environmental samples using the following
equation.
Mi-A/2
PD(%) = — x 100 (1)
B 1.2.2 Linearity
Linearity will be determined by plotting the CDOM measurements taken while analyzing
varying concentrations of analytes which are known to fluoresce (y-axis) against the analyte
concentration (x-axis) and performing linear curve fitting to determine the slope (m) and
intercept (b) in the equation y=mx +b. Correlation coefficients such as the Pearson's r values
and coefficient of determination (r2) values will be calculated.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 29 of 44
Version 1.0
Date: 1/30/2007
B1.2.3 Precision
The standard deviation (S) of the results for the replicate analyses of the same sample
will be calculated as follows.
S =
1 ^ (
-—I
11 1 k=\
1/2
(2)
where n is the number of replicate samples, Mk is the CDOM measurement for the kih sample,
and Mis the average CDOM measurement of the replicate samples. The BWE screening tool
technology precision for each sample will be reported in terms of the relative standard deviation
(RSD), which will be calculated as follows.
' S
RSD(%)= ^
M
x 100 (3)
B 1.2. 4 Method Detection Limit
The method detection limit (MDL) will be determined according to procedures described
in 40CFR136 Appendix B7 and will be assessed from seven replicate analyses of a fortified
sample. Fortified samples will be generated by adding known fluorescing compounds such as
quinine sulfate and SR fulvic acid to Burdick and Jackson HPLC grade water, or equivalent. The
target analyte will be added at a concentration approximately five times the vendor-stated
detection limit. The MDL will be calculated using the following equation:
MDL =
Where tis the Student's value of 3.143 for a 99% confidence level when the degrees of freedom
(N-l) equals six, and S is the standard deviation of the replicate samples.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 30 of 44
Version 1.0
Date: 1/30/2007
B.I. 2.5 Inter-unit Reproducibility
Inter-unit reproducibility will be determined by evaluating the RPD between the average
of replicate measurements made for each sample tested using two separate units of the
technology being verified. The equation for RPD, reported in percent, is as follows:
Mi-A/2
RPD(%) = — —
Mi + A/2
x 200 (5)
where M; is the average of replicate measurements made by the first unit of the technology and
M2 is the average of replicate measurement made by the second unit of the technology being
evaluated.
B.I.2.6 Temperature Effects
Temperature effects will be determined by measuring the PD (using the equation listed in
B.I.2.1) between the average of replicate measurements made for each sample at either 4 °C or
34 °C (Mi) against the average measurement taken at 24 °C (M^). T-tests may be applied to the
data to help determine whether PD measurements are significant.
B.I.2.7 Matrix Effects
Matrix effects will be determined by comparing the accuracy PD measurements for the
PT samples to the accuracy PD measurements for the environmental samples. PD will be
determined as described in B.I.2.1. T-tests may be applied to the data to help determine whether
the differences in PD measurements between the PT and environmental samples are significant.
B.I.2.8 Data Completeness
Data Completeness will be calculated as the percentage of the total possible data by
taking the number of valid data measurements generated by each technology (Mva/^, and
dividing by the total number of data measurements included in verification testing (Mtotai).
IVLvalid fs:\
Completeness^ = x 100 (6)
Mtotai
The cause of any substantial loss of data will be established from operator observations or
technology records, and noted in the discussion of the data completeness results.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 31 of 44
Version 1.0
Date: 1/30/2007
Bl.2.9 Operational Factors
There are no statistical calculations applicable to operational factors. Operational factors
will be determined based on documented observations of the testing staff and the Verification
Test Coordinator.
B1.3 Reporting
The data obtained in the verification test will be compiled separately for each vendor's
technology, and the data evaluations will be applied to each technology's data set without
reference to any other. At no time will data from different vendor's technology be
intercompared or ranked. Following completion of the data evaluations, a draft verification
report and verification statement will be prepared for each vendor's technology, stating the
verification test procedures and documenting the performance observed. For example,
descriptions of the data acquisition procedures, use of vendor supplied proprietary software,
consumables used, repairs and maintenance needed, and the nature of any problems will be
presented in the draft report. Each report will briefly describe the ETV Program, the AMS
Center, and the procedures used in verification testing. The results of the verification test will be
stated quantitatively, without comparison to any other technology tested, or comment on the
acceptability of the technology's performance. Each draft verification report will be submitted
for review by the respective technology vendor and by EPA and other peer reviewers.
Comments on the draft report will be addressed in revisions of the report. The peer review
comments and responses will be tabulated to document the peer review process. The reporting
and review process will be conducted according to the requirements of the AMS Center QMP.1
B2 SAMPLING REQUIREMENTS
B2.1 Sample Collection, Storage and Shipment
Pre-cleaned 500 mL amber glass jars (ESS QC grade, or equivalent) with Teflon-lined
caps will be used as containers for all samples collected specifically for this study. Some samples
that have been collected for other programs may be used for verification testing and in those
cases, the samples will be provided in the container selected by the other program and
contamination control and storage conditions from that program will be documented. For
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 32 of 44
Version 1.0
Date: 1/30/2007
samples collected specifically for this study and not in conjunction with any other program, just
prior to sample collection, the sample jars and caps will be rinsed three times with water from the
collection location. Where possible these samples will be collected from a dock or other similar
feature. If wading is necessary to collect the samples, care will be taken to obtain waist deep
water and to avoid any sediments resuspended while wading.
Sample collection will also avoid any surface sheen, which may cause interference with
CDOM fluorescence measurements. If surface sheen is present and a better location cannot be
found, the jar will be submerged at least 6 inches below the water surface with the cap on. The
cap will then be unscrewed and the sample collected and the jar recapped while under water. If
surface sheen is not present, the jar will be uncapped and filled just below surface level. In all
instances, once the sample has been collected, a small portion of the sample will be poured off
and the jar recapped in order to provide an air space that will facilitate later mixing. Duplicate
samples will be collected at each location to provide backup samples in case of loss during
shipping and handling, and to provide a backup archive sample during testing. The samples will
be refrigerated or frozen (if frozen, care must be taken to ensure that the sample can expand
without breaking the sample jar) until shipment to Battelle on ice.
Shipments will be via a trackable overnight delivery service to the Battelle sample
custodian. At Battelle, the samples will be stored refrigerated or frozen (depending on how the
sample was sent to Battelle) until testing. While some samples may be filtered in the field upon
collection, all samples (including those previously filtered in the field) will be filtered just prior
to measuring the sample fluorescence in the laboratory. Samples (both those for use with the
BWE screening tool and those used for the reference method) will be filtered with the size and
type of filter recommended by the BWE screening tool vendor for their technology. All samples,
including PT, environmental, and QC samples, will be equilibrated at the testing temperature
(~ 4 °C, ~ 24 °C, or ~ 34 °C) before fluorescence measurements commence. The actual testing
temperature will be recorded during testing.
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Sample custody will be documented throughout collection, transport, shipping (if
necessary), and analysis using standard chain-of-custody (COC) forms provided by Battelle or
supplied by others providing samples for testing, as appropriate. Samples transferred within
Battelle may be documented in bound sample login LRBs. Each COC form will summarize the
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 3 3 of 44
Version 1.0
Date: 1/30/2007
samples collected. The COC forms will track sample release from the sampling location to
Battelle. Each COC form will be signed by the person relinquishing the samples once that person
has verified that the COC form is accurate. The original sample COC forms will accompany the
samples; the shipper will keep a copy. Upon receipt at Battelle, COC forms will be signed by the
person receiving the samples once that person has verified that all samples identified on the COC
forms are present. Any discrepancies will be noted on the form and the sample receiver will
immediately contact the Verification Test Coordinator to report missing, broken, or otherwise
compromised samples. Copies of all COC forms will be delivered to the Verification Test
Coordinator, and maintained with the test records. Samples being transferred within Battelle may
be documented in a sample login LRB which will be used to note the current location of all
samples housed at Battelle.
B4 LABORATORY REFERENCE METHOD
The BWE screening tools provide a measure of CDOM by measuring fluorescence at
various wavelengths using a hand-held instrument. Reference measurements for CDOM will be
performed by Battelle using a laboratory bench scale Varian Gary Eclipse spectrometer. Settings
for the Varian Gary Eclipse spectrometer (e.g., wavelengths, slit widths, angle for fluorescence
measurement, path lengths, etc.) will be set to resemble those used by the BWE screening tool.
For example, slits of 10 nm will be used on the Varian Gary Eclipse spectrometer if filters of
10 nm bandwidth are used on the BWE screening tool.
Results from the Varian Gary Eclipse spectrometer will be calculated and reported in the
same manner that the BWE screening tool calculates and reports results. For example, if the
screening tool measures CDOM as a ratio of intensities at two wavelengths, then the Varian Gary
Eclipse spectrometer will be used to measure the same two wavelengths under the same
excitation and emission conditions and a ratio at the same wavelengths will be calculated using
the Varian Gary Eclipse data. It is not expected or necessary that the ratio be exactly the same.
Efficiencies of gratings and other instrument conditions will vary from instrument to instrument
and will, in fact, likely make this ratio different. The ratios from the BWE screening tool will be
calibrated against a series of quinine sulfate concentrations. The Varian Gary Eclipse
spectrometer will be calibrated in the same range with the same standards. If both instruments
are linear in that range and track through zero when a zero concentration standard is analyzed,
there will be a single value constant that can be used to multiply one result to correlate to the
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 34 of 44
Version 1.0
Date: 1/30/2007
other result. If the relationship is not linear, the correlation can be a more complex relationship,
but if proper zeroing is performed on each instrument and the calibration concentration ranges
are the same, then the linearity of the BWE screening tool and the Varian Gary Eclipse
spectrometer can be directly compared through a relationship established based on the standard
curves analyzed on each instrument.
The Varian Gary Eclipse spectrometer will be operated according to the procedures
described in the Varian instrument manuals and any deviations will be documented. Specific
operating procedures relevant to the Varian Gary Eclipse spectrometer measurements will
depend on the BWE screening technology being evaluated and how it measures CDOM. For
example, if only two wavelengths need to be measured, the Varian Gary Eclipse individual
wavelength software package will be used rather than the scanning package software. The PMT
voltage will be set to provide counts in the 0-1000 range for the strongest anticipated
fluorescence. A standard operating procedure (SOP) for the Varian Gary Eclipse spectrometer
reference measurement will be prepared for each technology tested. Each SOP will include
instructions for:
• Setting the Varian Gary Eclipse spectrometer parameters so that the spectrometer out
performs the technology being tested. Long enough integration time constants will be
used to generate signal-to-noise greater than the technology produces.
• Validating all instrument parameters and storing the parameters used in a method file.
• Calibrating intensity.
• Properly cleaning and checking visual transmittance of cells between measurements.
• Checking sample-related instability and span consistency by analyzing calibration
check samples and positive control samples during each sample set.
The quinine sulfate solution used for calibration will be verified by a second source
standard or analysis of two independently prepared solutions from the same lot at least once
during the tests.
B5 QUALITY CONTROL
Steps will be taken to maintain the quality of data collected during this verification test.
This will include analyzing specific quality control samples (QCS) at a regular frequency. QCSs
will include negative controls, positive controls, and calibration checks. Negative control
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 35 of 44
Version 1.0
Date: 1/30/2007
samples will be used to help ensure that no sources of contamination are introduced in the
sample handling and analysis procedures. The positive control samples and calibration checks
will indicate to the operator whether or not the technology is functioning properly. The vendor
will provide the approximate endpoint that should result upon analysis of the positive control and
calibration check by their technology. QCSs producing results that do not meet the anticipated
results specified by the vendor will be reanalyzed and corrective action taken if needed to ensure
that test sample results are not affected. Corrective actions may include reanalyzing samples to
verify that the technology has been operated properly, conducting maintenance, or recalibrating.
Positive and negative controls will be analyzed at a frequency of approximately 5 % based on the
total number of test samples (see Table 5) for the BWE screening tools. More frequent negative
controls may be analyzed using the Varian Gary Eclipse spectrometer to check cell cleanliness
and instrument zero. Calibration checks will be analyzed at a minimum after every nine
measurements of PT or environmental samples with the BWE screening tool and at a minimum
as 5% of the total number of test samples for the Varian Gary Eclipse spectrometer.
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The equipment used by Battelle (i.e., instrumentation used for CDOM reference
measurements, balances or pipettes used to prepare standards, etc.) will be tested, inspected, and
maintained as per the standard operating procedures of Battelle and/or the manufacturer's
recommendations so as to meet the performance requirements established in this document.
When technical staff operate and maintain technologies undergoing testing, those activities will
follow directions provided by the technology vendor.
B7 CALIBRATION/VERIFICATION OF TEST PROCEDURES
The Varian Gary Eclipse spectrometer used for reference analyses will be calibrated as
appropriate before any samples are analyzed and recalibrated as needed based on the results of
any calibration check samples.
Technologies undergoing testing will be calibrated initially by the respective technology
vendor prior to shipping the technology to Battelle or during training, and will be repeated
according to direction from the vendor. Calibration checks will be performed after every 9
measurements during verification testing for the BWE screening tools. In the event that
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 36 of 44
Version 1.0
Date: 1/30/2007
recalibration is necessary, the recalibration will be carried out following the vendor's
instructions. All calibrations will be documented as appropriate by the technical staff.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
In general, this verification test relies on the materials and equipment provided by the
vendors. Battelle will provide the following equipment and materials needed for the preparation
of the performance test, environmental and QC samples:
• Burdick and Jackson HPLC grade water, or equivalent
• various laboratory supplies necessary for accurate preparation of the test samples and
subsequent dilutions (i.e., volumetric pipets; pipet bulbs; Eppendorf micro pipettes
and pipette tips, or equivalent; volumetric flasks; disposable pipets; balances; etc.)
• reference standards with a known level of purity for target analytes - NIST traceable,
or equivalent if available
• glass sample containers
• equipment and consumables for the Varian Gary Eclipse spectrometer
• personal protective equipment
All materials, supplies, and consumables will be ordered by the Verification Test
Coordinator or designee. Where possible, Battelle will rely on material and consumable sources
that have been used previously without problems for ETV verification testing. Battelle will also
rely on previous experience or recommendations from EPA advisors, stakeholders, test
collaborators, subcontractors, or technology vendors. Upon receipt of any supplies or
consumables, the Verification Test Coordinator or designee will visually inspect and ensure that
the materials received are those that were ordered and that there are no visual signs of damage
that could compromise the suitability of the materials. If damaged or inappropriate goods are
received, they will be returned or disposed of and arrangements will be made to receive
replacement materials. Certificates of analysis (COA) or other documentation of analytical
purity will be checked for all reagents and standards to ensure suitability for the verification test
and a copy will be stored with the test files. Where possible, reagents (i.e., HPLC grade water,
sulfuric acid, etc.) used to prepare standards will be checked for background fluorescence prior
to their use. Unsuitable materials will be returned or disposed of and arrangements for the
receipt of replacement materials will be made.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 37 of 44
Version 1.0
Date: 1/30/2007
B9 NON-DIRECT MEASUREMENTS
No non-direct measurements will be used during this verification test.
BIO DATA MANAGEMENT
Various types of data will be acquired and recorded electronically or manually by
Battelle during the verification test. Table 2 summarizes the types of data to be recorded. All
maintenance activities, repairs, calibrations, and operator observations relevant to the technology
operation will be documented by technical staff in LRBs or on data sheets. Results from the
reference methods, including raw data, analyses, and final results, will be compiled by Battelle.
Records received by or generated by any technical staff during the verification test will
be reviewed by a Battelle staff member within two weeks of generation or receipt, before the
records are used to calculate, evaluate, or report verification results. If a Battelle staff member
generated the record, this review will be performed by a Battelle technical staff member involved
in the verification test, but not the staff member who originally generated the record. The review
will be documented by the person performing the review by adding his/her initials and date to the
hard copy of the record being reviewed. In addition, any calculations performed by technical
staff will be spot-checked by Battelle QA and/or technical staff to ensure that calculations are
performed correctly. Calculations to be checked include any statistical calculations described in
this test/QA plan. The data obtained from this verification test will be compiled and reported
independently for each technology. Results for technologies from different vendors will not be
compared with each other.
Among the QA activities conducted by Battelle QA staff will be an audit of data quality.
This audit will consist of a review by the Battelle Quality Manager (or his designee) of at least
10% of the test data. During the course of any such audit, the Battelle Quality Manager will
inform the technical staff of any findings and any need for immediate corrective action. If serious
data quality problems exist, the Battelle Quality Manager will request that Battelle's AMS
Center Manager issue a stop work order. Once the assessment report has been prepared, the
Verification Test Coordinator will ensure that a response is provided for each adverse finding or
potential problem, and will implement any necessary follow-up corrective action. The Battelle
Quality Manager will ensure that follow-up corrective action has been taken.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 3 8 of 44
Version 1.0
Date: 1/30/2007
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in this verification test to anticipate and resolve potential
problems before the quality of performance is compromised. One of the major objectives of this
test/QA plan is to establish mechanisms necessary to ensure this. Internal quality control
measures described in this test/QA plan, which is peer reviewed by a panel of outside experts,
implemented by the technical staff and monitored by the Verification Test Coordinator, will give
information on data quality on a day-to-day basis. The responsibility for interpreting the results
of these checks and resolving any potential problems resides with the Verification Test
Coordinator. Technical staff have the responsibility to identify problems that could affect data
quality or the ability to use the data. Any problems that are identified will be reported to the
Verification Test Coordinator, who will work with the Battelle Quality Manager to resolve any
issues. Action will be taken to control the problem, identify a solution to the problem, and
minimize losses and correct data, where possible. Independent of any EPA QA activities,
Battelle will be responsible for ensuring that the following audits are conducted as part of this
verification test.
Cl.l Performance Evaluation Audits
A Performance Evaluation (PE) audit will be conducted to assess the quality of the
reference method measurements made in this verification test. The reference method PE audit
will be performed by supplying an independent, NIST-traceable solid standard, if available, or at
a minimum a second quinine sulfate standard solution prepared from a different source of
quinine sulfate than that used in verification testing. The PE audit samples will be analyzed in
the same manner as all other samples and the analytical results for the PE audit samples will be
compared to the nominal concentration. The target criterion for this PE audit is agreement of the
analytical result within 3% of the nominal concentration, after the data have been adjusted for the
Overall Instrument Sensitivity based on the observation of the naphthalene-anthracene standard.
If the PE audit result does not meet the target criterion, the PE audit will be repeated. If the
outlying results persist, the source of error will be investigated and corrective action taken as
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 3 9 of 44
Version 1.0
Date: 1/30/2007
necessary until successful PE audit results are obtained. This audit will be performed once prior
to the start of the test, and will be the responsibility of the Verification Test Coordinator or
designee.
C1.2 Technical Systems Audits
The Battelle Quality Manager or designee will perform a TSA at least once during this
verification test. The purpose of this audit is to ensure that the verification test is being
performed in accordance with the AMS Center QMP1, this test/QA plan, any published reference
methods, and any Standard Operating Procedures (SOPs) used. In the TSA, the Battelle Quality
Manager, or a designee, may review the reference methods used, compare actual test procedures
to those specified or referenced in this plan, and review data acquisition and handling
procedures. In the TSA, the Battelle Quality Manager will tour the laboratory where verification
and reference testing are taking place, inspect sample COC documentation, and review
technology-specific record books. He or she will also check standard certifications and
technology data acquisition procedures, and may confer with the technology vendors and
technical staff. A TSA report will be prepared, including a statement of findings and the actions
taken to address any adverse findings. The EPA AMS Center Quality Manager will receive a
copy of Battelle's TSA report. At EPA's discretion, EPA QA staff may also conduct an
independent on-site TSA during the verification test. The TSA findings will be communicated to
technical staff at the time of the audit and documented in a TSA report.
C1.3 Data Quality Audits
The Battelle Quality Manager or his designee will audit at least 10% of the verification
data acquired in the verification test. The Battelle Quality Manager will trace the data from initial
acquisition, through reduction and statistical comparisons, to final reporting. All calculations
performed on the data undergoing the audit will be checked.
C1.4 QA/QC Reporting
Each assessment and audit will be documented in accordance with Section 3.3.4 of the
AMS Center QMP.1 The results of the technical systems audit will be submitted to EPA.
Assessment reports will include the following:
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 40 of 44
Version 1.0
Date: 1/30/2007
Identification of any adverse findings or potential problems
Response to adverse findings or potential problems
Recommendations for resolving problems
Confirmation that solutions have been implemented and are effective
Citation of any noteworthy practices that may be of use to others.
C2 REPORTS TO MANAGEMENT
The Battelle Quality Manager, during the course of any assessment or audit, will identify
to the technical staff performing experimental activities any immediate corrective action that
should be taken. If serious quality problems exist, the Battelle Quality Manager is authorized to
request that Battelle's AMS Center Manager issue a stop work order. Once the assessment report
has been prepared, the Verification Test Coordinator will ensure that a response is provided for
each adverse finding or potential problem and will implement any necessary follow-up corrective
action. The Battelle Quality Manager will ensure that follow-up corrective action has been taken.
The test/QA plan and final report are reviewed by EPA AMS Center QA staff and EPA AMS
Center program management staff. Upon final review and approval, both documents will then be
posted on the ETV website (www.epa.gov/etv).
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 41 of 44
Version 1.0
Date: 1/30/2007
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are stated in Section BIO of
this test/QA plan. In general, the data review requirements specify that the data generated during
this test will be reviewed by a Battelle technical staff member within two weeks of data
generation. The reviewer will be familiar with the technical aspects of the verification test, but
will not be the person who generated the data. This process will serve both as the data review
and the data verification, and will ensure that data have been recorded, transmitted, and
processed properly. Furthermore, this process will ensure that the BWE screening tool data and
the reference method data are collected under appropriate testing conditions and that the
reference method data meet the reference method specifications.
The data validation requirements for this test involve a data quality assessment relative to
the DQIs and audit acceptance criteria specified for this test. The DQIs listed in Section A 7.1
and B5 will be used to validate the data quality. The QA audits described within Section C of
this document, including the performance evaluation audit and data quality audit, are designed to
validate the data quality.
D2 VALIDATION AND VERIFICATION METHODS
Data verification is conducted as part of the data review, as described in Section BIO for
this test/QA plan. A visual inspection of handwritten data will be conducted to ensure that all
entries were properly recorded or transcribed, and that any erroneous entries were properly noted
(i.e., single line through the entry with an error code and the initials of the recorder and date of
entry). Electronic data from the technologies and other instruments used during the test will be
inspected to ensure proper transfer from the datalogging system. Data manually incorporated
into spreadsheets for use in calculations will be checked against handwritten data to ensure that
transcription errors have not occurred. All calculations used to transform the data will be
reviewed to ensure the accuracy and the appropriateness of the calculations. Calculations
performed manually will be reviewed and repeated using a handheld calculator or commercial
software (e.g., Excel). Calculations performed using standard commercial office software (e.g.,
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 42 of 44
Version 1.0
Date: 1/30/2007
Excel) will be reviewed by inspecting the equations used in calculations and verifying selected
calculations by handheld calculator. Calculations performed using specialized commercial
software (i.e., for analytical instrumentation) will be reviewed by inspection and, when feasible,
verified by handheld calculator, or standard commercial office software.
To ensure that the data generated from this test meet the goals of the test, a number of
data validation procedures will be performed. Section C of this test/QA plan provides a
description of the validation safeguards employed for this verification test. Data validation and
verification efforts include the completion of QC activities and the performance of ISA and PE
audits as described in Section C. The data from this test will be evaluated relative to the
measurement DQIs described in Section A7.1 and B5, and the PE audit acceptance criteria given
in Section Cl. 1 of this test/QA plan. Data failing to meet these criteria will be flagged in the data
set and not used for evaluation of the technologies, unless these deviations are accompanied by
descriptions of their potential impacts on the data quality.
A data quality audit will be conducted by the Battelle Quality Manager to ensure that data
review, verification, and validation procedures were completed, and to assure the overall data
quality.
D3 RECONCILIATION WITH USER REQUIREMENTS
The purpose of a verification test performed following this test/QA plan is to evaluate the
performance of commercial technologies which screen for ballast water exchange by measuring
CDOM. This test evaluates the CDOM measurement method capability only and is not a
verification of whether or not the measurement quantitatively measures CDOM or detects ballast
water exchange. This evaluation will include comparisons of the results from the technologies to
results from standard reference techniques. To meet the requirements of the user community, the
data obtained in such a verification test will include thorough documentation of the technology's
performance during the verification test. The data review, verification, and validation
procedures described above will assure that verification test data meet these requirements, are
accurately presented in the verification reports generated from the test, and that data not meeting
these requirements are appropriately flagged and discussed in the verification reports.
Additionally, all data generated using the reference method, which are used to evaluate
technology results during the verification test, should meet the QA requirements of any
applicable standard operating procedures or instrumentation instruction manuals.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 43 of 44
Version 1.0
Date: 1/30/2007
This test/QA plan and any resulting ETV verification report(s) generated following
procedures described in this test/QA plan will be subjected to review by participating technology
vendors, ETV AMS Center staff, test collaborators, EPA, and external expert peer reviewers.
These reviews will assure that this test/QA plan, verification test(s) of BWE screening tools, and
the resulting report(s) meet the needs of potential users and regulators. The final report(s) will be
submitted to EPA in 508 compliant Adobe Portable Document Format (pdf) and subsequently
posted on the ETV website.
-------
Ballast Water Exchange Screening Tools
Test/QA Plan
Page 44 of 44
Version 1.0
Date: 1/30/2007
SECTION E
REFERENCES
El REFERENCES
1. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center, U.S.
EPA Environmental Technology Verification Program, prepared by Battelle, Columbus,
Ohio, Version 6.0, November 2005.
2. "Environmental Technology Verification Program Quality Management Plan" (QMP),
December 2002, EPA/600/R-03/021.
3. Hunt, Carlton D., Tanis, Deborah, Bruce, Elizabeth, and Taylor, Michael. In Press. Optical
Signatures of Seawater and their Potential Use in Verification of At Sea Ballast Water
Exchange Marine Ecology and Progress Series.
4. Murphy, K., Ruiz, G., Dunsmuir, W., Waite, T. 2006. Optimized Parameters for
Fluorescence-Based Verification of Ballast Water Exchange by Ships. Environ. Sci.
Technol., 40, 2357-2362.
5. Murphy, K., Boehme, J., Coble, P., Cullen, J., Field, P., Moore, W., Perry, E., Sherrell, R.,
and Ruiz, G. 2004. Verification of mid-ocean ballast water exchange using naturally
occurring coastal tracers. Marine Pollution Bulletin, 48, 711-730.
6. ASTM E579-04, "Standard Test Method for Limit of Detection of Fluorescence of Quinine
Sulfate in Solution," ASTM International.
7. Code of Federal Regulations, Title 40, Part 136, Appendix B, Definition and Procedure for
the Determination of the Method Detection Limit-Revision 1.11.
------- |