Baneiie
The Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Test/Quality Assurance Plan
for Verification of
Microcystin Test Kits
-------
TEST/QUALITY ASSURANCE PLAN
for
Verification of
Microcystin Test Kits
July 6,2010
Prepared by
Ryan James
Anne Gregg
Amy Dindal
Battelle
505 King Avenue
Columbus, OH 43201-2693
-------
Microcystin Test Kits
Test/QA Plan
Page 3 of 46
Version 1.0
Date: 7/6/10
SECTION A
PROJECT MANAGEMENT
Al VENDOR APPROVAL PAGE
ETV Advanced Monitoring Systems Center
Test/Quality Assurance Plan
for Verification of
Microcystin Test Kits
APPROVAL:
Name
Company
Date
-------
Microcystin Test Kits
Test/QA Plan
Page 4 of 46
Version 1.0
Date: 7/6/10
A2 TABLE OF CONTENTS
Section Page
SECTION A PROJECT MANAGEMENT 3
Al Vendor Approval Page 3
A2 Table of Contents 4
A3 Distribution List 6
A4 List of Acronyms and Abbreviations 7
A5 Verification Test Organization 8
A6 Background 15
A7 Verification Test Description and Schedule 17
A8 Quality Objectives and Criteria for Measurement Data 20
A9 Special Training/Certification 21
A10 Documentation and Records 22
SECTION B MEASUREMENT AND DATA ACQUISITION 24
Bl Experimental Design 24
B2 Sampling Method Requirements 31
B3 Sample Handling and Custody Requirements 32
B4 Laboratory Reference Method 33
B5 Quality Control requirements 34
B6 Instrument/Equipment Testing, Inspection, and Maintenance 35
B7 Calibration Verification 35
B8 Inspection/Acceptance of Supplies and Consumables 35
B9 Non-Direct Measurements 36
BIO Data Management 36
SECTION C ASSESSMENT AND OVERSIGHT 38
Cl Assessments and Response Actions 38
C2 Reports to Management 42
SECTION D DATA VALIDATION AND USABILITY 43
Dl Data Review, Validation, and Verification Requirements 43
D2 Validation and Verification Methods 43
D3 Reconciliation with User Requirements 43
SECTION E REFERENCES 44
El References 44
-------
Microcystin Test Kits
Test/QA Plan
Page 5 of 46
Version 1.0
Date: 7/6/10
List of Figures Page
Figure 1. Organization Chart for the Verification Test 9
List of Tables
Table 1. Microcystin Variants 16
Table 2. Proposed Verification Test Schedule 19
Table 3. DQIs and Criteria for Critical Measurements for Reference Methods 21
Table 4. Summary of Data Recording Process 23
Table 5. Preparation of PT Samples 27
Table 6. Summary of Test Samples 28
Table 7. Microcystin Reference Method Information 34
Appendix A: Ease of Use Questionnaire
-------
A3 DISTRIBUTION LIST
Microcystin Test Kits
Test/QA Plan
Page 6 of 46
Version 1.0
Date: 7/6/10
Vendors
Fernando Rubio
Abraxis
54 Steam whistle Drive
Warminster, PA 18974
Art Trombley
Beacon Analytical Systems, Inc.
383 Presumpscot Street
Portland, Maine 04103
Elena Dominguez
ZEU-INMUNOTEC, S.L.
C/ Bari 25 dpdo. Poligono PLAZA.
50197- Zaragoza- SPAIN
EPA
John McKernan
Michelle Henderson
U.S. Environmental Protection Agency
26 West Martin Luther King Drive
Mail code: 208
Cincinnati OH 45268
Battelle
Ryan James
Anne Gregg
Amy Dindal
Zachary Willenberg
Battelle
505 King Ave.
Columbus, OH 43201
Verification Test Stakeholders
David Schumacher
Nebraska Department of Environmental
Quality
1200 "N" Street, Suite 400
P.O. Box 98922
Lincoln, Nebraska 68509
Robert Waters
Suffolk County Department of Health
Services
360 Yaphank Avenue Suite 2 B
Yaphank,NY119
Andrew Lincoff
US EPA Region 9 Lab
1337 S. 46th St., Bldg. 201
Richmond, CA 94804
Subcontractor
Daniel Snow
Water Center
202 Water Sciences Laboratory
University of Nebraska
Lincoln, NE 68583-0844 USA
-------
Microcystin Test Kits
Test/QA Plan
Page 7 of 46
Version 1.0
Date: 7/6/10
A4 LIST OF ACRONYMS AND ABBREVIATIONS
AMS Advanced Monitory Systems
ASTM American Society for Testing and Materials
COC chain of custody
CAS Chemical Abstract Service
CCV continuing calibration verification
CR cross reactivity
DQI data quality indicator
DQO data quality obj ective
DI deionized
DL detection limit
EPA Environmental Protection Agency
ETV Environmental Technology Verification
ELISA Enzyme-Linked Immunosorbent Assay
DDL instrument detection limit
LFM laboratory fortified matrix
LRB laboratory record book
LC-MS-MS liquid chromatography tandem mass spectrometry
MB method blank
MDL method detection limit
MRM multiple reaction monitoring
NIST National Institute of Standards and Technology
NRC National Research Council
NDEQ Nebraska Department of Environmental Quality
ppb parts per billion
%D percent different
PEA performance evaluation audit
PT performance test
PO project officer
QA quality assurance
QAM quality assurance manger
QAO quality assurance officer
QC quality control
QMP quality management plan
RMO records management office
RW recreational water
RPD relative percent different
RSD relative standard deviation
SOP standard operating procedure
SCDHS Suffolk County Department of Health Services
TSA technical systems audit
WSL Water Sciences Laboratory
-------
Microcystin Test Kits
Test/QA Plan
Page 8 of 46
Version 1.0
Date: 7/6/10
A5 VERIFICATION TEST ORGANIZATION
The verification test will be conducted under the U.S. Environmental Protection Agency
(EPA) Environmental Technology Verification (ETV) Program. It will be performed by
Battelle, which is managing the ETV Advanced Monitoring Systems (AMS) Center through a
cooperative agreement with EPA. The scope of the AMS Center covers verification of
monitoring technologies for contaminants and natural species in air, water, and soil.
The day to day operations of this verification test will be coordinated and performed by
Battelle, with the participation of vendors who will have the performance of their technologies
verified. Testing will be conducted by Battelle staff at the Battelle laboratories in Columbus,
Ohio. Each vendor will provide Battelle with their respective technologies and will train the
verification staff in their technologies use.
Quality Assurance (QA) oversight will be provided by the Battelle Quality Assurance
Manager (QAM) and also by the EPA AMS Center Quality Manager (EPA QM), at her
discretion. The organization chart in Figure 1 identifies the responsibilities of the organizations
and individuals associated with the verification test. Roles and responsibilities are defined
further below.
-------
Microcystin Test Kits
Test/QA Plan
Page 9 of 46
Version 1.0
Date: 7/6/10
Battelle
Management
Amy Dindal
Battelle AMS
Center Manager
Rosanna Buhl
Battelle Quality
Assurance Manager
Zachary Willenberg
Battelle Quality
Assurance Officer
Nebraska
Department of
Environmental
Quality
Suffolk County
Department of
Health Services
John McKernan
EPA AMS Center
Project Officer
Michelle Henderson
EPA AMS Center
Quality Manager
Ryan James
Battelle
Verification
Test Coordinator
and
Battelle
Technical Staff
Microcystin Test Kit
Vendor
Representatives
University
of Nebraska,
Water Sciences
Laboratory
Figure 1. Organization Chart for the Verification Test
-------
Microcystin Test Kits
Test/QA Plan
Page 10 of 46
Version 1.0
Date: 7/6/10
A5.1 Battelle
Dr. Ryan James is the AMS Center's Verification Test Coordinator for this test. In this
role, Dr. James will have overall responsibility for ensuring that the technical, schedule, and cost
goals established for the verification test are met. Specifically, Dr. James will:
• Prepare the draft test/QA plan, verification reports, and verification statements.
• Establish a budget for the verification test and manage staff to ensure the budget is
not exceeded.
• Revise the draft test/QA plan, verification reports, and verification statements in
response to reviewer comments.
• Assemble a qualified technical staff to conduct the verification test.
• Direct the staff in performing the verification test in accordance with this test/QA
plan.
• Hold a kick-off meeting approximately one week prior to the start of the verification
test to review the critical logistical, technical, and administrative aspects of the
verification test. Responsibility for each aspect of the verification test will be
confirmed.
• Ensure that all quality procedures specified in this EPA Quality Level III test/QA
plan and in the AMS Center Quality Management Plan1 (QMP) are followed.
• Serve as the primary point of contact for vendor representatives.
• Ensure that confidentiality of sensitive vendor information is maintained.
• Assist vendors as needed during verification testing.
• Become familiar with the operation and maintenance of the technologies through
instruction by the vendors.
• Respond to any issues raised in assessment reports, audits, or from verification staff
observations, and institute corrective action as necessary.
• Coordinate distribution of the final test/QA plan, verification reports, and verification
statements.
Ms. Amy Dindal is Battelle's Manager for the AMS Center. As such, Ms. Dindal will
oversee the various stages of verification testing. Ms. Dindal will:
-------
Microcystin Test Kits
Test/QA Plan
Page 11 of 46
Version 1.0
Date: 7/6/10
• Review the draft and final test/QA plan.
• Review the draft and final verification reports and verification statements.
• Ensure that necessary Battelle resources, including staff and facilities, are committed
to the verification test.
• Ensure that confidentiality of sensitive vendor information is maintained.
• Support Dr. James in responding to any issues raised in assessment reports and audits.
• Maintain communication with EPA's technical and quality managers.
• Issue a stop work order if Battelle or EPA QA staff discovers adverse findings that
will compromise test results.
Battelle Technical Staff will support Dr. James in planning and conducting the
verification test. The responsibilities of the technical staff will be to:
• Assist in planning for the test, and making arrangements for the receipt of and
training on the technologies.
• Attend the verification test kick-off meeting.
• Assist vendor staff as needed during test kit receipt and training.
• Coordinate and conduct verification testing using each participating technology,
following all aspects of the ETV AMS Center QMP1 as well as the test/QA plan for
this verification.
• Support Dr. James in the preparation of the test/QA plan and reports, as necessary.
• Support Dr. James in responding to any issues raised in assessment reports and audits
related to statistics and data reduction as needed.
Ms. Rosanna Buhl is Battelle's Quality Assurance Manager (QAM) for the AMS Center. Ms.
Buhl will:
• Review the draft and final test/QA plan.
• Assign a Quality Assurance Officer (QAO) for each verification test.
• Delegate to other Battelle quality staff any QAO responsibilities assigned below as
needed to meet project schedules.
• Review any audit checklists prepared by the QAO for completeness and detail.
-------
Microcystin Test Kits
Test/QA Plan
Page 12 of 46
Version 1.0
Date: 7/6/10
• Review draft audit reports prior to release to the Verification Test Coordinator and/or
EPA for clarity and appropriate assessment of findings.
• Review audit responses for appropriateness.
• Review and approve test/QA plans, test/QA plan amendments, deviations and audit
reports.
• Maintain real-time communication with the QAO on QA activities, audit results, and
concerns.
• Work with the QAO, Verification Test Coordinator, and Battelle's AMS Center
Manager to resolve data quality concerns and disputes.
• Recommend a stop work order if audits indicate that data quality or safety is being
compromised.
Mr. Zachary Willenberg is Battelle's QAO for this test. Mr. Willenberg will:
• Attend the verification test kick-off meeting and lead the discussion of the QA
elements of the kickoff meeting checklist.
• Prior to the start of verification testing, verify the presence of applicable training
records, including any vendor training on test equipment.
• Conduct a technical systems audit at least once during the verification test.
• Conduct audits to verification data quality.
• Prepare and distribute an audit report for each audit.
• Verify that audit responses for each audit finding and observation are appropriate and
that corrective action has been implemented effectively.
• Communicate to the Verification Test Coordinator and/or technical staff the need for
immediate corrective action if an audit identifies test/QA plan deviations or practices
that threaten data quality.
• Provide a summary of the QA/QC activities and results for the verification reports.
• Review the draft and final verification report(s) and verification statement(s).
• Maintain real-time communication with the Battelle QAM on QA activities, audit
results, and concerns, including potential schedule and budget problems.
-------
Microcystin Test Kits
Test/QA Plan
Page 13 of 46
Version 1.0
Date: 7/6/10
Communicate data quality concerns to the Verification Test Coordinator and/or
Battelle's AMS Center QAM and Manager; recommend the need for a stop work
order if audits indicate that data quality or safety is being compromised.
A5.2 Technology Vendors
The responsibilities of the technology vendors are as follows:
Review and provide comments on the draft test/QA plan.
Accept (by signature of a company representative) the final test/QA plan prior to test
•
•
initiation
•
•
Provide their technology for evaluation during the verification test.
Provide all other equipment/supplies/reagents/consumables needed to operate their
technology for the duration of the verification test.
Supply training on the use of the technology, and provide written consent and
instructions for verification staff to carry out testing, including written instructions for
routine operation of their technology.
Provide maintenance and repair support for their technology, on-site if necessary,
throughout the duration of the verification test.
Review and provide comments on the draft verification report and statement for their
respective technology.
A5.3 EPA
EPA's responsibilities in the AMS Center are based on the requirements stated in the
"Environmental Technology Verification Program Quality Management Plan" (ETV QMP)2.
The roles of specific EPA staff are as follows:
Ms. Michelle Henderson is EPA's AMS Center Quality Manager (EPA QM). For the
verification test, Ms. Henderson will:
• Review the draft test/QA plan.
• Perform at her option one external technical systems audit during the verification test.
• Notify the EPA AMS Center Project Officer of the need for a stop work order if the
external audit indicates that data quality is being compromised.
-------
Microcystin Test Kits
Test/QA Plan
Page 14 of 46
Version 1.0
Date: 7/6/10
• Prepare and distribute an assessment report summarizing results of the external audit.
• Review draft verification reports and verification statements.
Dr. John McKernan is EPA's Project Officer (EPA PO) for the AMS Center. Dr.
McKernan, or designee will:
• Review the draft test/QA plan.
• Approve the final test/QA plan.
• Be available during the verification test to authorize any test/QA plan deviations by
phone and provide the name of a delegate to the Battelle AMS Center Manager
should he not be available during the testing period.
• Review the draft verification reports and verification statements.
• Oversee the EPA review process for the test/QA plan, verification reports, and
verification statements.
• Coordinate the submission of verification reports and verification statements for final
EPA approval.
• Post the test/QA plan, verification reports, and verification statements on the ETV
web site.
A5.4 Verification Test Stakeholders
This verification test will be conducted in collaboration with David Schumacher and his
technical staff at the Nebraska Department of Environmental Quality (NDEQ). They have
provided recreational water (RW) samples for verification testing. In addition, the Suffolk
County Department of Health Services (SCDHS) is collaborating on this verification test by
providing RW samples.
This test/QA plan and the verification report(s) and verification statement(s) based on
testing described in this document will be reviewed by experts in the fields related to microcystin
determination in water. The following experts have been providing input to this test/QA plan
and have agreed to provide a peer review:
• Robert Waters, Suffolk County Department of Health
• Andrew Lincoff, US EPA Region 9 Laboratories
-------
Microcystin Test Kits
Test/QA Plan
Page 15 of 46
Version 1.0
Date: 7/6/10
The activities responsibilities of verification test stakeholders and/or peer reviewers include:
• Participate in stakeholder discussions to provide input to the test design.
• Review and provide input to the test/QA plan
• Review and provide input to the verification report(s)/verification statement(s).
The AMS Center Water Stakeholder Committee has considered the technology category of
microcystin immunoassay kits a priority area since 2005. The Battelle Verification Test
Coordinator presented the fundamentals of the test design in a stakeholder committee
teleconference in November 2009 to gather input from the stakeholders on the approach.
A5.5 University of Nebraska Water Sciences Laboratory (WSL)
Mr. Daniel Snow and his technical staff at the WSL will:
• Perform the reference analyses by solid phase extraction liquid chromatography
tandem mass spectrometry (LC-MS-MS). This will include all necessary QC
requirements, such as performance evaluation audit (PEA) samples to confirm the
accuracy of the reference method prior to testing.
A6 BACKGROUND
A6.1 Technology Need
The ETV Program's AMS Center conducts third-party performance testing of
commercially available technologies that detect or monitor natural species or contaminants in air,
water, and soil. The purpose of ETV is to provide objective and quality assured performance
data on environmental technologies, so that users, developers, regulators, and consultants can
make informed decisions about purchasing and applying these technologies. The ETV Water
Stakeholder Committee, made up of buyers and users of such technologies recommend
technology categories, and technologies within those categories, as priorities for testing. Among
the technology categories recommended for testing are microcystin test kits. In particular, the
use of microcystin test kits for the monitoring of recreational waters was identified as an area of
interest for technology verification.
Microcystins are compounds (nonribosomal peptides) produced by cyanobacteria, also
known as blue-green algae, which may pose a significant threat to human and animal health.
-------
Microcystin Test Kits
Test/QA Plan
Page 16 of 46
Version 1.0
Date: 7/6/10
Exposure to microcystin could result in skin rashes, eye irritations, respiratory symptoms, and
liver damage3. While alive, the toxins are contained inside the bacterial cell; however, when
these cells become damaged or die (lysis), the toxins are released into the water. There are
approximately 80 structural variants3 (also called congeners) of microcystin that have been
identified as highly toxic. The most common and most extensively studied variant is
microcystin-LR. The World Health Organization (WHO) has set a provisional drinking-water
guideline value of 1 microgram/liter (ug/L) for microcystin-LR4. For recreational use, the WHO
Guideline for Safe Recreational Water Environments is 20 ug/L, 20 times the drinking water
guideline concentration for microcystin-LR4. In addition to microcystin-LR, microcystin-LA
and -RR will also be used to test the performance of the microcystin test kits. Microcystin-LA
is highly toxic and microcystin-RR is more prevalent in the environment but not as toxic3. Table
1 shows the microcystin variants and their respective amino acid identifiers to be analyzed by the
test kits.
Table 1. Microcystin Variants3
Name
Microcystin-LR
Microcystin-LA
Microcystin-RR
Variable Amino Acids
Leucine (L)
Leucine (L)
Arginine (R)
Arginine (R)
Alanine (A)
Arginine (R)
Molecular Weight
995.17
910.06
1038.2
A6.2 Technology Description
Microcystin test kits are used to quantitatively measure total microcystin in recreational
waters. These test kits are based on enzyme-linked immunosorbent assays (ELISA) with
antibodies that bind specifically to microcystins or phosphate activity inhibition where the
phosphatase hydrolyzes to determine the total toxicity of microcystin present in the sample. The
kits report total microcystin or total toxicity in a water sample and therefore do not differentiate
between the different variants of microcystin. Microcystin concentrations are indicated by a
color measurement that is inversely proportional to the concentration of the total microcystins in
the sample, that is, the color disappears if microcystin is present. The color change of the test
kits is calibrated against microcystin-LR standards provided with the test kits. Variants bind
differently to the immunosorbent resulting in different cross reactivity (CR) for the variants. The
CR for specific variants are determined by the vendor and reported in the instructions manual.
Kits are available in multiple formats, including 96-well microplates, tube assays, and test strips.
-------
Microcystin Test Kits
Test/QA Plan
Page 17 of 46
Version 1.0
Date: 7/6/10
The 96-well microplates provide quantitative results when they are used in conjunction with a
spectrophotometric plate reader set at 450 nanometers for the ELISA kits and 405 nanometers for
the toxicity test kit. The tube assays can also provide quantitative results when used with a
single-cell spectrophotometer. Both of those types of kits may also provide semi-quantitative
results, using visual comparison of the color change. The photometer data are then reduced
either by manual calculations or by a data reduction program. The test strip kits are semi-
quantitative, indicating distinct colors for specified ranges of microcystin concentrations.
A7 VERIFICATION TEST DESCRIPTION AND SCHEDULE
This verification test will assess the performance of the test kits relative to key
verification parameters including accuracy, precision, and method detection limit. Correct
preparation of test solutions will be confirmed through a comparison to reference method results.
In performing the verification test, Battelle will follow the technical and QA procedures
specified in this test/QA plan and will comply with the data quality requirements in the AMS
Center QMP1.
A.7.1 Verification Test Description
The objective of this verification test is to evaluate the microcystin test kit performance
against known concentrations of microcystin in DI water, as well as against natural recreational
water samples. The test will be performed in collaboration with the NDEQ and the SCDHS.
Technologies undergoing verification will be used to analyze a variety of water samples for the
variants: microcystin-LR, microcystin-LA, and microcystin-RR. The quantitative results from
the microcystin test kits will be compared to the results from the reference method by calculating
percent differences between the results. The kits provide a quantitative or semi-quantitative
determination of microcystin and will be evaluated in terms of:
• Accuracy - comparison of test kit results (samples prepared in ASTM Type II
deionized water (DI) as well as RW samples) to results from a reference method
• Precision - repeatability of test kit results from three sample replicates analyzed in DI
water and recreational waters
-------
Microcystin Test Kits
Test/QA Plan
Page 18 of 46
Version 1.0
Date: 7/6/10
• Linearity - determination of whether or not the test kit response increases in direct
proportion to the known concentration of microcystin
• Method detection limit - the lowest quantity of toxin that can be distinguished from
the absence of that toxin (a blank value) at a 99% confidence level
• Inter-kit lot reproducibility - determination of whether or not the test kit response is
significantly different between two different lots of calibration standards within the
kits.
• Matrix Interference - evaluation of the effect of natural recreational matrices and
chlorophyll-a on the results of the test kits.
• Operational factors - general operation, data acquisition, set-up, consumables, etc.
Subsequent to the verification test, verification reports describing the test will be drafted.
These reports will be reviewed by the vendor and by peer reviewers, revised, and submitted to
EPA for final approval. In performing the verification test, Battelle will follow the technical and
QA procedures specified in this test/QA plan and will comply with the data quality requirements
in the AMS Center QMP.1
A.7.2 Proposed Verification Test Schedule
Table 2 shows the proposed schedule of testing, auditing, and data analysis/reporting
activities to be conducted during this verification. The performance evaluation audit (PEA) will
take place before testing begins. The verification of microcystin test kits is planned to be
completed over the course of a week after the PEA data are received. The verification test is
expected to be conducted in July 2010. The technical systems audit (TSA) will take place during
testing and the audit of data quality (ADQ) will take place after the data are reviewed by the
Verification Test Coordinator, or designee.
-------
Microcystin Test Kits
Test/QA Plan
Page 19 of 46
Version 1.0
Date: 7/6/10
Table 2. Proposed Verification Test Schedule
Approximate Date(s)
July-August 2009
February-July 20 10
June- July 20 10
July 20 10
July-August 20 10
August-September 2010
September 20 10
Testing Activities
Recreational water sampling in
Nebraska and New York
Test/QA plan design and approval
Perform Performance Evaluation Audit
Verification testing
Perform Technical Systems Audit
Reference analysis
Perform Audit of Data Quality
Prepare draft verification reports and
statements
Coordinate reviews of draft verification
reports and statements
Prepare final verification reports and
statements
Data Analysis and Reporting
Not applicable
Not applicable
Compile PEA reference method results
Prepare report template
Compile data from test kits
Review and summarize testing staff
observations
Compile reference method results
Begin draft reports
Perform data analysis
Complete draft verification reports and
statements
Complete peer review and vendor review
of draft reports
Revise draft verification reports and
statements
Submit final reports for EPA approval
A7.3 Test Facility
This verification test will take place in Columbus, Ohio at the Battelle laboratories.
Recreational water samples were collected from nine local lakes in Nebraska by the NDEQ or in
New York by the SCDHS. All samples were collected and frozen prior to testing in at least 120
milliliter volumes.
A7.4 Health and Safety
Battelle will conduct all verification testing following the safety and health protocols in
place for the Battelle laboratory and facilities. This includes maintaining a safe work
environment and a current awareness of handling potentially toxic chemicals. Exposure to
potentially toxic chemicals will be minimized, personal protective equipment will be worn, and
safe laboratory practices will be followed.
-------
Microcystin Test Kits
Test/QA Plan
Page 20 of 46
Version 1.0
Date: 7/6/10
A8 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA
In performing the verification test, Battelle will follow the technical and QA procedures
specified in this test/QA plan and will comply with the data quality requirements in the AMS
Center QMP.1 QA level III, Applied Research has been specified for this test by the EPA Project
Officer.
To ensure that this verification test provides suitable data for a robust evaluation of
performance, a variety of data quality objectives (DQOs) have been established for this test. The
DQOs indicate the minimum quality of data required to meet the objectives of the verification
test. The DQOs for this verification test were established to assess the performance of the
microcystin test kits relative to reference measurements. In order to provide a suitable
benchmark for comparison, the reference measurements must meet the DQOs. The DQOs for
this verification test include specific objectives for reference method measurements and data
completeness. The DQOs are quantitatively defined in Table 3 in terms of specific data quality
indicators (DQIs) and their acceptance criteria.
The quality of the reference method measurements will be assured by adherence to these
DQI criteria and the requirements of the reference methods, including the calibration and QA/QC
requirements of the method. Blank samples will be required to generate results below the
detection limit and the Laboratory Fortified Matrix (LFM) sample and PEA sample results will
be required to be within 30% of the expected results. Prior to testing, Battelle's QAO will
contact the reference laboratory and request submission of that laboratory's QA plan and
associated records. In addition, Battelle will visit the reference laboratory and audit the QA
document associated with the samples analyzed during this ETV test. More details about the QC
requirements for the reference method are given in Section B5.
PEA samples will be used to independently confirm the accuracy of the reference
measurements. Before testing begins, standards will be diluted to a concentration within the
measureable range of the reference method and sent to the reference laboratory for analysis.
Currently, National Institute of Standards and Technology (NIST) traceable certified microcystin
standards are not available on the market. However, the Canadian National Research Council
(NRC), Institute for Marine Biosciences is in the process of certifying microcystin-LR and -RR.
These standards will be obtained and diluted for the PEA. In addition, standards of microcystin-
LR, RR, and -LA will also be obtained from Abraxis and sent for reference analysis.
-------
Microcystin Test Kits
Test/QA Plan
Page 21 of 46
Version 1.0
Date: 7/6/10
Table 3. DQIs and Criteria for Critical Measurements for Reference Methods.
DQI
Performance
Evaluation Audit
(PEA)
Method
contamination
check
Method
Calibration
Check
Method precision
Method accuracy
Method of
Assessment
PEA Samples
Method Blank (MB)
Continuing Calibration
Verification (CCV)
Laboratory Duplicates
Laboratory Fortified
Matrix (LFM) Spikes
Frequency
Once before
testing begins
Once every 20
samples
Once every 10
samples
Once every 20
samples
Once every 20
samples
Acceptance
Criteria
70% - 130%
recovery of
target analytes
Target analytes
< lowest
calibration
standard
80% - 120%
recovery of
target analytes
Target analytes
< 30%
Difference
70% -130%
recovery of
target analytes
Corrective Action
Review data to troubleshoot
results and adjust reference
method as necessary, reanalyze
samples.
Review data and analysis for
possible sources of
contamination. Reanalyze
and/or document corrective
action.
Review data to troubleshoot
results and adjust reference
method as necessary, reanalyze
samples.
Review data to assess impact of
matrix. If other QC data are
acceptable, then reprocess
duplicate. If not possible, then
flag associated reference
method data.
Review data to assess impact of
matrix. If other QC data are
acceptable, then reprocess
duplicate. If not possible, then
flag associated reference
method data.
The Battelle QAO or his designee will perform a TSA at least once during this
verification test to augment these QA/QC requirements. The EPA QM also may conduct an
independent TSA, at her discretion.
A9 SPECIAL TRAINING/CERTIFICATION
Documentation of training related to technology testing, data analysis, and reporting is
maintained for all Battelle technical staff in training files at their respective Battelle location.
The Battelle QAO will verify the presence of appropriate training records prior to the start of
testing. The vendors will be required to train the Battelle technical staff prior to the start of
testing. Battelle will document this training with a consent form, signed by the vendor, which
states which Battelle technical staff have been trained to use their test kits and can train other
staff. In the event that other staff members are required to use the test kits, they will be trained
by the operators that were trained by the vendors. All technical staff will have a minimum of a
-------
Microcystin Test Kits
Test/QA Plan
Page 22 of 46
Version 1.0
Date: 7/6/10
bachelor's degree in science/engineering or equivalent work experience (e.g., experience using
ELISA test kits).
Battelle will conduct all verification testing following the safety and health protocols in
place at the verification testing facilities. This includes maintaining a safe work environment
and a current awareness of handling potentially toxic chemicals. Exposure to potentially toxic
chemicals will be minimized, personal protective equipment will be worn, and safe laboratory
practices will be followed.
A10 DOCUMENTATION AND RECORDS
The documents and records for this verification test will include the test/QA plan,
laboratory record books (LRB), data collection forms, electronic files (both raw data and
spreadsheets), and the final verification report. Table 4 summarizes the types of data to be
recorded. Documentation of Battelle staff training by vendors and copies of other project
specific training will also be included in the project files. All of these records will be maintained
in the Verification Test Coordinator's office during the test and will be transferred to permanent
storage at Battelle's Records Management Office (RMO) at the conclusion of the verification
test. All Battelle LRBs are stored indefinitely with the project files, either by the Verification
Test Coordinator or Battelle's RMO. The raw and final results from the reference measurements
will be submitted to Battelle upon obtaining the results of the analyses. Section BIO further
details the data recording practices and responsibilities.
-------
Microcystin Test Kits
Test/QA Plan
Page 23 of 46
Version 1.0
Date: 7/6/10
Table 4. Summary of Data Recording Process
Data to Be
Recorded
Dates, times of test
events
Test parameters
Field sampling data
PEA sample records
Reference sample
data
Reference method
sample analysis,
chain of custody, and
results
Responsible
Party
Battelle
Battelle
NDEQ or SCDHS
Battelle and WSL
Battelle
Battelle and WSL
Where
Recorded
Laboratory record
books or data
collection sheets
Laboratory record
books or data
collection sheets
Laboratory record
books or data
collection sheets
Laboratory record
books or data
collection sheets
Laboratory record
books or data
collection sheets
Laboratory record
books, chain of
custody forms,
data collection
sheets, or data
acquisition system,
as appropriate
How often
recorded
Start/end of test, and
at each change of a
test parameter.
When set or changed,
or as needed to
document test
notable details during
testing
During each
sampling event
During sample
preparation and
analysis
When test samples
are aliquoted for the
reference analysis
Throughout sample
handling and analysis
process
Disposition of
Data (a)
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary.
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary.
Used to characterize
the recreational water
sample.
Used to verify the
performance of the
reference method
Used to organize/check
test results; manually
incorporated in data
spreadsheets as
necessary.
Transferred to
spreadsheets/agreed
upon report; project
files. Retained for
documentation of
reference method
performance.
(a) All activities subsequent to data recording are carried out by Battelle.
-------
Microcystin Test Kits
Test/QA Plan
Page 24 of 46
Version 1.0
Date: 7/6/10
SECTION B
MEASUREMENT AND DATA ACQUISITION
Bl EXPERIMENTAL DESIGN
Technologies undergoing verification will be used to analyze a variety of water samples
for the variants: microcystin-LR, microcystin-LA, and microcystin-RR. Where appropriate, the
quantitative results from the microcystin test kits will be compared to the results from the
reference method by calculating percent differences between the results. The kits provide a
quantitative or semi-quantitative determination of microcystin and will be evaluated in terms of:
• accuracy
• precision
• linearity
• method detection limit
• inter-kit lot reproducibility
• matrix effects, and
• operational factors.
Each microcystin test kit will be operated according to the vendor's instructions. This
includes kit provided calibration standards and positive and negative controls. The samples will
also be tested according to the kit instructions, i.e. samples and calibration standards analyzed in
duplicate and the frequency of positive and negative controls. Table 5 presents the test samples
to be analyzed during this verification test.
Bl.l Testing Procedures
The ability of each microcystin test kit to determine the concentration of microcystin will
be challenged using quality control (QC) samples, performance test (PT) samples and
recreational water (RW) samples. These sample results will also be compared to reference
method results. QC samples will include laboratory reagent blanks (RB). RB samples will be
prepared from DI water and will be exposed to identical handling and analysis procedures as
other prepared samples, including the addition of all reagents. These samples will be used to
help ensure that no sources of contamination are introduced in the sample handling and analysis
-------
Microcystin Test Kits
Test/QA Plan
Page 25 of 46
Version 1.0
Date: 7/6/10
procedures. At least 10% of all the prepared samples to be analyzed will be RBs. Other QC
samples, positive and negative controls, are included in this test from the test kit procedure.
PT samples will be used to help determine the accuracy, precision, linearity, method
detection limit, and inter-kit lot reproducibility of the test kits. All PT samples will be prepared
at Battelle using DI water as the water source. PT samples will be individually spiked with
microcystin-LR, microcystin-LA, and microcystin-RR. Additionally, solutions will be prepared
to assess the linearity over a concentration range and analyzed in triplicate. The concentration
levels will be 0.1, 0.5, 1.0, 2.0, and 4.0 parts-per billion (ppb) to test the dynamic range of the
test kits. These concentration levels will be used for microcystin-LR. Because of estimated CR
of the -LA and -RR microcystin congeners, a 7.0 ppb concentration level will also be included to
evaluate the dynamic range of the test kits for these two congeners. If applicable to the test kits
that participate in the verification test (i.e., a semi-quantitative test strip), a 15 ppb PT sample
will also be tested in order to test the semi-quantitative capability of indicating a concentration
higher than 10 ppb. To determine the detection limit of the quantitative test kits, a solution with
a concentration five times the vendor's reported detection limit (DL) will be used5. Seven
replicate analyses of this solution will be made individually for each variant to obtain precision
data with which to determine the method detection limit5. The detection limits for the
quantitative test kits being verified range from 0.1 to 0.3 ppb.
RW samples have been obtained from lakes in and around Lincoln, Nebraska and Suffolk
County, New York to assess kit performance in recreational waters. The RW samples have been
frozen and thawed three times to lyse the cyanobacteria followed by filtration. Then the sample
will be split for verification testing and reference analysis. The procedure for collecting and
preparing the samples for verification testing and reference analysis is described in Section B.2.
The NDEQ staff are aware of the approximate microcystin level of the lakes from which the
water samples that were collected. Using this information, the samples that will be used for
testing will be selected from lakes that are expected to have both detectable and not-detectable
microcystin concentrations. There will be at least nine RW samples used for this verification
test. Some of the samples will have been samples from Nebraska and New York. Ideally, three
of the RW samples will have microcystin concentration > 20 ppb, three RW samples will have
concentrations > 10 ppb, and three RW samples will have non-detectable (ND) concentrations of
microcystin. All RW samples will be tested in triplicate by the test kits.
-------
Microcystin Test Kits
Test/QA Plan
Page 26 of 46
Version 1.0
Date: 7/6/10
The test kits with specific vendor recommended lysing procedures will analyze three
additional RW samples in triplicate. These test samples will not undergo the three iterations of
the freeze-thaw lysing procedure. They will be vigorously shaken but not mechanically
homogenized. The cyanobacteria need to remain intact but also be evenly distributed to split the
sample for testing and reference analysis. The reference sample aliquot for these three RW
samples will be split before lysing of the samples and will follow the freeze-thaw lysing method
and subsequent reference analysis.
In addition to a variety of RW samples, matrix interference samples will be tested using a
RW sample that has a low level or below detection level of native microcystin concentration.
This RW sample will be serial diluted by a factor of 10 with DI water to provide a less
concentrated level of the RW matrix. Then, each matrix level will be fortified with 4 ppb or 2
ppb of microcystin-LR, -LA, or -RR. The spike level chosen will be within the kit detection
range. The test kit results in each of the matrices will be compared in order to determine the
impact of the matrix concentration on the test kit results. For example, if there is no matrix
interference, the expectation would be that the test kit results would not change across the matrix
dilutions. In addition, they will be compared with the PT sample in DI water of the same
microcystin concentration. To evaluate the effect of chlorophyll-a as an interference, a DI water
sample that is fortified with 10 milligram/Liter of chlorophyll-a will be treated in an identical
fashion as the above RW sample. The solution of chlorophyll-a will be serial diluted by a factor
of 10 to provide solutions of 10 and 1 milligram/Liter chlorophyll-a. Then, each of these
concentration levels will be fortified with 4 or 2 ppb of microcystin-LR, -LA, or -RR. The test
kit results in each of the matrices will be compared in order to determine the impact of the matrix
concentration on the test kit results.
Lastly, the calibration standards provided with the microcystin test kits from different lots
could cause variability in the results across test kits. Therefore, two separate lots of calibration
standards will be analyzed using the kits and compared to determine the inter-kit lot
reproducibility.
QC, PT, and RW samples will be prepared by Battelle technical staff. Replicate samples
for the test kits will be taken from the same sample bottle. The QC, PT, and RW samples will
also be prepared blindly for the operator and will be coded to ensure the results are not
influenced by the operator's knowledge of the sample concentration.
-------
Microcystin Test Kits
Test/QA Plan
Page 27 of 46
Version 1.0
Date: 7/6/10
Reference samples will be an aliquot of the PT or RW samples. Because the reference
method is mass specific for different congeners, the PT samples for the three different congeners
at each spiking concentration will be combined into a volumetric flask and brought up to a
known volume with DI water. Then the calculated dilution factor will correct the reference
method result to the true PT sample concentration. The RW samples will be sent for reference
analysis without dilution. The results of each sample analysis by the test kits will be compared
to the reference method results of the same sample. Table 5 presents a solution preparation
scheme for the PT samples and Table 6 presents the test samples to be analyzed during this
verification test. It assumes the stock solutions are diluted to prepare spiking solutions of the
different congeners at 250 ppb.
TableS. Preparation of PT Samples
Target Cone.
(ppb)
7.0
4.0
2.0
1.5
1.0
0.5
0.1
0.0
Spiking Solution Cone.
(ppb)
250
250
250
250
250
250
250
250
Volume Spiking Solution
(mL)
3.5
2.0
1.0
0.75
0.5
0.25
0.05
0
Final Volume
(mL)
125
125
125
125
125
125
125
125
-------
Microcystin Test Kits
Test/QA Plan
Page 28 of 46
Version 1.0
Date: 7/6/10
Table 6. Summary of Test Samples
Type of Sample
QC- Laboratory Reagent
Blank (RB)
Performance Test (PT)
Samples - DI Water
Recreational Water (RW)
Samples- Lysed and
Filtered
Additional RW Samples
for test kits with specific
lysing procedure
RW Matrix Interference
Samples: NDRW
sample and 10 serial
dilution
Chloraphyll-a Matrix
Interference Samples:
Chlorpphyll-a sample and
10 serial dilution
Inter-kit lot
reproducibility
Microcystin
Variant
none
LR
LA
RR
LR
LA
RR
Unknown
Unknown
LR
LA
RR
LR
LA
RR
Microcystin
Concentration
(ppb)
0
0.1,0.5,1.0,2.0,4.0
ppb
0.5, 1.0,2.0,4.0,7.0
ppb
0.5, 1.0,2.0,4.0,7.0
ppb
5 times the vendor
stated MDL
5 times the vendor
stated MDL
5 times the vendor
stated MDL
3 samples >20 ppb, 3
samples >10 ppb, 3
samples ND
3 samples at
unknown
concentrations
4 ppb or 2 ppb*
4 ppb or 2 ppb*
4 ppb or 2 ppb*
4 ppb or 2 ppb*
4 ppb or 2 ppb*
4 ppb or 2 ppb*
Replicates
3
3
3
3
7
7
7
3
3
3
3
3
3
3
3
Total Number
of Samples per
Test Kit
10% of total test
samples, 2
15
15
15
7
7
7
27
9
6
6
6
6
6
6
A second set of vendor provided calibration standards from a different lot
analyzed following the vendor's procedure
""concentration that is within the calibration range of the test kit
B1.2 Statistical Analysis
The microcystin test kits being verified report total microcystin and are also calibrated
against microcystin-LR. Because of this, the reference method data will need to be converted to
microcystin-LR equivalents to compare the test kit results to the reference method results for all
-------
Microcystin Test Kits
Test/QA Plan
Page 29 of 46
Version 1.0
Date: 7/6/10
PT samples. Using cross reactivity data provided by each vendor (specific to each test kit), the
microcystin-LR equivalents will be calculated as follows:
C-LRequiv = ^ref cone X CR (1)
where Crefconcis the reference method result of the microcystin variant and CR is the mass-based
cross reactivity of the variant in equivalents of microcystin-LR.6
For the RW samples, each variant identified (through analysis by the reference method)
will be converted to LR-equivalents, and added together to calculate the total microcystin. The
total microcystin-LR equivalents from the RW reference analyses will be compared to the total
microcystin results from the test kits as described in the following sections. Because not all
possible variants are monitored by the reference method, there could be a discrepancy between
the test kit results and the total microcystin determined by the reference method.
B 1.2.1 Accuracy
Accuracy of the test kits being verified will be assessed relative to the results obtained
from the reference analyses. The results for each set of analyses will be expressed in terms of a
percent difference (%£)) as calculated from the following equation:
CT -CR
%D = — - -xlOO
where CT is the results from the test kits being verified and CR is the concentration as determined
by the reference method.
Bl.2.2 Linearity
Linearity will be determined by linear regression with the toxin concentration measured
by the reference method as the independent variable, and the test kit result being verified as the
dependent variable. Linearity will be expressed in terms of the slope, intercept, and the
r\
coefficient of determination (r ). In addition, plots of the observed and predicated concentration
values will be constructed to depict the linearity for each variant of microcystin being tested.
-------
Microcystin Test Kits
Test/QA Plan
Page 30 of 46
Version 1.0
Date: 7/6/10
Bl.2.3 Precision
The standard deviation (S) of the results for the replicate samples will be calculated and
used as a measure of test kit precision at each concentration. S will be calculated from the
following equation:
nl/2
H~ k=\
(3)
where « is the number of replicate samples, Ck is the concentration measure for the k* sample,
and C is the average concentration of the replicate samples. The kit precision at each
concentration will be reported in terms of the relative standard deviation (RSD) presented below
as equation 4.
RSD =
xlOO
(4)
B. 1.2.4 Method Detection Limit
Method detection limit (MDL) will be determined by seven replicate analyses of a
fortified sample with the toxin concentration of five times the vendor's estimated detection limit.
The MDL will be calculated from the following equation:
MDL = txS (5)
where / is the Student's value for a 95% confidence level, and S is the standard deviation of the
replicate samples.
B. 1.2.5 Inter-Kit Lot Reproducibility
Inter-kit lot reproducibility will be assessed by performing a linear regression of sample
results generated by kits using calibration solutions from two different lots. The slope, intercept,
and r2 will be used to evaluate the degree of inter-lot reproducibility. A paired t-test will also
conducted to evaluate whether the two sets of sample results were significantly different at a
95% confidence level.
B. 1.2.6 Matrix Effects
The effect of a natural matrix and of chlorophyll-a in DI water will be evaluated by
comparing the response of the test kits of the samples with matrix to the 4 or 2 ppb microcystin
-------
Microcystin Test Kits
Test/QA Plan
Page 31 of 46
Version 1.0
Date: 7/6/10
PT sample in DI water without matrix. The percent difference between the test kit results from 4
or 2 ppb microcystin in DI water and the test kit result of 4 or 2 ppb in each RW and chlorophyll-
a matrix interference sample will be calculated. If there is no matrix interference, the percent
difference should be negligible in all cases.
B1.2.7 Operational Factors
Operational factors such as maintenance needs, calibration frequency, data output,
consumables used, ease of use, repair requirements, waste production, and sample throughput
will be documented based on operator and Verification Test Coordinator observations. An
example of an ease of use questionnaire is provided in Appendix A.
B1.3 Reporting
Separate verification reports will be prepared for each vendor that is participating in the
verification testing. The statistical comparisons described above will be conducted separately for
each of the test kits being tested, and information on the operational factors will also be compiled
and reported separately for each test kit. The verification report will present the test procedures,
test data as statistical evaluation of those data, and discuss any deviations from the approved
test/QA plan.
Operational aspects of the monitoring systems will be recorded by the testing staff at the
time of observation during the verification test, and summarized in the verification report. The
verification report will briefly describe the ETV program, the AMS Center, and the procedures
used in verification testing. The results of the verification test regarding microcystin test kit
performance will be stated quantitatively. Each draft verification report will be subjected to
review by the vendor, EPA, and peer reviewers. The resulting review comments will be
addressed in a subsequent revision of the report, and the peer review comments and responses
will be tabulated to document the peer review process, and submitted to EPA. The reporting and
review process will be conducted according to the requirements of the AMS Center QMP.1
B2 SAMPLING METHOD REQUIREMENTS
As described above, multiple recreational water samples were collected for this
verification test. The samples were collected according to the sample collection and handling
-------
Microcystin Test Kits
Test/QA Plan
Page 32 of 46
Version 1.0
Date: 7/6/10
instructions included in the NDEQ standard operating procedure for microcystin analysis (SOP#
SWS-2320.1 A)7. In brief, recreational water samples were collected in either plastic or glass
amber collection containers. The sampling staff collected the samples throughout the summer of
2009 from locations that are representative of where human exposure would be expected (e.g., a
swim area in knee deep water). In addition, when algae were present, the sampling staff
collected the "worst case" sample by agitating the scum layer and collecting the sample six to
eight inches below the surface. A small amount of head space was left to allow for proper
shaking and mixing prior to analysis. Samples were frozen immediately after collection and
stored at < -10 °C until testing takes place. Samples may be kept at 4 °C +/- 3 °C for up to one
week after collection or frozen (at < -10 °C) if held for longer periods (e.g., more than one
week)7. Temperatures for the refrigerator and freezer will be logged on a monthly basis. If
found to be outside of the specified range above, the samples will be transferred to an acceptable
refrigerator or freezer and the deviation will be noted in the LRB, in a deviation report, and in the
final verification reports. In cases where there is not enough sample volume for testing and
reference analysis, RW samples may be combined. They will not be diluted with DI water.
The same SOP for microcystin analysis also contains a procedure for lysing the
cyanobacteria to release the microcystin into the water sample for analysis7. A procedure will be
necessary to perform on all of the RW samples. This procedure goes through three iterations of
completely freezing and thawing in order to breakdown the cell walls of the bacteria7. Then the
samples will be filtered. Once the RW samples are lysed and filtered, the sample will be
transferred, handled, and stored in glass containers to minimize any potential absorption of
microcystin by plastic. Then more than 100 milliliters of the samples will be aliquoted into
individual glass vials and transported to the reference laboratory on ice.
If the test kits require approaches other than freezing and thawing to lysing the
cyanobacteria, that approach will be used for the applicable test kits. For the toxicity test kit,
three additional RW samples will be included in testing that will not undergo freeze-thaw before
being analyzed by the kit. For these three samples, the reference will be split before lysing.
B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS
Sample custody will be documented throughout collection and analysis of the test
samples following the Battelle SOP for Chain of Custody8. A chain-of-custody (COC) form will
-------
Microcystin Test Kits
Test/QA Plan
Page 3 3 of 46
Version 1.0
Date: 7/6/10
include details about the sample such as the time, date, location, and person collecting the
sample. The COC form will track sample release from the sampling location to the analysis
laboratory. Each COC form will be signed by the person relinquishing samples once that person
has verified that the COC form is accurate. Upon arrival at the analysis laboratory, COC forms
will be signed by the person receiving the samples (if different from the sample collector) once
that person has verified that all samples identified on the COC forms are present. Copies of all
COC forms will be delivered to the Verification Test Coordinator and maintained with the test
records. When samples are delivered to a reference laboratory, a second COC form will be
completed as described above.8
B4 LABORATORY REFERENCE METHOD
Technology verification will involve comparison of the results from each test kit being
verified to the results obtained from an appropriate reference method. The reference method
chosen for this verification test is proven liquid chromatography tandem mass spectrometry (LC-
MS/MS) analysis method for the determination of algal toxins9. This method has been slightly
modified from the publication but follows the scientific approach of utilizing a reversed-phase
separation column, electrospray ionization, and multiple reaction monitoring (MRM) in positive
ion mode to detect the specific mass-to-charge (m/z) precursor and product ions associated with
the variants of microcystin. By monitoring specific m/z values of precursor and product ions, the
method is specific to the different variants of microcystin. It utilizes an internal standard,
Nodularin, to minimize any matrix effects from the water samples and a surrogate recovery
standard, Enkephalin to normalize the extraction efficiency of the extraction method. Table 4
shows the Chemical Abstract Service (CAS) number, MRM reaction monitored by the reference
laboratory on each microcystin of interest for this verification test. Because to the DDLs of the
reference method are higher than some of the PT samples, samples sent for analysis will go
through solid-phase extraction (SPE) and concentration steps10. The reference laboratory will
receive all samples blindly.
The reference laboratory, University of Nebraska Water Center in Lincoln, Nebraska,
will perform the analysis following the QA/QC procedures described in Section B5. In addition,
prior to testing, Battelle's QAO will contact the reference laboratory and request submission of
that laboratory's QA plan and associated records. In addition, Battelle will visit the reference
-------
Microcystin Test Kits
Test/QA Plan
Page 34 of 46
Version 1.0
Date: 7/6/10
laboratory and audit the QA document associated with the samples analyzed during this ETV
test.
Table 7. Microcystin Reference Method Information
Name
Microcystin-LR
Microcystin-LA
Microcystin-RR
Nodularin (Internal Standard)
Enkephalin (Surrogate Recovery
Standard)
CAS number
101043-37-2
96180-79-9
111755-37-4
118399-22-7
58822-25-6
Positive Ion MRM Reaction
995.35>135.1
910.2>135.1
520.0>135.0
825.1>135.1
556.1>278.0
B5 QUALITY CONTROL REQUIREMENTS
Quality control steps will follow the vendor specified frequency and levels for the
microcystin test kits. All of the test kits require a positive and negative control and the
quantitative test kits also include multiple concentrations of calibration standards.
The reference method requires the analysis of a method blanks (MB), Laboratory
Fortified Matrix (LFM) samples and duplicate samples. One MB, LFM, and duplicate sample
analysis will be performed during reference analysis for every 20 samples analyzed. The MB
should be rejected if the microcystin concentration is above the reporting limit. The LFM is
acceptable if within 30% of the expected concentration. The relative percent difference (RPD)
of the duplicate measurements will be required to be less than 30%. RPD will be calculated as in
Equation 6 below, where d will be the absolute difference between the duplicate samples and
C will be the average of the duplicate sample results. A continuing calibration verification
(CCV) standard will be analyzed every 10 samples to ensure that the calibration is still valid. It
will be a mid-level standard and must be within 20% of the expected value. The reporting limits
for the three congeners used in this verification test must be < 0.1 ppb. See Table 3 for a
summary of these requirements.
\d
~ ~C ^
Sample sets producing results not meeting these requirements may be reanalyzed by the
reference method. If the results are still outside the required tolerance, the reference instrument
will be recalibrated (if applicable) and/or the reference samples reanalyzed. If the outlying
-------
Microcystin Test Kits
Test/QA Plan
Page 3 5 of 46
Version 1.0
Date: 7/6/10
results persist, the repeat of the appropriate parts of the verification test or use of a different
reference instrument may be considered.
B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE
The instruments used for the reference analyses will be tested and inspected as per the
instrument manuals, the standard operating procedures of the analysis laboratory, or the methods
being used to make each measurement. Operation of the test kits during the verification test will
be performed by Battelle technical staff as directed by the vendor.
B7 CALIBRATION VERIFICATION
The instruments used for the reference analyses will be calibrated per the instrument
manual, the methods being used to make each measurement or the standard operating procedures
of the analysis laboratory. The vendor will provide the Battelle verification staff with the
necessary training/information to properly maintain each test kit. All calibrations performed will
be documented by the verification staff in the project LRB or data collection forms.
Calibration of the test kit will be done as often as suggested by the vendor. Vendors will
be required to supply the necessary calibration solutions and devices specific to the test kits
being verified. Balances and pipettes used during test solution preparation will be maintained
and calibrated per the manufacturer's procedures which will be reviewed by the Battelle QAO
prior to the verification test.
B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES
All materials, supplies, and consumables will be ordered by the Verification Test
Coordinator or designee. Where possible, Battelle will rely on sources of materials and
consumables that have been used previously as part of ETV verification testing without
problems. Battelle will also rely on previous experience or recommendations from NDEQ
technical staff and the vendors to guide selection of manufacturers and materials. The
manufacturer's criteria for acceptance/purity will be required to be met. Microcystin certified
standards are not available; therefore, the source used for this verification test will be purchased
and verified by the reference method before use.
-------
Microcystin Test Kits
Test/QA Plan
Page 36 of 46
Version 1.0
Date: 7/6/10
B9 NON-DIRECT MEASUREMENTS
Data published previously in the scientific literature will not be used during this
verification test.
BIO DATA MANAGEMENT
Records received by or generated by any of the verification staff during the verification
test will be reviewed by the Verification Test Coordinator or designee. Test data will be
reviewed at intervals sufficient to ensure that test data are meeting the DQOs. The Verification
Test Coordinator or delegate will review 100% of the first data batch within one week of
delivery. Given the short duration of this test, all of the test samples and subsequent aliquots for
reference method analysis will be prepared the day before testing begins. The reference samples
will be shipped to the reference laboratory for analysis. Therefore, the first batch of reference
method data is defined as all of the reference method data. However, the first batch of testing
data is defined as the testing data collected on the first day of testing. The Verification Test
Coordinator's review will verify that:
• All data are reported as required in the test/QA plan.
• Calibration and QC results are reported and are acceptable.
• Data are reasonable (within expected ranges).
• Technologies appear to be generating data as expected.
Records will be reviewed and verified prior to use to calculate, evaluate, or report
verification results. These checks will include:
• QC samples and calibration standards were analyzed according to the test/QA plan.
• Calibration and QC sample results are reported and the acceptance criteria were met.
• Corrective action for exceedances was implemented.
• 100% hand-entered and/or manually calculated data were checked for accuracy.
• Calculations performed by software are verified at a frequency sufficient to ensure that
the formulas are correct, appropriate, and consistent.
-------
Microcystin Test Kits
Test/QA Plan
Page 37 of 46
Version 1.0
Date: 7/6/10
• For each cut and paste function, the first and last data value was verified vs. the source
data.
• Data are reported in the units specified in the test/QA plan.
Records reviews will be documented as the dated initials of the reviewer. Any issues
identified during the data review will be addressed with the testing staff in real time (i.e., < 5 few
days of discovery) so that corrective action can be implemented and testing stopped, if needed, to
ensure that data of sufficient quality are collected to meet the DQOs.
-------
Microcystin Test Kits
Test/QA Plan
Page 3 8 of 46
Version 1.0
Date: 7/6/10
SECTION C
ASSESSMENT AND OVERSIGHT
Cl ASSESSMENTS AND RESPONSE ACTIONS
Every effort will be made in this verification test to anticipate and resolve potential
problems before the quality of performance is compromised. One of the major objectives of the
test/QA plan is to establish mechanisms necessary to ensure this. Internal quality control
measures described in this test/QA plan, which is peer reviewed by external experts,
implemented by the technical staff and monitored by the Verification Test Coordinator, will give
information on data quality on a day-to-day basis. The responsibility for interpreting the results
of these checks and resolving any potential problems resides with the Verification Test
Coordinator. Technical staff have the responsibility to identify problems that could affect data
quality or the ability to use the data. Any problems that are identified will be reported to the
Verification Test Coordinator, who will work with the Battelle QAO to resolve any issues.
Action will be taken to control the problem, identify a solution to the problem, and minimize
losses and correct data, where possible. Battelle will be responsible for ensuring that the
following audits are conducted as part of this verification test. See Table 2 for the proposed
verification test schedule of audits.
Any changes to the approved test/QA plan must be reported within 24 hours and
documented in a formal deviation submitted to the Battelle AMS Center Manager, EPA PO, and
EPA QM. If approval by EPA or its designee is not received within 24 hours of notification,
testing will be halted until a suitable resolution has been achieved.
Cl.l Performance Evaluation Audits
A PEA will be conducted to assess the quality of the reference measurements made in
this verification test. Before testing begins, blind samples prepared from independent standards
will be submitted to the WSL for analysis. As NIST standards are not available, Microcystin-LR
and -RR will be obtained from the Canadian NRC (with draft certificates of analyses) and
microcystin-LR, -RR, and -LA will be obtained from Abraxis. A dilution of these standards will
be sent for reference analysis. The NRC standards will be diluted together in a volumetric flask
-------
Microcystin Test Kits
Test/QA Plan
Page 3 9 of 46
Version 1.0
Date: 7/6/10
into one PEA sample, and the Abraxis standards will also be diluted together into a second PEA
sample. The results of the PEA samples must be within the acceptable tolerance of 30%. If the
results do not meet this requirement, they will be repeated. If the outlying results persist, the
Verification Test Coordinator, or designee, and the reference laboratory representative will
discuss corrective actions, and a repeat of the PEA will have to be performed. Testing will not
take place unless the PEA samples are within the acceptable range. The results from the PEA
will be sent to the EPA PO and EPA QM within 10 days of receipt from the reference laboratory.
The PEA report will include the raw data, draft certificate from the Canadian NRC, calculations
of the comparison to the expected concentration and a discussion of corrective action, if
applicable.
C1.2 Technical Systems Audits
Battelle QAO or delegate will perform a technical systems audit (TSA) at least once
during this verification test. The purpose of this audit is to ensure that the verification test is
being performed in accordance with the AMS Center QMP1 and this test/QA plan. The primary
focus of the audit will be operation of the technologies being verified. The audit will compare
actual test procedures to those specified or referenced in the test/QA Plan, and will review data
acquisition and handling procedures. The audit of the technologies will include verification that:
• The technologies are calibrated and operated as defined in the test/QA plan.
• Any test/QA plan specifications and QC are implemented.
• The data generated by the technologies are 'reasonable' based on the vendor
specifications.
• Documentation and sample labeling are sufficient to ensure data traceability.
The audit of the reference method laboratory may include:
• A review of the testing facility and equipment (instrument/equipment calibration,
maintenance, and operation.
• Sample handling procedures.
• Comparison of test procedures to the reference method specifications.
-------
Microcystin Test Kits
Test/QA Plan
Page 40 of 46
Version 1.0
Date: 7/6/10
• Verification that calibration and QC procedures conform to the method and that
the results meet the acceptance criteria.
• Review of documentation procedures.
Based on available time, the focus of the reference method audit will be on ensuring that
the method is fully implemented and that instrument calibration and QC results are acceptable.
The auditor will confer with the reference laboratory staff and Battelle technical staff during the
audit, as needed. The TSA will be guided by a project-specific checklist based on the test/QA
plan and reference method. A TSA debriefing will be conducted with the testing staff at the
conclusion of the audit. The EPA PO and EPA QM will be invited to the debriefing.
A TSA report will be prepared as a memo to the Verification Test Coordinator within 10
business days after completion of the audit; the completed checklist will be attached. The
Battelle AMS Center Manager, EPA PO, and EPA QM will be copied on the memo. The
Verification Test Coordinator will respond to the audit within 10 business days. The Battelle
QAO or designate will verify that all audit Findings and Observations have been addressed and
that corrective actions are appropriately implemented. A copy of the complete TSA report with
corrective actions will be provided to the EPA PO and EPA QM within 10 business days after
receipt of the audit memo. At EPA's discretion, EPA QA staff may also conduct an independent
on-site TSA during the verification test. The TSA findings will be communicated to technical
staff at the time of the audit and documented in a TSA report.
C1.3 Data Quality Audits
The Battelle QAO or designee will perform a data quality audit (DQA) on at least 10% of
the sample results acquired in the verification test and 100% of the calibration and QC data vs.
the test/QA plan requirements. The exact percentage of data results audited is less critical than
the overall reasonableness of the data. If data quality errors are detected the auditor will track
the data to identify upstream causes and downstream impacts. A checklist based on the test/QA
plan will guide the audit. The primary focus of the audit will be the reference method data
although the testing data will also be audited.
An initial data quality audit will be conducted on the first batch of test data within 3
business days of when data were posted on the project SharePoint site to identify errors early in
the data reduction process. Given the short duration of this test, all of the test samples and
-------
Microcystin Test Kits
Test/QA Plan
Page 41 of 46
Version 1.0
Date: 7/6/10
subsequent aliquots for reference method analysis will be prepared the day before testing begins.
The reference samples will be shipped to the reference laboratory for analysis. Therefore, the
first batch of reference method data is defined as all of the reference method data. However, the
first batch of testing data is defined as the testing data collected on the first day of testing. The
remaining data will be audited once all data for the technologies have been posted on the project
SharePoint site and once all statistical analyses are complete. The Battelle QAO, or designee,
will trace the data from initial acquisition, through reduction and statistical comparisons, to final
reporting. The audit will reproduce the reported results based the raw data and any calculations
and data reduction procedures performed on the data to ensure that the reported results are
traceable. The review of testing data will be limited to ensuring that calibrations were performed
as defined in the test/QA Plan and will review 10% of the data calculations and transcriptions to
identify errors and verify that the reported data are traceable to the raw data.
A DQA audit report will be prepared as a memo to the Verification Test Coordinator
within 15 business days after the data are posted; the completed checklist will be attached. The
Battelle AMS Center Manager, EPA PO, and EPA QM will be copied on the memo. The
Verification Test Coordinator will respond to the audit within 10 business days. The Battelle
QAO or designate will verify that all audit Findings and Observations have been addressed and
that corrective actions are appropriately implemented. A copy of the complete DQA report with
corrective actions will be provided to the EPA PO and EPA QM within 10 business days after
receipt of the audit memo. At EPA's discretion, EPA QA staff may also conduct an independent
audit of data quality.
C1.4 QA/QC Reporting
Each assessment and audit will be documented in accordance with Section 3.3.4 of the
AMS Center QMP1. The results of the TSA and DQA will be submitted to EPA. Assessment
reports will include the following:
• Identification of Findings and Observations.
• Recommendations for resolving problems.
• Response to adverse findings or potential problems.
-------
Microcystin Test Kits
Test/QA Plan
Page 42 of 46
Version 1.0
Date: 7/6/10
• Confirmation that solutions have been implemented and are effective.
• Citation of any noteworthy practices that may be of use to others.
C2 REPORTS TO MANAGEMENT
The Battelle QAO, during the course of any assessment or audit, will identify to the
technical staff performing experimental activities any immediate corrective action that should be
taken. If serious quality problems exist, the Battelle QAO is authorized to notify the Battelle
AMS Center Manager who will issue the stop work. Once the assessment report has been
prepared, the Verification Test Coordinator will ensure that a response is provided for each
adverse finding or potential problem and will implement any necessary follow-up corrective
action. The Battelle QAO will ensure that follow-up corrective action has been taken.
In addition to this test/QA plan, a final report and a verification statement for each vendor
will be prepared and reviewed, with test kit data presented separately for each technology. The
final report is a comprehensive document describing the verification test. The verification
statement is a two-to-three page summary of the technology, the test procedures, and the test
results. Each draft report and verification statement will be submitted to the respective vendor
for review. They are then reviewed by EPA QM and the EPA PO. Upon approval by EPA, each
verification statement will be signed by a senior manager of Battelle and by an EPA laboratory
director. Original signed verification statements will be provided to the respective vendors for
use in marketing their technology. Upon final review and approval, the documents will then be
posted on the ETV website (www.epa.gov/etv).
-------
Microcystin Test Kits
Test/QA Plan
Page 43 of 46
Version 1.0
Date: 7/6/10
SECTION D
DATA VALIDATION AND USABILITY
Dl DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS
The key data review requirements for the verification test are the collection of QC
samples as outlined in the test/QA plan, a comparison of data sheet comments against final data
to flag any suspect data, and a review of final data to resolve any questions about apparent
outliers. The QA audits, as described within this document are designed to assure the quality of
this data.
D2 VALIDATION AND VERIFICATION METHODS
Section C of this test/QA plan provides a description of the validation safeguards
employed for this verification test. Data validation and verification efforts include the analysis
of QC samples as required in this document, and the performance of the TSA and PEA as
described in Section C.
D3 RECONCILIATION WITH USER REQUIREMENTS
This test/QA plan and the resulting ETV verification report(s) will be subjected to review
by the microcystin test kit vendors, EPA, and expert peer reviewers. These reviews will assure
that this test/QA plan and the resulting verification report(s) meet the needs of potential users of
the microcystin test kits. Performance data for the microcystin test kits, collected under
conditions where the quality control requirements for the duplicate and PEA samples were met,
will be presented in the final verification report without any further comment. Performance data
and reference measurements that do not meet these criteria will be noted and a discussion of the
possible impact of the failed requirements on the performance evaluation will be presented in the
final verification report. The final verification report(s) will be submitted to EPA in MS Word
and Adobe portable document format (PDF) and subsequently posted on the ETV website
(www.epa.gov/etv).
-------
Microcystin Test Kits
Test/QA Plan
Page 44 of 46
Version 1.0
Date: 7/6/10
SECTION E
REFERENCES
El REFERENCES
1. Quality Management Plan for the ETVAdvancedMonitoring Systems Center, Version 7.
U.S. Environmental Technology Verification Program, Battelle, November 2008.
2. Environmental Technology Verification Program Quality Management Plan. January,
2008(EPA/600/R-08/009).
3. Butler, N., et al., Microcystins: A brief overview of their toxicity and effects, with special
reference to fish, wildlife, and livestock. January, 2009, Office of Environmental Health
Hazard Assessment.
4. Guidelines for Safe Recreational Water Environments, Volume 1, Coastal and Fresh
Waters. 2003: World Health Organization.
5. "Guidelines Establishing Test Procedures for the Analysis of Pollutants. ", USEP A,
Editor. 2000, U.S. Code of Federal Regulations.
6. Loftin, K.A., et al., Comparison of Two Cell Lysis Procedures for Recovery of
Microcystins in Water Samples from Silver Lake in Dover, Delaware, with Microcystin
Producing Cyanobacterial Accumulations, in USGS Open-File Report 2008 -1341. 2008.
p. 9.
7. SOP# SWS-2320.1A: Microcystin Analysis Using the Abraxis ELISA (Enzyme-Linked
Immuno-Sorbent Assay) Method. Nebraska Department of Environmental Quality.
8. SOP ENV-ADM-009, Standard Operating Procedure for Sample Chain-of-Custody
Battelle, September 2007.
9. Hollrah, M., Standard Operating Procedure (SOP) Determination ofalgaltoxin residues
in water extracts by liquid chromatography (LC)- atmospheric pressure electrospray
ionization tandem mass spectrometry (MS/MS). December, 2005, Water Sciences
Laboratory, University of Nebraska.
10. Cong, L.H., B.; Chen, Q.; Lu, B.; Zhang, J.; Ren, Y., Determination of trace amount of
microcystins in water samples using liquid chromatography coupled with triple
quadrupole mass spectrometry. Anal. Chim. Acta, 2006. 569 (1-2): p. 157-168.
-------
Microcystin Test Kits
Test/QA Plan
Page 45 of 46
Version 1.0
Date: 7/6/10
APPENDIX A
EASE OF USE QUESTIONNAIRE
Technology Evaluated:
Operator: Date:
Kit-
1) Clarity of instruction manual:
2) Solutions/reagents easily identifiable?
3) Storage conditions of solutions/reagents readily marked/easily available?
4) Number of samples that can be processed per kit?
5) All containers/packaging easy to open?
Reagents-
1) Ease of reagent preparation:
2) Reagent storage requirements:
3) Shelf life of reagents as received in kit:
4) Shelf life of reagents once prepared for analysis:
5) Equipment/materials required for reagent prep (i.e., balances, pipettes, etc)? Anything
specialized?
Equipment-
1) User-friendliness of software or electronic readout:
-------
Microcystin Test Kits
Test/QA Plan
Page 46 of 46
Version 1.0
Date: 7/6/10
2) Data endpoint or reaction easy to visually observe?
3) Does equipment require any special preparation before use?
4) Is equipment easy to clean off?
5) Does equipment require any routine maintenance?
Overall-
1) Comments on general convenience of product:
2) Estimate of training/education required to carry out testing with this technology?
3) Does this product generate a lot of solvent or solid waste?
4) Are the wastes generated hazardous (i.e., need special disposal)?
5) Does the vendor provide support (phone or otherwise?)
6) If you encountered any problems with the technology was it easy to remedy?
Other Comments:
------- |