Addendum to the Method 1668A Interlaboratory Validation Study Report
Addendum to the Method 1668A Interlaboratory Validation Study Report
March 2010
Pub Number EPA-820-R-10-003
Summary
This addendum to the Method 1668A Interlaboratory Validation Study Report (dated November
2008) revises Table 4-1, "Congener Detection Rates and Concentrations in Study Samples (by Matrix and
Level of Chlorination)," and revises the section on quality control (QC) acceptance criteria in the report,
including revision of the QC acceptance criteria in Table 5-1 of the report. The criteria were reassessed in
response to laboratory feedback that some of the criteria in the report were unrealistically restrictive, and
were revised based on an assessment of recently submitted data. The revised QC acceptance criteria in
this addendum were developed based on the combination of a statistical analysis and a holistic view of the
data.
This March 2010 revision to the addendum made the following changes:
• Footnote 1 to Table 4-1 was corrected to show that the biosolids concentration is in wet-weight
units (because not all laboratories provided % solids and dry-weight results).
• Footnote 2 to Table 4-1 was revised to clarify that the mean, median, and maximum results are
based on any detected congener in an LOC, and expanded to further explain that concentrations
for coeluted congeners are for combined concentrations of all congeners within that coelution.
• The section on revision of QC acceptance criteria was completely revised to take into account
calibration verification data received from AXYS Analytical and TestAmerica-Knoxville and to
make the criteria consistent across all performance tests.
• Table A-l presents the revised QC acceptance criteria.
• A section was added at the end of the addendum to explain that the revised QC acceptance
criteria will appear in Revision C to Method 1668.
Background
EPA initially published Method 1668A in 1999. Since that time, the Agency has:
• Revised the method in 2000 and 2003 to reflect user suggestions and peer reviews,
• Published a study plan for the Method 1668A interlaboratory validation study in 2003,
• Conducted the interlaboratory validation study in 2003-2004,
• Published the peer reviewed validation study report, and
• Published a revised method that reflects peer review and user suggestions and data.
Additional details regarding the method revisions, the study, and the peer review are available in
Revisions A and B of Method 1668, and the study plan, peer review report, validation study report, and
validation study peer review report listed above.
After Revision B was published, AXYS Analytical Services, Ltd. (AXYS) informed EPA that the
revised initial precision and recovery (IPR) and ongoing precision and recovery (OPR) QC acceptance
criteria for some congeners could not be met during routine laboratory operations because these criteria
did not allow recoveries exceeding 100% for many congeners. (In developing the Method 1668B criteria,
EPA set the upper limit on recovery to 100 percent for congeners for which a statistical analysis resulted
in recoveries less than 100 percent.) EPA responded to this concern by using recently submitted QC data
March 2010 ]
-------
Addendum to the Method 1668A Interlaboratory Validation Study Report
to examine the QC acceptance criteria published in Section 5 of the November 2008 interlaboratory study
report and in Method 1668 Revision B. This examination resulted in revised criteria, as presented below.
The examination also identified errors in Table 4-1 of the November 2008 report. These changes are
documented in this addendum.
Corrections to Table 4-1 of the Interlaboratory Validation Study Report
The following is a corrected table. The corrections result from conversion of biosolids samples
from dry weight to wet weight, so that the results from all four laboratories were reported on the same
basis. To allow quick comparison with the values in Table 4-1 of the original validation study report, the
corrected values in the table below are shown in boldface type.
Table 4-1. Congener Detection Rates and Concentrations in Study Samples (by Matrix and Level
of Chlorination)
Matrix
Biosolids
Tissue
Water
LOG
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
#
Labs
4
6
6
# Congeners
Analyzed
24
88
160
240
237
254
169
81
24
8
36
131
232
352
347
362
240
114
35
12
36
128
233
356
344
362
235
116
35
12
# Congeners
Detected
23
64
134
195
166
196
129
72
23
8
26
90
181
288
258
270
182
105
35
12
25
118
223
356
344
362
235
116
35
12
% Congeners
Detected
96
73
84
81
70
77
76
89
96
100
72
69
78
82
74
75
76
92
100
100
69
92
96
100
100
100
100
100
100
100
Concentration (detects only)1'2
Mean
71
259
514
667
1090
602
362
195
161
166
4
47
267
402
418
429
276
157
162
200
27
533
1100
2850
2660
2190
1750
2410
1760
1740
Median
74
140
387
277
488
224
181
140
91
161
3
27
150
130
128
108
120
108
137
201
20
505
946
2170
1750
1660
1420
1740
1520
1510
Maximum
94
967
2370
4130
4720
4450
1670
583
630
193
12
188
1610
3330
15700
10700
3560
709
390
236
106
1460
3430
15300
21800
11800
7370
9560
3350
3170
'Biosolids concentrations in ng/kg (wet weight); tissue concentrations in ng/kg (wet weight); water concentrations in pg/L
2Mean, median, and maximum concentrations at each LOG are based on any detected congeners in that LOG. When coelution of
two or more congeners occurred, the combined value of those co-eluted congeners was used.
Revision of QC Acceptance Criteria
In response to the information from AXYS, the IPR and OPR QC acceptance criteria were re-
evaluated using more OPR data than were available from the Method 1668A interlaboratory validation
study. Specifically, AXYS and TestAmerica-Knoxville (TestAmerica) provided EPA with large sets of
March 2010
-------
Addendum to the Method 1668A Interlaboratory Validation Study Report
OPR data they had generated as part of their routine sample analysis activities. Both sets of additional
data were from analyses performed after completion of the Method 1668A interlaboratory study. AXYS
provided 113 sets of results from aqueous and solid OPR samples, and TestAmerica provided 112 sets of
results from aqueous and solid OPR samples.
When these recent data were compared to QC acceptance criteria in Table 5-1 of the
interlaboratory validation study report (and the identical Method 1668B criteria), EPA observed that:
1) failure rates were notably higher than expected for high chlorination level labeled analogs, and
2) the failure rate was higher than anticipated when assessed on a per-sample basis (i.e., if at least
one congener would fail, the OPR would fail and, therefore, the OPR and associated batch of
samples would have to be reanalyzed for all congeners).
As a result, EPA used these data, along with the OPR data from the method validation study, to revise the
QC acceptance criteria that were published in the 2008 validation study report and in Method 1668B.
When determining QC acceptance criteria, it is assumed that all data used in the calculation are
representative of the population of results from laboratories performing the method properly, and that any
extreme results produced would be due to analytical variability, and not to laboratory issues. When using
existing data to establish QC acceptance criteria, this assumption can be problematic because it cannot
easily be determined whether a result is unusually high or low due to chance or due to a problem with the
sample preparation or analysis. However, the large number of PCB congeners tested in an OPR sample
(27 native and 27 labeled congeners) allows an assessment of the consistency of each individual sample
with the overall dataset. An OPR sample for which the recoveries for many congeners are consistently
higher or lower than those for other samples gives an indication that there may have been an issue with
the analysis of that sample, whereas an OPR sample for which only one or two congeners yielded unusual
recoveries is more likely to be failing by chance alone. Therefore, each OPR sample submitted by the
two laboratories was assessed for a high frequency of unusually high or low recoveries. Based on this
assessment, five OPR samples were removed from the dataset.
After removal of the five OPR samples, all remaining data from the two laboratories were
combined, and the distribution of recoveries was examined for each native and labeled congener. Based
on this examination, three subgroups were identified that yielded similar distributions. These subgroups
were defined as follows:
1) All native congeners
2) Labeled congeners 1 to 54
3) Labeled congeners 77 to 209
A revised set of OPR recovery criteria was chosen for each of these subgroups that would result
in an approximate 5% failure rate on a per-sample basis. To further assess these chosen criteria, OPR
results from the two laboratories were combined with the Method 1668A validation study data. This
approach allowed the chosen criteria to be assessed using data with a larger between-laboratory variance
component. Because there were many more OPR results from the two post-study laboratories than from
the validation study laboratories, 100 sets of data were simulated. For each simulation run, four OPR
results per congener were chosen randomly (such that the same sample would not be chosen for all
congeners) from each of the two post-study laboratories. For each of these simulation runs, OPR criteria
were calculated using the same formulas used to produce Method 1668B criteria from the method
validation study. Because the resulting simulated QC criteria tended to be tighter than the nominal
criteria, it was concluded that the added between-laboratory variability was not large enough to
necessitate widening the chosen criteria.
March 2010
-------
Addendum to the Method 1668A Interlaboratory Validation Study Report
After choosing the OPR criteria, IPR criteria were chosen based on the OPR criteria. Unlike
OPRs, which are evaluated on an individual sample basis, IPRs are analyzed and evaluated in sets of four.
Because means of four measurements are less variable than single measurements, IPR recovery criteria
tend to be tighter (by approximately 10-15%) than OPR recovery criteria. Based on this assumption,
nominal IPR criteria were chosen for each of the three congener subsets that were approximately 10-15%
tighter than the corresponding OPR recovery criteria. These criteria then were evaluated using the data
submitted by the two post-study laboratories. Because these data comprised OPR samples only, IPR sets
needed to be simulated to assess the IPR criteria. To do this, 100 sets of 4 OPRs were chosen randomly
in order to simulate an IPR "set." The number of occurrences where an IPR set failed the nominal criteria
was then determined for each congener. It would be expected that the failure rate could be slightly larger
than the target 5%, because the simulated sets included a larger amount of temporal variability than a set
of IPRs analyzed by a laboratory in practice (which are usually run within a single batch). However, the
observed failure rates were well within the expected 5%, and therefore supported the chosen nominal
criteria.
Revision of QC Acceptance Criteria for Calibration Verification
QC acceptance criteria for calibration verification were not revised based on the 2003-2004
interlaboratory study because calibration and calibration verification data were not gathered in the study.
After completion of the interlaboratory study, calibration verification data were gathered from AXYS and
TestAmerica. AXYS supplied results of analysis of 686 calibration verification samples and TestAmerica
supplied results of 1160 calibration verification samples. Using a similar approach to that described for
the OPR criteria modification, the results from the two laboratories were assessed to arrive at calibration
verification criteria with an approximate per-sample failure rate of 5%.
• The criteria for all native congeners were set to 75 - 125%, vs. 70 - 130% in Method 1668A
• The criteria for labeled congeners 1L to 209L were set to 50 - 145%, vs. 50 - 150% in Method
1668A
• The criterion for labeled cleanup standard 28L was set to 65 - 135%, vs. 60 - 130% in Method
1668A, and
• The criteria for labeled cleanup standards 111L and 178L were set to 75 - 125% vs. 60 - 130% in
Method 1668A.
For labeled compounds 1L - 209L and the cleanup standards, the observed per-sample failure rate for the
AXYS data was 5.3 percent and the observed per-sample failure rate for the TestAmerica data was 3.0
percent. The adjusted criteria are shown in Table A-l.
Adjustment of QC Acceptance Criteria for OPR and Labeled Compound Recovery from
Samples
When the revised QC acceptance criteria for calibration verification, IPR, OPR, and labeled compound
recovery from samples were compared, the upper limit of the calibration verification criteria was less
restrictive than the upper limits of the OPR and the labeled compound recovery from samples. To make
all criteria consistent, upper limits of criteria for OPR and labeled compound recovery from samples were
increased to be greater than or equal to the upper limits of the calibration verification criteria. For
example, for labeled congeners 1L - 54L, OPR criteria for recovery, based on a 5% failure rate, were 15 -
140%, necessitating an increase in the upper limit to 145%. The upper limits on IPR recovery for labeled
compounds 1L - 54L and 77L - 209L were not increased to be greater than or equal to calibration
verification criteria because an IPR consists of the average of results of 4 tests, whereas calibration
verification is a single test. The adjusted criteria are shown in Table A-l. These revised criteria replace
the criteria in Section 5 and Table 5-1 of the November 2008 report.
March 2010
-------
Addendum to the Method 1668A Interlaboratory Validation Study Report
Table A-1. Revised QC Acceptance Criteria for Calibration Verification, Initial Precision and
Recovery (IPR), On-going Precision and Recovery (OPR), and Labeled Compound
Recovery from Samples
Congener set
Native Toxics and LOCs
Labeled congeners
1Lto54L
77Lto209L
Cleanup standards
28L
111Land 178L
Calibration
Verification
75-125%
50-145%
50-145%
65-135%
75-125%
IPR
Recovery
70-130%
20-135%
45-135%
20-135%
45-135%
IPR
Precision
25%
70%
50%
70%
50%
OPR
Recovery
60-135%
15-145%
40-145%
15-145%
40-145%
Labeled Compound
Recovery from Samples
NA
5-145%
10-145%
5-145%
10-145%
NA = Not applicable
Revision C to Method 1668
To accommodate the changes to the QC acceptance criteria presented in this addendum, EPA had
the option of revising Method 1668B, or creating Revision C to Method 1668. The advantage to creating
Method 1668C is that it minimizes confusion associated with multiple versions of Revision B. EPA has
therefore chosen to create Method 1668C to include the revised QC acceptance criteria presented in this
addendum.
March 2010
------- |