Addendum to the Method 1668A Interlaboratory Validation Study Report Addendum to the Method 1668A Interlaboratory Validation Study Report March 2010 Pub Number EPA-820-R-10-003 Summary This addendum to the Method 1668A Interlaboratory Validation Study Report (dated November 2008) revises Table 4-1, "Congener Detection Rates and Concentrations in Study Samples (by Matrix and Level of Chlorination)," and revises the section on quality control (QC) acceptance criteria in the report, including revision of the QC acceptance criteria in Table 5-1 of the report. The criteria were reassessed in response to laboratory feedback that some of the criteria in the report were unrealistically restrictive, and were revised based on an assessment of recently submitted data. The revised QC acceptance criteria in this addendum were developed based on the combination of a statistical analysis and a holistic view of the data. This March 2010 revision to the addendum made the following changes: • Footnote 1 to Table 4-1 was corrected to show that the biosolids concentration is in wet-weight units (because not all laboratories provided % solids and dry-weight results). • Footnote 2 to Table 4-1 was revised to clarify that the mean, median, and maximum results are based on any detected congener in an LOC, and expanded to further explain that concentrations for coeluted congeners are for combined concentrations of all congeners within that coelution. • The section on revision of QC acceptance criteria was completely revised to take into account calibration verification data received from AXYS Analytical and TestAmerica-Knoxville and to make the criteria consistent across all performance tests. • Table A-l presents the revised QC acceptance criteria. • A section was added at the end of the addendum to explain that the revised QC acceptance criteria will appear in Revision C to Method 1668. Background EPA initially published Method 1668A in 1999. Since that time, the Agency has: • Revised the method in 2000 and 2003 to reflect user suggestions and peer reviews, • Published a study plan for the Method 1668A interlaboratory validation study in 2003, • Conducted the interlaboratory validation study in 2003-2004, • Published the peer reviewed validation study report, and • Published a revised method that reflects peer review and user suggestions and data. Additional details regarding the method revisions, the study, and the peer review are available in Revisions A and B of Method 1668, and the study plan, peer review report, validation study report, and validation study peer review report listed above. After Revision B was published, AXYS Analytical Services, Ltd. (AXYS) informed EPA that the revised initial precision and recovery (IPR) and ongoing precision and recovery (OPR) QC acceptance criteria for some congeners could not be met during routine laboratory operations because these criteria did not allow recoveries exceeding 100% for many congeners. (In developing the Method 1668B criteria, EPA set the upper limit on recovery to 100 percent for congeners for which a statistical analysis resulted in recoveries less than 100 percent.) EPA responded to this concern by using recently submitted QC data March 2010 ] ------- Addendum to the Method 1668A Interlaboratory Validation Study Report to examine the QC acceptance criteria published in Section 5 of the November 2008 interlaboratory study report and in Method 1668 Revision B. This examination resulted in revised criteria, as presented below. The examination also identified errors in Table 4-1 of the November 2008 report. These changes are documented in this addendum. Corrections to Table 4-1 of the Interlaboratory Validation Study Report The following is a corrected table. The corrections result from conversion of biosolids samples from dry weight to wet weight, so that the results from all four laboratories were reported on the same basis. To allow quick comparison with the values in Table 4-1 of the original validation study report, the corrected values in the table below are shown in boldface type. Table 4-1. Congener Detection Rates and Concentrations in Study Samples (by Matrix and Level of Chlorination) Matrix Biosolids Tissue Water LOG 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 # Labs 4 6 6 # Congeners Analyzed 24 88 160 240 237 254 169 81 24 8 36 131 232 352 347 362 240 114 35 12 36 128 233 356 344 362 235 116 35 12 # Congeners Detected 23 64 134 195 166 196 129 72 23 8 26 90 181 288 258 270 182 105 35 12 25 118 223 356 344 362 235 116 35 12 % Congeners Detected 96 73 84 81 70 77 76 89 96 100 72 69 78 82 74 75 76 92 100 100 69 92 96 100 100 100 100 100 100 100 Concentration (detects only)1'2 Mean 71 259 514 667 1090 602 362 195 161 166 4 47 267 402 418 429 276 157 162 200 27 533 1100 2850 2660 2190 1750 2410 1760 1740 Median 74 140 387 277 488 224 181 140 91 161 3 27 150 130 128 108 120 108 137 201 20 505 946 2170 1750 1660 1420 1740 1520 1510 Maximum 94 967 2370 4130 4720 4450 1670 583 630 193 12 188 1610 3330 15700 10700 3560 709 390 236 106 1460 3430 15300 21800 11800 7370 9560 3350 3170 'Biosolids concentrations in ng/kg (wet weight); tissue concentrations in ng/kg (wet weight); water concentrations in pg/L 2Mean, median, and maximum concentrations at each LOG are based on any detected congeners in that LOG. When coelution of two or more congeners occurred, the combined value of those co-eluted congeners was used. Revision of QC Acceptance Criteria In response to the information from AXYS, the IPR and OPR QC acceptance criteria were re- evaluated using more OPR data than were available from the Method 1668A interlaboratory validation study. Specifically, AXYS and TestAmerica-Knoxville (TestAmerica) provided EPA with large sets of March 2010 ------- Addendum to the Method 1668A Interlaboratory Validation Study Report OPR data they had generated as part of their routine sample analysis activities. Both sets of additional data were from analyses performed after completion of the Method 1668A interlaboratory study. AXYS provided 113 sets of results from aqueous and solid OPR samples, and TestAmerica provided 112 sets of results from aqueous and solid OPR samples. When these recent data were compared to QC acceptance criteria in Table 5-1 of the interlaboratory validation study report (and the identical Method 1668B criteria), EPA observed that: 1) failure rates were notably higher than expected for high chlorination level labeled analogs, and 2) the failure rate was higher than anticipated when assessed on a per-sample basis (i.e., if at least one congener would fail, the OPR would fail and, therefore, the OPR and associated batch of samples would have to be reanalyzed for all congeners). As a result, EPA used these data, along with the OPR data from the method validation study, to revise the QC acceptance criteria that were published in the 2008 validation study report and in Method 1668B. When determining QC acceptance criteria, it is assumed that all data used in the calculation are representative of the population of results from laboratories performing the method properly, and that any extreme results produced would be due to analytical variability, and not to laboratory issues. When using existing data to establish QC acceptance criteria, this assumption can be problematic because it cannot easily be determined whether a result is unusually high or low due to chance or due to a problem with the sample preparation or analysis. However, the large number of PCB congeners tested in an OPR sample (27 native and 27 labeled congeners) allows an assessment of the consistency of each individual sample with the overall dataset. An OPR sample for which the recoveries for many congeners are consistently higher or lower than those for other samples gives an indication that there may have been an issue with the analysis of that sample, whereas an OPR sample for which only one or two congeners yielded unusual recoveries is more likely to be failing by chance alone. Therefore, each OPR sample submitted by the two laboratories was assessed for a high frequency of unusually high or low recoveries. Based on this assessment, five OPR samples were removed from the dataset. After removal of the five OPR samples, all remaining data from the two laboratories were combined, and the distribution of recoveries was examined for each native and labeled congener. Based on this examination, three subgroups were identified that yielded similar distributions. These subgroups were defined as follows: 1) All native congeners 2) Labeled congeners 1 to 54 3) Labeled congeners 77 to 209 A revised set of OPR recovery criteria was chosen for each of these subgroups that would result in an approximate 5% failure rate on a per-sample basis. To further assess these chosen criteria, OPR results from the two laboratories were combined with the Method 1668A validation study data. This approach allowed the chosen criteria to be assessed using data with a larger between-laboratory variance component. Because there were many more OPR results from the two post-study laboratories than from the validation study laboratories, 100 sets of data were simulated. For each simulation run, four OPR results per congener were chosen randomly (such that the same sample would not be chosen for all congeners) from each of the two post-study laboratories. For each of these simulation runs, OPR criteria were calculated using the same formulas used to produce Method 1668B criteria from the method validation study. Because the resulting simulated QC criteria tended to be tighter than the nominal criteria, it was concluded that the added between-laboratory variability was not large enough to necessitate widening the chosen criteria. March 2010 ------- Addendum to the Method 1668A Interlaboratory Validation Study Report After choosing the OPR criteria, IPR criteria were chosen based on the OPR criteria. Unlike OPRs, which are evaluated on an individual sample basis, IPRs are analyzed and evaluated in sets of four. Because means of four measurements are less variable than single measurements, IPR recovery criteria tend to be tighter (by approximately 10-15%) than OPR recovery criteria. Based on this assumption, nominal IPR criteria were chosen for each of the three congener subsets that were approximately 10-15% tighter than the corresponding OPR recovery criteria. These criteria then were evaluated using the data submitted by the two post-study laboratories. Because these data comprised OPR samples only, IPR sets needed to be simulated to assess the IPR criteria. To do this, 100 sets of 4 OPRs were chosen randomly in order to simulate an IPR "set." The number of occurrences where an IPR set failed the nominal criteria was then determined for each congener. It would be expected that the failure rate could be slightly larger than the target 5%, because the simulated sets included a larger amount of temporal variability than a set of IPRs analyzed by a laboratory in practice (which are usually run within a single batch). However, the observed failure rates were well within the expected 5%, and therefore supported the chosen nominal criteria. Revision of QC Acceptance Criteria for Calibration Verification QC acceptance criteria for calibration verification were not revised based on the 2003-2004 interlaboratory study because calibration and calibration verification data were not gathered in the study. After completion of the interlaboratory study, calibration verification data were gathered from AXYS and TestAmerica. AXYS supplied results of analysis of 686 calibration verification samples and TestAmerica supplied results of 1160 calibration verification samples. Using a similar approach to that described for the OPR criteria modification, the results from the two laboratories were assessed to arrive at calibration verification criteria with an approximate per-sample failure rate of 5%. • The criteria for all native congeners were set to 75 - 125%, vs. 70 - 130% in Method 1668A • The criteria for labeled congeners 1L to 209L were set to 50 - 145%, vs. 50 - 150% in Method 1668A • The criterion for labeled cleanup standard 28L was set to 65 - 135%, vs. 60 - 130% in Method 1668A, and • The criteria for labeled cleanup standards 111L and 178L were set to 75 - 125% vs. 60 - 130% in Method 1668A. For labeled compounds 1L - 209L and the cleanup standards, the observed per-sample failure rate for the AXYS data was 5.3 percent and the observed per-sample failure rate for the TestAmerica data was 3.0 percent. The adjusted criteria are shown in Table A-l. Adjustment of QC Acceptance Criteria for OPR and Labeled Compound Recovery from Samples When the revised QC acceptance criteria for calibration verification, IPR, OPR, and labeled compound recovery from samples were compared, the upper limit of the calibration verification criteria was less restrictive than the upper limits of the OPR and the labeled compound recovery from samples. To make all criteria consistent, upper limits of criteria for OPR and labeled compound recovery from samples were increased to be greater than or equal to the upper limits of the calibration verification criteria. For example, for labeled congeners 1L - 54L, OPR criteria for recovery, based on a 5% failure rate, were 15 - 140%, necessitating an increase in the upper limit to 145%. The upper limits on IPR recovery for labeled compounds 1L - 54L and 77L - 209L were not increased to be greater than or equal to calibration verification criteria because an IPR consists of the average of results of 4 tests, whereas calibration verification is a single test. The adjusted criteria are shown in Table A-l. These revised criteria replace the criteria in Section 5 and Table 5-1 of the November 2008 report. March 2010 ------- Addendum to the Method 1668A Interlaboratory Validation Study Report Table A-1. Revised QC Acceptance Criteria for Calibration Verification, Initial Precision and Recovery (IPR), On-going Precision and Recovery (OPR), and Labeled Compound Recovery from Samples Congener set Native Toxics and LOCs Labeled congeners 1Lto54L 77Lto209L Cleanup standards 28L 111Land 178L Calibration Verification 75-125% 50-145% 50-145% 65-135% 75-125% IPR Recovery 70-130% 20-135% 45-135% 20-135% 45-135% IPR Precision 25% 70% 50% 70% 50% OPR Recovery 60-135% 15-145% 40-145% 15-145% 40-145% Labeled Compound Recovery from Samples NA 5-145% 10-145% 5-145% 10-145% NA = Not applicable Revision C to Method 1668 To accommodate the changes to the QC acceptance criteria presented in this addendum, EPA had the option of revising Method 1668B, or creating Revision C to Method 1668. The advantage to creating Method 1668C is that it minimizes confusion associated with multiple versions of Revision B. EPA has therefore chosen to create Method 1668C to include the revised QC acceptance criteria presented in this addendum. March 2010 ------- |