EPA Document # 815-R-05-006
Statistical Protocol for the Determination of the Single-Laboratory
Lowest Concentration Minimum Reporting Level (LCMRL) and
Validation of Laboratory Performance
at or Below the Minimum Reporting Level (MRL)
Prepared for:
Mr. David Munch
and
Ms. Phyllis Branson
U.S. Environmental Protection Agency
Office of Ground Water and Drinking Water
Standards and Risk Management Division
Technical Support Center
26 W. Martin Luther King Drive (MS-140)
Cincinnati, OH 45268
November 2004
-------
Protocol for the Determination of the LCMRL November 2004
Acknowledgments
This document has been prepared in conjunction with the Environmental Protection
Agency's (USEPA) Office of Ground Water and Drinking Water, Standards and Risk
Management Division, Technical Support Center (TSC). David Munch served as USEPA TSC's
Work Assignment Manager. Chris Frebis of USEPA's TSC, Elizabeth Hedrick of EPA's
National Exposure Research Laboratory/Office of Research and Development, and Stephen
Winslow and Barry Pepich of Shaw Environmental and Infrastructure, Inc. provided technical
guidance. The Cadmus Group, Inc. served as primary contractor. Dr. George Hallberg served as
Project Manager, and the document was prepared by John Martin. This document is the product
of more than one year of interactive input from the above named individuals, peer reviewers, and
laboratory personnel who participated in a proof-of-concept interlaboratory study from
November 2003-March 2004. The input of the following individuals is acknowledged:
Richard H. Albert, Ph.D., United States Food and Drug Administration (retired) - peer review of
initial documents used to design the interlaboratory study and in the determination of Lowest
Concentration Minimum Reporting Levels/Minimum Reporting Levels (LCMRLs/MRLs);
William Horwitz, Ph.D., AOAC International - peer review of initial documents used to design
the interlaboratory study and in the determination of LCMRLs/MRLs;
William T. Foreman, Ph.D., United States Geological Survey - peer review of initial documents
used to design the interlaboratory study and in the determination of LCMRLs/MRLs;
Andrew D. Eaton, Ph.D., MWH Laboratories - peer review of initial documents used to design
the interlaboratory study and in the determination of LCMRLs/MRLs and laboratory analyses;
William Telliard and personnel from EPA's Engineering and Analysis Division - peer review of
initial documents used to design the interlaboratory study and in the determination of
LCMRLs/MRLs;
Brad Venner of USEPA's National Enforcement Investigations Center - peer review of initial
documents used in the determination of LCMRLs/MRLs;
Michael D. Wichman, Ph.D., University Hygienic Laboratory - laboratory analyses;
Phillip Godorov, Philadelphia Water Department Laboratory - laboratory analyses;
John V. Morris, EPA Region 5 Laboratory - laboratory analyses; and
Lisa Wool, EPA Region 6 Laboratory - laboratory analyses.
Mention of trade names and/or laboratories does not constitute endorsement or
recommendation for use.
-------
Protocol for the Determination of the LCMRL November 2004
Contents
Acknowledgments i
Contents ii
Exhibits iii
List of Acronyms iv
1.0 INTRODUCTION 1
2.0 DETERMINATION OF LCMRL DURING METHOD DEVELOPMENT 2
2.1 Determination of the LCMRL 2
2.2 Diagnostic Procedures and Potential Issues 4
2.2.1 Diagnostic Procedures 4
2.2.2 Potential Issues 5
3.0 BACKGROUND AND STATISTICAL BASIS OF THE LCMRL 7
3.1 History of Selected Detection and Quantitation Procedures 8
3.2 Basis of the LCMRL 11
4.0 DETERMINATION OF MRLs FROM LCMRLs 12
5.0 EXAMPLE LCMRL DETERMINATIONS 12
6.0 MRL VALIDATION PROCEDURE SUMMARY 18
7.0 STATISTICAL BASIS OF MRL VALIDATION 19
8.0 EXAMPLE OF VALIDATION OF LABORATORY PERFORMANCE
AT OR BELOW THE MRL AND THE DAILY PERFORMANCE CHECK . . 21
9.0 REFERENCES 23
-------
Protocol for the Determination of the LCMRL November 2004
Exhibits
Exhibit 1: Critical Values of Q at the 99 % Confidence Level 7
Exhibit 2: Atrazine by EPA Method 507 LCMRL Determination by
Ordinary Least Squares All Data 14
Exhibit 3: Atrazine by EPA Method 507 LCMRL Determination by
Variance Weighted Least Squares All Data 15
Exhibit 4: Atrazine by EPA Method 507 LCMRL Determination by
Ordinary Least Squares Outlier Excluded 16
Exhibit 5: Atrazine by EPA Method 507 LCMRL Determination by
Variance Weighted Least Squares Outlier Excluded 17
Exhibit 6: Student's t Values for 5 to 10 Replicates 19
Exhibit 7: The Constant Factor (C) to be Multiplied by the Standard Deviation to Determine
the Half Range Interval of the PIR (Student's 199 % Confidence Level) 20
Exhibit 8: Use of PIR and QC interval with low-level data for EPA
Method 531.2 (carbamates) by HPLC 21
in
-------
Protocol for the Determination of the LCMRL
November 2004
List of Acronyms
AML Alternate Minimum Level
ASTM American Society for Testing Materials
CFR Code of Federal Regulations
HRPIR Half-Range Prediction Interval of Results
ICR Information Collection Rule
IDC Initial Demonstration of Capability
IDL Instrument Detection Level
IQCL Instrument Quality Control Level
IQEZo/0 Interlaboratory Quantitation Estimate (with Z% Relative Standard Deviation)
LCMRL (Single laboratory) Lowest Concentration Minimum Reporting Level
LRL Laboratory Reporting Level
LT-MDL Long Term Method Detection Limit
MCL Maximum Contaminant Level
MDL Method Detection Limit
ML Minimum Level
MQCL Method Quality Control Level
MRL Minimum Reporting Level
NWQL National Water Quality Laboratory
OGWDW Office of Ground Water and Drinking Water
OLS Ordinary Least Squares
PE(S) Performance Evaluation (Study)
PIR Prediction Interval of Results
PQL Practical Quantitation Limit
PWS Public Water Systems
QCL Quality Control Level
RSD Relative Standard Deviation
UCMR Unregulated Contaminant Monitoring Regulation
USEPA United States Environmental Protection Agency
USGS United States Geological Survey
VWLS Variance Weighted Least Squares
IV
-------
Protocol for the Determination of the LCMRL November 2004
1.0 INTRODUCTION
The Safe Drinking Water Act Amendments of 1996 require EPA to establish criteria for a
monitoring program and to publish a list of not more than 30 unregulated contaminants for which
public water systems (PWS) are to monitor. The monitoring program will provide a national
assessment of the occurrence of these contaminants in public drinking water, that will be used to
help decide which contaminants may or may not require regulation in the future. In 1999, EPA
revised the approach for unregulated contaminant monitoring in the Unregulated Contaminant
Monitoring Regulation (UCMR) (64 FR 50556; USEPA, 1999) and subsequent revisions.
PWSs will be required to monitor for a variety of contaminants under the UCMR. A
Minimum Reporting Level (MRL) will be assigned to each contaminant, and laboratories will be
required to report all occurrences of these contaminants at concentrations that are equal to or
greater than the established MRL. The MRL has been developed, in part, as an alternative to the
Practical Quantitation Limit (PQL) which, in the past, has been determined by either evaluating
EPA Performance Evaluation Study (PE) data or by applying a multiplication factor to the
Method Detection Limit (MDL) (as described at 40 CFR Appendix B to Part 136), that had been
established during the development of the analytical method. The MRL differs from the MDL
by considering not only the standard deviation of low concentration analyses (precision), but
also the accuracy of the measurements. In addition, since the privatization of PE programs by
EPA, the data that are required for PQL determination are no longer readily available. The MRL
was introduced with the new analytes and new methods for implementing the new UCMR. The
MRL may be useful as an alternative to the PQL for setting future regulatory limits, as well.
MRLs have often been determined by analytical laboratories using expert professional
judgement, but consistent criteria for MRL determination have not been established. In both the
Information Collection Rule (ICR) and the UCMR, OGWDW specified MRLs and an accuracy
requirement for recovery at the MRL so that data quality was documented daily. The most
difficult issue for the MRL has been developing a consistent procedure to set the MRL. EPA has
developed a statistical approach for determination of single-laboratory Lowest Concentration
MRLs (LCMRLs) using linear regression and prediction intervals. This approach has been
evaluated through expert review and through the performance of a pilot-scale interlaboratory
study.
The MRL is the lowest analyte concentration which demonstrates known quantitative
quality. The LCMRL, as calculated by the procedure presented in this paper, is the lowest true
concentration for which the future recovery is predicted to fall, with high confidence (99%),
between 50 and 150% recovery. A result below the MRL is considered to be an estimated value
that does not satisfy quality control objectives. It should be noted that the decision to report
estimated data is dependent upon the objectives of a study and not a point of discussion here.
In this paper, we present a systematic procedure for determining an LCMRL. The
LCMRL is used to determine the MRL for an analyte by using either a multiplying factor or by
-------
Protocol for the Determination of the LCMRL November 2004
pooling the results from a multi-laboratory study. Drinking water laboratories will confirm that
they are capable of meeting a required MRL during their Initial Demonstration of Capability
(IDC) using the procedure described in Section 6.0 of this paper. The LCMRL protocol is based
on a linear regression procedure as described in Section 2.0; the statistical basis for the LCMRL
is described in Section 3.0; possible procedures for the determination of MRLs from LCMRLs
are described in Section 4.0; example LCMRL determinations are presented in Section 5.0; the
statistical basis of MRL validation is presented in Section 7.0; and an example of validation of
laboratory performance at or below the MRL is presented in Section 8.0. It should be noted that
three distinct procedures are presented in this document:
LCMRL - determined by selected laboratories during method development. All
laboratories are encouraged to determine LCMRLs that are unique to their laboratory, as
this may aid them in establishing spiking concentrations for validation of performance at
or below an MRL; however, this is not required.
• Validation of laboratory performance at or below an MRL - performed by each
laboratory that is analyzing samples for UCMR as part of IDC.
Daily check at or below an MRL - performed daily by each participating laboratory to
demonstrate meeting Data Quality Objectives (DQOs) for all UCMR analytes (DQO =
within 50 - 150% recovery).
2.0 DETERMINATION OF LCMRL DURING METHOD DEVELOPMENT
This section includes a description of the steps in the process used to determine the
LCMRL for a particular analyte and several key procedural issues. This step is limited to
laboratories that are participating in method development and to those that desire to establish
laboratory-specific LCMRLs.
2.1 Determination of the LCMRL
The range of instrument calibration standards must encompass the levels being evaluated,
otherwise the determined LCMRL may be unreliable. The instrument calibration range and the
type of regression model used to fit the instrument calibration data for the LCMRL
determination must be used in the future as the normal day-to-day calibration for the analyte and
method in question. The calculated LCMRL cannot be lower than the lowest calibration
standard.
It is preferable that, at each of at least four levels of determination, a minimum of seven
replicate samples in reagent water are processed through the entire method procedure (including
extraction and all preservatives, where applicable). At an absolute minimum, five samples at
each of four concentrations, or seven samples at each of three concentrations are processed
through the entire method procedure. An initial estimate of spiking level should consider a
concentration of reliable quantitation and an analyte peak area at least three times greater than
-------
Protocol for the Determination of the LCMRL November 2004
that found in a blank sample processed through the entire method procedure.
The LCMRL is determined via the following five steps, using EPA's Internet-accessible
LCMRL calculator or commercially available statistical software with the requisite capabilities.
The following steps are performed by the LCMRL calculator, but are presented here for
transparency and to allow for the optional use of other statistical software.
For each analyte, the spiked concentrations (x-axis) are plotted against the measured
concentrations (y-axis);
The LCMRL data are regressed using Ordinary Least Squares (OLS), with a straight line
regression equation and a 99% prediction interval around the regression line. Do not
force the regression line through the origin. A test for constant variance over the range of
spiking concentrations must be performed. The threshold for passing the test of constant
variance is 1%. This corresponds to a probability of 0.01 for concluding that the variance
is not constant with respect to concentration when it actually is constant. If the data do
not pass the test of constant variance, a Variance Weighted Least Squares (VWLS)
regression model must be employed. Details regarding the test for constant variance and
the VWLS model are presented in Section 2.2.1 of this document.
• Plot or draw lines that correspond to 150% and 50% recovery of the spiked
concentration.
Drop a perpendicular to the x-axis starting from the point at which the upper prediction
interval line intersects the 150% recovery line. Drop a second perpendicular to the x-axis
starting from the point at which the lower prediction interval line intersects the 50%
recovery line. The location of the perpendiculars and the LCMRL are determined
mathematically as follows:
- Assume that the 99% prediction interval is a straight line between the two known
true (spiked) concentrations that encompass the intersection in question;
- Determine the slope (m) of the prediction interval between the two true
concentrations (m = Ay/Ax);
- For a given x and y, use m to calculate the y-intercept, b;
- For the upper recovery line, y = 1.5x; for the lower recovery line, y = 0.5x;
- Since at the intersection of the prediction interval boundary and the recovery line,
there is one value for y, mx + b = 1.5 x (for the intersection at the upper recovery
line) and mx + b = 0.5 x (for the intersection at the lower recovery line). The
LCMRL equals the larger value of x. If the prediction interval is contained
-------
Protocol for the Determination of the LCMRL November 2004
entirely within the 50 to 150% recovery range, the LCMRL is set equal to the
lowest spiked concentration. In cases where the prediction interval is located
entirely outside of the recovery range or is located outside the recovery range at
the highest spiked concentration, the LCMRL is indeterminable, at a
concentration that is greater than the highest spiking concentration.
• The LCMRL for a particular analyte is the larger of the two values indicated by the
intersection of the perpendiculars with the x-axis; however, the LCMRL cannot be less
than the lowest spiked concentration or lowest calibration standard for a particular
analyte.
2.2 Diagnostic Procedures and Potential Issues
To avoid the perception of a "black box" model, and to maintain transparency of this
statistical protocol, the instrument calibration regressions, the replicate analyses, and the
regression data should be analyzed to determine whether the regression and associated prediction
intervals have been appropriately modeled. For the purposes of this procedure, a test of constant
variance is considered to be the most appropriate diagnostic procedure for determining the
validity of the regression model.
2.2.1 Diagnostic Procedures
Prior to the analysis of the replicate samples, instrument calibration curves should be
evaluated for goodness-of-fit to the calibration data. Hence, each calibration curve must be
evaluated to determine whether the calibration curve data are linear. For some methods and/or
analytes, a quadratic regression equation may provide the better fit for the instrument calibration
data. The fit of the instrument calibration equation is indicated by the adjusted coefficient of
determination (R2) value. As the degree of the polynomial increases, the number of predictors
also increases. This results in a larger R2 value. While a larger R2 value implies a better fit, the
model with the larger adjusted}*2 value represents the better fit. The R2 value explains the
proportion of the variation in the dependent variable that is explained by the regression equation;
the adjusted}*? value accounts for the dependence of R2 on the degrees of freedom. Hence, the
adjusted}*? value addresses relative variance, where variance equals variation divided by the
degrees of freedom.
n- I
The adjusted coefficient of determination, R2adj = 1 - (1 - R2
\_n-p
where: n = the number of observations; and
p = the number of predictors, or the number of independent variables in the
regression equation (including the constant).
The best-fit regression equation may be necessary to produce linear results for the
LCMRL data. However, most analytical instrumentation software calculates either the
-------
Protocol for the Determination of the LCMRL November 2004
correlation coefficient (R) or the R2 value. If a value for R2 of 0.99 (i.e., an R value of 0.995) or
greater is obtained for the instrument calibration regression, the use of a better fit regression
model will likely have little effect on the LCMRL that is obtained. To obtain the lowest value
for the LCMRL, a linear polynomial regression equation must be employed during the analysis
of the replicate samples as part of the LCMRL determination process.
While several methods for testing the assumption of constant variance are available, the
Cook-Weisberg test is used in this procedure. A quantity, S, is computed by dividing the Mean
Sum of Squares (MSS) by two. S has a x2 distribution, with one degree of freedom. The data
pass the constant variance test if the probability (P) > x2 is 0.01 or greater. For a null hypothesis
of "the variance is constant", this means that there is a 1% probability of Type I error, or
concluding that the variance is not constant when it actually is constant. In the VWLS model,
the measured and spiked concentration data are weighted by dividing each value by the standard
deviation (i.e., the square root of the variance) of the measured concentrations at its
corresponding spiking concentration. An important assumption of VWLS is that the population
variance and standard deviation at each spiking concentration are known. The variance and
standard deviation that are obtained from the LCMRL replicate data (i.e., sample variance and
standard deviation) are used in this procedure. Another assumption inherent to this process is
that the replicate data are normally distributed.
2.2.2 Potential Issues
• Range of Spike Concentrations: The range of spike concentrations must be contained
within the range of instrument calibration standards, and this range must be used as the
routine daily calibration range for subsequent analyses. The range of instrument
calibration and spike concentrations should not exceed two orders of magnitude, except
in relatively rare cases of extended linearity, such as certain metals analyses, where the
analytical method and/or instrumentation specifies a broader range. If the magnitude of
the difference between the upper and lower spike concentrations is greater than
approximately 10-20 times, adjustment of the linear scale range may be necessary to
obtain the resolution needed to visually determine LCMRLs from the intersection of the
perpendicular with the x-axis.
• Linearity and Non-Linearity of Analytical Data: The replicate analyses may result in
a non-linear relationship between the spiked and measured concentrations for certain
analytes and/or analytical methods. For an analyte and/or analytical method, the
following are some of the factors that may result in non-linearity of the LCMRL
regression curve:
the better-fit model (i.e., straight line or quadratic) was not utilized for the
instrument calibration curve; and
the replicate analyses were performed using more than one calibration curve; and
-------
Protocol for the Determination of the LCMRL November 2004
As previously mentioned, it is useful to evaluate the fit of the instrument calibration
curves for each analyte prior to the analysis of the replicate samples so the LCMRL data
are as close to linear as possible. Linear and quadratic regression models should be
applied to the instrument calibration data, the adjustedR2 values should be compared,
and the model with the largest adjusted}*? values should be selected as the best fit
calibration model. However, if a value for R2 of 0.99 or greater (i.e., R > 0.995) is
obtained, the use of a better fit instrument calibration regression model will likely have
little effect on the resultant LCMRL.
• Outliers in the Data Set: The effect of a potential outlier on the determination of the
LCMRL depends on the magnitude of the residual error and the distance of the outlier
from the mean of the spiked data. While outliers may represent actual laboratory
conditions, the presence of outliers may result in artificially high or even indeterminable
LCMRLs. If the reason for an outlier is known and justified (e.g., analyst error), or if the
outlier is identified with specified (99%) confidence based on the results of Dixon's Q
Test (described below), an outlier may be omitted from the LCMRL determination on a
case-by-case basis. Note that the outlier exclusion process for LCMRL determination is
limited to a single outlier at a single concentration. Outliers may not be excluded from
data sets used to demonstrate validation of laboratory performance at or below an MRL.
While there are several tests that can be performed to evaluate outliers, Dixon's Q Test
has been widely proposed for application in analytical chemistry evaluations (Rorabacher, 1991)
and is relatively simple to apply. The test for detecting a single outlier in a data set is derived
from Rorabacher (1991) in Equation 1. The original terminology presented in the paper has been
modified somewhat for clarification:
\Xo - Xc\
Xhi - Xlo
where: x0 = the potential outlier;
xc = the closest value to the potential outlier;
xw = the highest value in the data set (including the outlier where applicable); and
xlo = the lowest value in the data set (including the outlier where applicable).
The calculated value of Q is compared to tabulated critical values of Q. If the calculated
Q exceeds the critical value of Q, the outlier is rejected at the confidence level from which the
critical value was taken. In this test, a confidence level of a = 99% was used. This corresponds
to rejecting only 1% of values as outliers when they are not truly outliers (Type I error). For the
testing of a single outlier, Q values for the r10 test, as presented in Table 1 of Rorabacher (1991)
were used. The designation "r10" refers to 1 outlier on one "end" of a data set, and 0 outliers on
the opposite "end" of the data set. The critical values of Q for n = 5 to 30 are presented in
-------
Protocol for the Determination of the LCMRL
November 2004
Exhibit 1 (Rorabacher, 1991).
Exhibit 1: Critical Values of Q at the 99 % Confidence Level
Replicates
5
6
7
8
9
10
11
12
13
14
15
16
17
Critical Value of Q
0.821
0.74
0.68
0.634
0.598
0.568
0.542
0.522
0.503
0.488
0.475
0.463
0.452
Replicates
18
19
20
21
22
23
24
25
26
27
28
29
30
Critical Value of Q
0.442
0.433
0.425
0.418
0.411
0.404
0.399
0.393
0.388
0.384
0.38
0.376
0.372
These critical values of Q are presented to allow for analysis of outliers for a variety of
LCMRL evaluations, from a single laboratory analyzing a minimum of five replicates, to a
pooled evaluation involving three laboratories, each analyzing ten replicates at the concentration
in question.
3.0 BACKGROUND AND STATISTICAL BASIS OF THE LCMRL
Confidence in quantitation depends on measurement accuracy as defined by precision
and bias. The determination of the quantitation limit has a lengthy history. Many past
procedures for quantitation limit determination used multiples of sample standard deviation of
blank signals or at low-level fortification, but did not consider the bias of measurement.
-------
Protocol for the Determination of the LCMRL November 2004
3.1 History of Selected Detection and Quantitation Procedures
International Union of Pure and Applied Chemistry (TUPAC) (Currie) Detection Limit
Procedure
The Currie detection limit procedure (Currie, 1968; Currie, 1999) describes three types of
detection limit relations:
Critical level (Lc). The critical level, Lc, is the lowest value that, with specified
confidence, does not result from a blank. The probability of exceeding Lc when analyte is absent
is a. A value for a of 0.01 signifies the interval at or above Lc should contain only 1% false
positives. The Lc is a minimum value of estimated net signal or concentration applied against
background noise.
Detection limit (LD). The detection limit (LD) is the minimum detectable value of the net
signal (or concentration) for which the false negative error is |3, which is the probability that a
true value at the LDis not measured as less than or equal to Lc. Given a normal distribution of
results, when samples contain an analyte at the LD, there is a 50% chance that analyzed results
will fall below this limit and not be reported (i.e., a false negative).
Determination or Quantitation limit (LQ). The quantitation limit (LQ) marks "the
ability of the chemical measurement process to adequately 'quantify' an analyte." Replicate
analysis at LQ will produce estimates with a relative standard deviation (%RSDQ), such as the
10% RSD mentioned by Currie.
Currie (IUPAC) procedure issues. Two issues with the IUPAC procedure are the lack
of bias accountability for the quantitation limit, and the difficulty with the determination of blank
variance in chromatographic methods. Since variance from replicate blanks determines the
region of reliable quantitation, there is not an accuracy requirement for the quantitation limit.
Measurement bias at low level is not addressed except to say that the bias bounds "require
skilled and exhaustive scientific evaluation of the entire structure of the chemical measurement
process."
The EPA Method Detection Limit
The EPA's Method Detection Limit (MDL) procedure (40 CFR 136, Appendix B) avoids
the problems of determining variance at zero concentration by fortifying samples at low levels
which must be 1 to 5 times the calculated estimated MDL. The MDL is defined as:
(2) MDL = r(n.u.a)*s
where: t = Student's t;
s = the standard deviation of replicate spikes at low-level;
-------
Protocol for the Determination of the LCMRL November 2004
1-a = the probability point; and
n-l= degrees of freedom.
The derivation is found in Glaser et al. (1981). The USEPA's MDL procedure uses the
standard deviation from low-level fortified replicates to estimate a confidence interval around
zero concentration that includes 99% of all false positives.
Standard Methods (17th Edition, 1989) Method Detection Procedure
The Standard Methods detection level procedure (APHA, AWWA, WPCF, 1989) uses
terms common to Currie's and USEPA MDL procedures. Standard Methods describes the
instrument detection level (IDL) as "the constituent concentration that is at least five times the
signal-to-noise ratio of the instrument." The IDL is determined using non-extracted standards.
Standard Methods refers to this as a "critical level," but is quite different from Currie's critical
level.
Standard Methods defines the lower level of detection (LLD) as "the constituent
concentration in reagent water that produces a signal 2 x (1.645) x s above the mean of blank
analyses" (i.e., twice the IDL). This sets both Type I error (a is the rate of false positives) and
Type II errors (p is the rate of false negatives) at 5%.
Standard Methods describes the method detection limit (MDL) in a manner similar to the
EPA MDL. Standard Methods qualifies the use of its MDL, by saying it "can be achieved by
experienced analysts operating well-calibrated instruments on a non-routine basis."
The level of quantitation (LOQ) is defined as "the constituent concentration that produces
a signal sufficiently greater than the blank that it can be detected within specified levels by good
laboratories during routine operating conditions." Typical concentration is "10 s above the
reagent water blank signal."
Long-Term Minimum Detection Level (LT-MDL) of the USGS NWQL
The United States Geological Survey National Water Quality Laboratory (NWQL) has
begun to use a reporting procedure based on a Long-Term Method Detection Level (LT-MDL)
(Childress et al., 1999). For the LT-MDL concentration, the risk of a false positive is set to no
more than 1 percent. At the LT-MDL concentration though, the risk of false negative is up to
50%. A laboratory reporting level (LRL) is set at twice the LT-MDL and concentrations
measured between the LRL and the LT-MDL are reported as estimated concentrations. The data
user is given the flexibility to censor the estimated data. Non-detections are censored at the
LRL. The LT-MDL is determined over an extended time by using all method instrumentation
and large number of replicate spike samples to obtain a more accurate and realistic measurement
of the standard deviation near the MDL.
-------
Protocol for the Determination of the LCMRL November 2004
Alternate Minimum Level (AML)
Gibbons, Coleman, and Maddalone suggested in 1997 the use of an alternate minimum
level (AML) which uses a multiple-concentration calibration procedure (Gibbons et al. 1997).
They felt that the standard deviation, s, used to calculate the minimum level (ML) was faulty
because s depended on the choice of spiking concentration due to non-constant variance. The
AML takes into account the uncertainty in the calibration function and in the standard deviation.
The determination requires spiked samples at levels below the initially estimated MDL
value to many times the estimated MRL. The determination of the AML involves 6 steps which
requires 11 calculations including an exponential regression model for standard deviation versus
concentration. While this is a very interesting procedure, the math may be too complex for
routine application. The AML calculations are sold as a software package by St@tServ -
Statistical Software at: http://www.statserv.com/softwares.html. It should be noted that the
AML is a detection-related value and not a quantitation-related value.
ASTM International's Interlaboratory Quantitation Estimate (IQEzo/o)
The following description is "adapted from ASTM D 6512-00 Standard Practice for
Interlaboratory Quantitation Estimate, copyright ASTM International, 100 Barr Harbor Drive,
West Conshohocken, PA 19083. The information is used with permission; ASTM International,
however, is not responsible for any changes made by the Exchange."
The IQEZO/0 is the lowest concentration for which a single measurement from a laboratory
selected from the population of qualified laboratories will have an estimated Z% RSD (relative
standard deviation), where Z is dictated by data quality objectives. This procedure uses a
regression approach to determine the point of 10% RSD among cross-lab mean values, with no
simplifying assumptions about the dependence of standard deviation on concentration. The IQE
is a minimum concentration at which most laboratories can be expected to reliably measure a
specific chemical contaminant during day-to-day analyses. The procedure is an interlaboratory
extension of the RSD approach used in the Gibbons AML procedure. In addition, the IQEZO/0
basically corresponds to the LQ (Currie, 1968), as the lowest concentration that produces Z%
RSD.
Quality Control Level (QCL)
The QCL was introduced in 1994 as a quantity that "determines the lowest concentration
that meets the data quality objectives of the data user in terms of the minimum acceptable
precision and accuracy" (Kimbrough and Wakakuwa, 1994). A Method Quality Control Level
(MQCL) is determined by first determining an Instrument Quality Control Level (IQCL) (i.e.,
interference-free) based on user-specified bias and RSD criteria. These same bias and RSD
criteria are then applied to matrix and method-specific conditions to determine the MQCL for a
given analyte, matrix, and analytical method. One estimate of the MQCL is determined from the
10
-------
Protocol for the Determination of the LCMRL November 2004
IQCL and any correction factors related to extraction, concentration, digestion, etc. A second
estimate is obtained by spiking aliquots of the matrix with analytes to generate solutions of
concentration that are equal to, less than, and greater than the first estimate of the MQCL. The
lowest concentration that meets the bias and RSD criteria is the second estimate of the MQCL.
A check is then performed at the second estimate of the MQCL to demonstrate whether the bias
and RSD criteria are met.
3.2 Basis of the LCMRL
The existence of several methods for establishing detection and quantitation levels has
created a need for uniformity in the process. EPA considered the procedures described in
Section 3.1, as well as others, and decided that a regression/prediction interval approach that also
combines desirable features of these procedures, with consideration for ease of application,
transparency, and cost, would best meet the objectives of the UCMR. Thus, the MRL described
in this paper is proposed as a quantitation metric that considers not only the standard deviation of
low concentration analyses (precision), but also the bias of the measurements. The predefined
QC interval and the confidence level of the Student's lvalue are quality assurance objectives that
can be tailored to fit future analytical and policy needs. The decision on how, or if, to report
values below the MRL will depend on the objectives of the study being conducted. The QC
interval of recovery chosen for use in this paper, 50 to 150%, is based upon experienced
judgement from chemical analysts. The prediction interval for the regression line that is derived
from the Student's t distribution was chosen as 99% because it is conservative, consistent with
other DQO's used in this procedure, it minimizes false positives, and is often used in other
statistical tests. It should be noted that this procedure is designed for data that are continuous
(e.g., Gaussian) rather than with data that are discrete, such as "counting" methods.
Ordinary Least Squares (OLS) is used to fit a regression of the measured concentrations
against the spiked concentrations. In a simple linear regression, OLS draws a line through the
data in a way that minimizes the sum of the squares of the deviations of the observed values
from the regression line. As the regression line may be a higher order polynomial, the general
form of the model is given by (StataCorp, 2001):
(3) y, = fa + fix, + &*? + •• •+ J3px? + ei
where: y; = the measured concentration for observation i;
x; = the spiked concentration for the observation i;
p = the order or degree of the polynomial (and hence the number of predictors);
and
e; = the residual error term.
11
-------
Protocol for the Determination of the LCMRL November 2004
The standard error (SE) is the sum of the residual error and the error of the prediction.
Given this standard error, a 99% prediction interval can be constructed around the regression
line. The 99% prediction interval is given by:
(4) y±tu-a,2.,-f-n*SE
where: y = the estimated value of y;
Vp-i) = tne value of the t distribution with (n-p-1) degrees of freedom that is
exceeded with a probability of (1-0.99)72, or 0.005; and
SE = the standard error.
The probability is (1-0.99)72 because there is a 1 percent probability the true value is
greater than or less than the predicted value; i.e., there is a 0.5% probability it is higher and a
0.5% probability it is lower. Thus, for a true concentration, x, a future measured concentration,
y, is predicted to fall, with 99% confidence, within the prediction interval described by Equation
4. Additional statistical background regarding the LCMRL is presented in "Evaluation of the
Lowest Concentration Minimum Reporting Level (LCMRL) and the Minimum Reporting Level
(MRL): Primary Analyte Analysis," which is available in the EPA Docket.
4.0 DETERMINATION OF MRLs FROM LCMRLs
During analytical method development as part of the UCMR, laboratories determine their
own values for the LCMRL. The mean of these LCMRL values was calculated for each analyte.
In cases where data from three or more laboratories were available, three times the standard
deviation of the LCMRLs was added to the mean of the LCMRLs. In cases where data from two
laboratories were available, three times the difference of the LCMRLs was added to the mean of
the LCMRLs. In statistical theory (Chebyshev's Inequality), three standard deviations around
the mean incorporates the vast majority (at least 88.9%) of the data points. In the case where
there are only two laboratories, the difference serves as a surrogate for the standard deviation,
because of the uncertainty in the estimate of the standard deviation with only two data points.
The MRL for each analyte was determined by rounding this number to two significant digits.
Note that this procedure differs from that presented in the LCMRL/MRL Evaluation report
(Primary Analyte Analysis), since the results of the primary analyte evaluation were used to
decide how LCMRLs would ultimately be determined from multiple laboratory data.
5.0 EXAMPLE LCMRL DETERMINATIONS
Exhibits 2-5 demonstrate the use of OLS and VWLS with and without an outlying datum.
As part of an interlaboratory study that was conducted to evaluate the LCMRL/MRL concept,
replicate data for atrazine by EPA Method 507 were generated by analyzing samples fortified at
12
-------
Protocol for the Determination of the LCMRL November 2004
0.05, 0.1, 0.4, and 0.5 [ig/L. Seven replicates were analyzed at each of these concentrations.
Note that the results of the constant variance test for the OLS regression presented in Exhibit 2
indicate that variance is not constant, but changes with concentration. For illustrative purposes,
the LCMRL is determined to be 0.39 |J,g/L; however, OLS is not appropriate for use in cases of
non-constant variance. The appropriate use of VWLS for these data is presented in Exhibit 3.
Note also that an outlying datum at a true (spiked) concentration of 0.4 ng/L is present in the
data sets that are used in Exhibit 2 and Exhibit 3. Exclusion of the outlying datum and
reevaluation of the LCMRL by OLS and VWLS is presented in Exhibit 4 and Exhibit 5,
respectively. Criteria for determining the validity of outlier exclusion in the LCMRL
determination process are presented in Section 2.2.2.
13
-------
Protocol for the Determination of the LCMRL
November 2004
Exhibit 2: Atrazine by EPA Method 507
LCMRL Determination by Ordinary Least Squares
All Data
o
c
o
O
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
150% Recovery Line
99% P.I. UpperBound
Regression Line
99% P.I. Lower Bound
50% Recovery Line
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
True Concentration (ug/L)
R = 0.9439 R2 = 0.8910 Adj.R2=0.£
y = 0.890(x) +0.023
Constant Variance Test: Failed (P>Chi2(l) = 0.0052)
LCMRL = 0.39 ng/
Since the variance is not constant, VWLS is used to regress the data as shown in Exhibit
14
-------
Protocol for the Determination of the LCMRL
November 2004
Exhibit 3: Atrazine by EPA Method 507
LCMRL Determination by Variance Weighted Least Squares
All Data
0.7
0.6
0.5
S 0.4
o
O
w
03
OJ
0.3
0.2
0.1
A
0.05 0.1 0.15 0.2 0.25 0.3 0.35
True Concentration (ug/L)
0.4 0.45 0.5
y = 0.909(x) + 0.024
LCMRL = 0.45 jjg/L
The presence of the potentially outlying datum at a true (spiked) concentration of 0.4
has a significant effect on the prediction interval and the resultant LCMRL. Exclusion of
the outlier allows for additional LCMRL determinations, as presented in Exhibit 4 and Exhibit 5.
15
-------
Protocol for the Determination of the LCMRL
November 2004
Exhibit 4: Atrazine by EPA Method 507
LCMRL Determination by Ordinary Least Squares
Outlier Excluded
0.7
0.6
0.5
S 0.4
o
O
w
03
OJ
0.3
0.2
0.1
U
I I I i i i
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
True Concentration (ug/L)
R = 0.9917 R2 = 0.9836 Adj. R2= 0.9829
y = 0.931(x) +0.024
Constant Variance Test: Failed (P>Chi2(l) = 0.001)
LCMRL = 0.16 ng/L
Despite exclusion of the outlier, the variance is not constant; hence, VWLS is used as
shown in Exhibit 5.
16
-------
Protocol for the Determination of the LCMRL
November 2004
Exhibit 5: Atrazine by EPA Method 507
LCMRL Determination by Variance Weighted Least Squares
Outlier Excluded
0.7
0.6
0.5
'£ 0.4
i=
o
O
0.3
0.2
0.1
0
J | | | | | L
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
True Concentration (ug/L)
y = 0.931(x) +0.023
LCMRL = 0.092
The value of the LCMRL of 0.092 (ig/L that is obtained by use of VWLS is lower than
the value obtained by the use of OLS. This is because in the OLS model, the width of the
prediction interval is highly influenced by the variance in measured values at the higher spiking
concentrations. When using VWLS, the measured and spiked concentration data are weighted
by dividing each value by the standard deviation of the measured concentration at its
corresponding spiking concentration. As a result, the shape of the prediction interval is strongly
influenced by the variance at each spiking concentration. The relatively smaller variance at the
lower spiking concentrations results in a value for the LCMRL that reflects this change in
variance with concentration. Criteria for determining the validity of outlier exclusion in the
LCMRL determination process are presented in Section 2.2.2.
17
-------
Protocol for the Determination of the LCMRL November 2004
6.0 MRL VALIDATION PROCEDURE SUMMARY
Laboratories using EPA drinking water methods would be required to demonstrate that
they are capable of achieving a required MRL. The procedure for validation of an MRL would
be incorporated into the Initial Demonstration of Capability section of the method.
The confidence level for the Student's t value and a QC interval (i.e., percent recovery)
will be defined by the data users of the study. For the purposes of this paper, the two-
sided confidence level for the Student's t value is 99% and the QC interval is 50 to 150%.
• Replicate analysis of at least seven spiked samples in reagent water are made at the MRL
and are processed through the entire method procedure.
• The MRL must be contained within the range of calibration.
A prediction interval of results (PIR), which is based on the estimated arithmetic mean of
analytical results and the estimated sample standard deviation of measurement results, is
determined by Equation 5 below (see also Section 7.0):
(5) PIR = Mean ±sx t(df^_a/2)
where: t is the Student's t value with df degrees of freedom and confidence level (1-a);
s is the standard deviation of n replicate samples fortified at the MRL; and
n is the number of samples.
• The values needed to calculate the PIR using equation 4 are:
i) number of replicates;
ii) Student's t value with a two-sided 99% confidence level and n number of replicates
(see Section 8.0);
iii) the average (mean) of at least seven replicates; and
iv) and the standard deviation of the replicate results.
The lower and upper result limits of the PIR are converted to percent recovery of the
concentration being tested. To pass criteria at a certain level, the PIR lower recovery
limits cannot be lower than the lower recovery limits of the QC interval, and the PIR
upper recovery limits cannot be greater than the upper recovery limits of the QC interval.
When the PIR recovery limits exceed the bounds of the QC interval of recovery, the
analyte fails at that concentration. If the PIR limits are contained within the bounds of
the QC interval, the MRL is validated for that analyte.
18
-------
Protocol for the Determination of the LCMRL
November 2004
During sample analysis, laboratories would need to run a daily check sample to
demonstrate that, at or below the MRL for each analyte, the measured recovery is within 50% to
150%, inclusive. The results for any analyte for which 50 to 150% recovery cannot be
demonstrated during the daily check would not be valid. Laboratories may elect to re-run the
daily performance check sample if the performance for any analyte or analytes cannot be
validated. If the performance for these analytes is validated, then the laboratory performance
would be considered validated. If not, or as an alternative to analysis of a second check sample,
the laboratory may re-calibrate and repeat the performance validation process for all analytes.
7.0 STATISTICAL BASIS OF MRL VALIDATION
"If a population is normally distributed with unknown mean and standard deviation, then
the mean and standard deviation (s) of a random sample could be used to form a prediction
interval for future observation.... If a random sample of size n is taken, a 100(1- a) percent
prediction interval can be written as..." (Dixon and Massey, 1983)
(5) Prediction Interval = Mean + s x t(df 1_a/2) x
where: t is the Student's t value with ^degrees of freedom and confidence level (1-oc),
s is the sample standard deviation of n replicate samples fortified at the MRL,
n is the number of replicates.
The values needed to calculate the PIR using Equation 5 are:
i) number of replicates;
ii) Student's lvalue with a two-sided 99% confidence level and n number of replicates
(Exhibit 6);
iii) the average (mean) of at least seven replicates; and
iv) and the sample standard deviation.
Exhibit 6: Student's t Values for 5 to 10 Replicates
Replicates
5
6
7
8
9
10
Degrees of freedom
(df)
4
5
6
7
8
9
Student's / Value
Confidence Level 99.0% (a/2
= 0.005)
4.604
4.032
3.707
3.499
3.355
3.25
Note: The critical t-value for a two-sided 99% confidence interval (as is used in this paper) is equivalent to the
critical t-value for a one-sided 99.5% confidence interval, due to the symmetry of the t-distribution.
19
-------
Protocol for the Determination of the LCMRL
November 2004
The factor:
is referred to as the Half Range PIR (HRPIR). For a certain number of replicates and for a certain
confidence level in Student's t, the factor:
t(df,.l-a/2) X Y+~
is constant, and can be tabulated according to replicate number and confidence level for the
Student's t. Exhibit 7 lists the constant factor (C) for replicate sample numbers 7 through 10 with
a confidence level of 99% for Student's t. The HRPIR is calculated by Equation 6:
(6) HRpm =
The PIR is calculated by Equation 7:
(7) PIR = Mean ± HR
PIR
Exhibit 7: The Constant Factor (C) to be Multiplied by the Standard Deviation to
Determine the Half Range Interval of the PIR (Student's f 99 % Confidence Level)
Replicates
7
8
9
10
Degrees of Freedom
6
7
8
9
Constant Factor (C) to be multiplied by
the standard deviation
3.963
3.711
3.536
3.409
Note: The critical t-value for a two-sided 99% confidence interval (as is used in this paper) is
equivalent to the critical t-value for a one-sided 99.5% confidence interval, due to the symmetry of the
t-distribution. PIR - Prediction Interval of Results.
20
-------
Protocol for the Determination of the LCMRL
November 2004
8.0 EXAMPLE OF VALIDATION OF LABORATORY PERFORMANCE AT OR
BELOW THE MRL AND THE DAILY PERFORMANCE CHECK
Using a QC interval of recovery of 50 to 150% of the MRL and the Student's t
confidence level of 99%, the PIR interval is calculated for aldicarb sulfoxide by entering the
following values from Exhibit 10 into the PIR equation:
i) the mean of results is 0.254 Lig/L;
ii) the Student's t value for 7 results and with a 99% two-sided confidence level is 3.707
(Exhibit 11);
iii) the standard deviation, s, is 0.0108; and
iv) the number of replicates, «, is 7.
Combining Equations 6 and 7:
PIR =0.254 ±0.0108 x 3.963
PIR = 0.254 ±0.043
The PIR lower result (0.254 Lig/L - 0.043 Lig/L = 0.211 Lig/L) is converted to a percent
recovery of the concentration being tested by dividing the PIR lower result by the spiked or true
concentration of 0.2 Lig/L and multiplying the result by 100. The percent recovery is calculated
to be (0.211 Lig/L / 0.2 Lig/L)*(100) = 106%, which is greater than 50% and satisfies the lower
limit requirement. The PIR upper result of 0.297 Lig/L is converted to a percent recovery of the
concentration being tested (0.297 Lig/L / 0.2 Lig/L)*(100) = 149%, which is less than 150%, and
satisfies the upper limit requirement. The laboratory passes the MRL validation requirement for
aldicarb sulfoxide at 0.20 Lig/L. As seen in Exhibit 8, oxamyl and carbofuran have PIR limits
that exceed the QC interval of 50 to 150% and are not validated at 0.20 Lig/L in this example.
Exhibit 8: Use of PIR and QC interval with low-level data for EPA Method 531.2
(carbamates) by HPLC
Aldicarb sulfoxide
Aldicarb sulfone
Oxamyl
Methomyl
3-HCF
Repli-
cates
7
7
7
7
7
True
value
Hfl/L
0.2
0.2
0.2
0.2
0.2
Mean of
Results
Mfl/L
0.254
0.204
0.240
0.207
0.195
Std
Dev,
s
Mfl/L
0.0108
0.0173
0.0168
0.0205
0.0064
PIR
Half-
range
Mfl/L
0.0428
0.0686
0.0666
0.0812
0.0254
PIR Lower
Limit
Result
Mfl/L
0.211
0.135
0.173
0.126
0.170
PIR Lower
Limit of
Recovery
spike
106%
67.5%
86.5%
63.0%
85.0%
PIR Upper
Limit Result
Mfl/L
0.297
0.273
0.307
0.288
0.220
PIR Upper
Limit of
Recovery
spike
149%
137%
154%
144%
110%
Passes
?
Yes
Yes
No
Yes
Yes
21
-------
Protocol for the Determination of the LCMRL
November 2004
Aldicarb
Propoxur
Carbofuran
Carbaryl
1-Naphthol
Methiocarb
Repli-
cates
7
7
7
7
7
7
True
value
Hfl/L
0.2
0.2
0.2
0.2
0.2
0.2
Mean of
Results
Mfl/L
0.201
0.203
0.192
0.180
0.210
0.186
Std
Dev,
s
Mfl/L
0.0138
0.0179
0.0341
0.0188
0.0176
0.0183
PIR
Half-
range
Mfl/L
0.0547
0.0709
0.1351
0.0745
0.0697
0.0725
PIR Lower
Limit
Result
Mfl/L
0.146
0.132
0.057
0.105
0.140
0.113
PIR Lower
Limit of
Recovery
spike
73.0%
66.0%
28.5%
52.5%
70.0%
56.5%
PIR Upper
Limit Result
Mfl/L
0.256
0.274
0.327
0.255
0.280
0.259
PIR Upper
Limit of
Recovery
spike
128%
137%
164%
128%
140%
130%
Passes
?
Yes
Yes
No
Yes
Yes
Yes
The confidence level for Student's t value is 99% and the QC interval of recovery is 50 to 150% of the level tested.
22
-------
Protocol for the Determination of the LCMRL November 2004
9.0 REFERENCES
American Public Health Association (APHA), American Water Works Association (AWWA),
and Water Pollution Control Federation (WPCF). 1989. Standard Methods for the Examination
of Water and Wastewater. 17th Ed., pp. 1-18 - 1-19 , 1030E, Method Detection Limit.
American Society for Testing Materials (ASTM) International. 2000. Standard Practice for
Intel-laboratory Quantitation Estimate. ASTM D 6512-00. ASTM International, 100 Barr Harbor
Drive, West Conshohocken, PA 19083.
Childress, C.J.O., W.T. Foreman, B.F. Conner, and T.J. Maloney. 1999. "New Reporting
Procedures Based on Long-Term Method Detection Levels and Some Considerations for
Interpretations of Water-Quality Data Provided by the U.S. Geological Survey National Water
Quality Laboratory," U.S. Department of the Interior, U.S. Geological Survey Open-File Report
99-193, Reston, Virginia.
Currie, L.A. 1968. "Limits for qualitative detection and quantitative determination: Application
to radiochemistry," Analytical Chemistry., 40, p. 586-593.
Currie, L.A. 1999. "Nomenclature in evaluation of analytical methods including detection and
quantification capabilities (TUPAC Recommendations 1995)," Analytical Chimica Acta, 391, pp.
105-126.
Dixon, WJ. and FJ. Massey, Jr. 1983. Introduction of Statistical Analysis. McGraw-Hill Book
Company, Fourth Edition, p. 92.
Gibbons, R.D., D.E. Coleman, and R.F. Maddalone. 1997. "An alternate minimum level
definition for analytical quantification," Environ. Sci. Technol., 31, 2071-2077.
Glaser, J.A., D.L. Forest, G.D. McKee, S.A. Quave, and W.L. Budde. 1981. Environ. Sci.
TechnoL, 15, 1426, December.
Rorabacher, D.B. 1991. Statistical Treatment for Rejection of Deviant Values: Critical Values
of Dixon's "Q" Parameter and Related Subrange Ratios at the 95% Confidence Level.
Analytical Chemistry, Vol. 63, pp. 139-146.
SPSS, Inc. 1998. SigmaPlof 5.0 Programming Guide. Chicago, IL.
StataCorp. 2001. Stata Statistical Software: Release 7.0 Reference Manual. College Station,
TX: Stata Corporation.
USEPA. 1999. Revisions to the Unregulated Contaminant Monitoring Regulation for Public
Water Systems; Final Rule. Federal Register. Vol. 64, No. 180. p. 50556, September 17, 1999.
23
------- |