CHAPTER 5
CALCULATION OF PRECISION, BIAS, AND
METHOD DETECTION LIMIT FOR CHEMICAL
AND^PHYSICAL MEASUREMENTS
Issued by
Duality Assurance ManagerM-nt and Special Studies Staff
Office of Monitoring Systems and Quality Assurance
Office of Research and Development
United States Environmental Protection Agency
Washington, D.C. 20460
March 3U, 1984
-------
6OOR841O8
TABLE OF CONTENTS
Section Page No.
5.1 INTRODUCTION (3 pages) 1
5.1.1 Purpose 1 of 3
5.1.2 Scope 1 of 3
5.1.3 Application 2 of 3
5.1.4 Contents 2 of 3
5.2 STATISTICAL CONCEPTS (5 pages) 1
5.2.1 Sample Statistics and Population
Parameters 1 of 6
5.2.2 Components of Data Quality 4 of 6
5.2.3 Components of Variance 5 of 6
5.3 PRECISION (14 pages) 1
5.3.1 Definitions 1 of 14
5.3.2 General Guidelines 2 of 14
5.3.3 Calculation of the Summary Precision
Statistics 3 of 14
5.3.4 Reporting Precision 11 of 14
5.3.5 Continual Precision Assessments 13 of 14
5.4 ASSESSMENT OF BIAS (9 pages) 1
5.4.1 Definitions 1 of 10
5.4.2 Measurement of Bias 4 of 10
5.4.3 Calculation of Bias Statistics 5 of 10
5.4.4 Reporting Bias 7 of 10
5.4.5 Continual Bias Assessment 7 of 10
5.5 METHOD DETECTION LIMIT (4 pages) 1
5.5.1 Discussion 1 of 2
5.5.2 MDL 1 of 2
-------
TABLE OF CONTENTS
(cont'd)
Section Page No.
5.6 A CASE STUDY (14 pages) 1
5.6.1 Within-Lot Summaries 2 of 14
5.6.2 Between-Lot Summaries 3 of 14
5.6.3 Regression Summaries and Reporting
Requirements for a Project or Time Period 7 of 14
5.7 SOURCES OF ADDITIONAL INFORMATION (6 pages) 1
5.7.1 Study Planning 1 of 6
5.7.2 Sampling 1 of 6
5.7.3 Assessment of Precision 2 of 6
5.7.4 Assessment of Bias 3 of 6
5.7.5 Use of Control Charts 4 of 6
5.7.6 Method Detection Limit 5 of 6
5.8 A GLOSSARY OF TERMS (3 pages) 1
5.9 RECOMMENDED FORMATS FOR REPORTING DATA
QUALITY (2 pages) 1
i i
-------
LIST OF ILLUSTRATIONS
Fi gu re Sect ion Page
No. No. No.
5-1 Precision Evaluation Samples 5.3 4 of 14
5-2 Plots of CV (%) versus X (CU) and
s (CU) versus X (CU) for Example 5-2. 5.3 7 of 14
5-3 Plots of CV L%) versus X (CU) and s
(CU) versus X (CU) for Example 5-3. 5.3 10 of 14
5-4 Plots of CV t%) versus X (CU) and s
(CU) versus X (CU) for Example 5-4. 5.3 12 of 14
5-5 The Use of Target Analyte Spikes for
Bias Estimation 5.4 3 of 10
5-6 Plot of Percent Recovery (Pi) versus
Concentration (Ti) for Sulfate Data
Quality Assessment Results 5.4 8 of 10
-
5-7 Plot of X (CU) versus T (CU) for Sulfate
Data Quality Assessment Samples
(Table 5-3) 5.6 9 of 14
5-8 Plot of Within-Lot Standard Deviation Sw
(CU) versus T (CU) for Sulfate Data
Quality Assessment Results (Table 5-3) 5.6 11 of 14
5-9 Plot of Between-Lot Standard Deviation sa
(CU) versus T (CU) for Sulfate Data
Quality Assessment Results (Table 5-3) 5.6 11 of 14
5-10 Plot of X (CU) versus T (CU) for Sulfate
Data Quality Assessment Results Reported
from August, 1980 to December, 1980 5.6 14 of 14
Table Section Page
No. No. No.
5-1 Relations Between Precision and
Concentrati on 5.3 6 of 14
5-2 Measurements and Within-Lot Summary
Statistics for Sulfate Data Quality
Assessment Solutions 5.6 4 of 14
5-3 Summary Statistics for Sulfate Data
Quality Assessments Solutions at
Fixed Concentrations, Concentration
Known 5.6 6 of 14
i;;
-------
Table
No.
5-4
5-5
5-6
LIST OF ILLUSTRATIONS
(conti d)
Summary Regressions for Sulfate
Analysis of Data from Prepared Solutions,
Reporting Period July, 1980 through
December, 1980
Measurement Data for Sulfate Filter
Samples
Summary Regressions for Sulfate Analysis
of Filter Data, Reporting Period August,
1980 to December, 1980
iv
Section
No.
5.6
5.6
5.6
Page
No.
9 of 14
11 of 14
12 of 14
-------
Section No. 5.1
Revision No. 0
Date: March 30. 1984
Page 1 of 3
Chapter 5
CALCULATION OF PRECISION, BIAS, AND METHOD DETECTION LIMIT
FOR CHEMICAL AND PHYSICAL MEASUREMENTS
5.1
I NTRODUCTI ON
One objective of the U.S. Environmental Protection Agency's
(USEPA) quality assurance proyram is to ensure that environmentally
related measurement data used by the Agency are of known and documented
quality. A most important measure of data quality is the variability
of the measurement system including the acts of sample acquisition
through sample analysis. One goal of the Ayency's QA Program is to
establish uniform procedures for calculating and reporting measurement
system data quality indicators (i .e., precision, bias, and method
detection limit).
5.1.1
Purpose
The purpose of this chapter is to present procedures for the
assessment and reporting of precision, bias, and method detection limit
(MDL) for environmentally related chemical and physical measurements.
These procedures should aid in the incorporation of such assessments
into all data acquisition activities and in reporting the assessments
in project reports and environmental data bases.
5.1.2
Scope
This chapter addresses the variability of the measurement system
includiny the activities of sample acquisition through analysis. It is
concerned with the error distributions of the routine measurement
system, not the distributions of the chemical and/or physical measure-
ments from a study project or a monitoring program. This chapter also
does not address the assessment of representativeness of the measure-
ments of the environmental conditions under investiyation (i .e., how
the measurement data relate to the target population).
All statistical concepts and analyses involve the use of data
quality assessment results obtained for the specific purposes of
-------
Section No. 5.1
Revision No. 0
Date: March 30, 1984
Paye 2 of 3
estimating data quality, i.e., bias, precision, and MOL. The proce-
dures discussed in this guideline are based on the assumption that
measurement errors for chemical and physical measurement systems are
normally or near normally distributed. It is stressed that for partic-
ular situations where this does not appear to be a valid assumption or
where problems occur outside the scope of this yuideline, e.g., as how
to treat outliers, the advice of a qualified statistician should be
obtained.
5.1.3
Application
Application of these procedures involves assessments of precision,
bias and MOL based on special measurements (hereafter referred to as
data quality assessments) of samples of known composition (e.g., refer-
ence materials, spiked samples, blanks) or of unknown composition
(e.g., replicate study samples or repeated analyses of study samples)
interspersed throuyhout the periods of routine operation of the
measurement system. Generally, study samples are received for analysis
in batches or lots (use of the term lots applies to any set or subsets
of data obtained from a study project or monitoring program) and data
quality assessments are made during the course of analysis of the study
samples in the lot. Application of the data quality assessments to the
measurements for the complete sample lot or groups of lots is based on
the assumption that all measurements, including the data quality
measurements, are made with the measurement system "in statistical con-
trol". A measurement system is considered to be in statistical control
when its variability is due only to chance causes. Data quality
assessment results used with control charts (Appendix H, ref. 13 of
Section 5.7.5) or statistical techniques such as the construction of
frequency distributions (Chapter 3, ref. 4 of Section 5.7.5) may be
used to assure that the measurement system is in statistical control.
The statistical measures of data quality prescribed in this chap-
ter should be used in conjunction with archived information on the
operational capability of measurement systems "Compilation of Data
Quality Information for Environmental Measurement Systems", (ref. 1,
Section 5.7.1) in the development of Quality Assurance Project Plans
for EPA environmental measurement programs as required in QAMS-005/80
(ref. 2 of Section 5.7.1).
-------
Section No. 5.1
Revision No. 0
Date: March 30, 1984
Page 3 of 3
5.1.4
Contents
This chapter includes: a discussion of basic statistical concepts
and definitions in Section S.2; procedures, with examples, for calcu-
lating and reporting estimates of precision, bias, and MOL in Sections
5.3. 5.4, and 5.5, respectively; and a case study illustrating the use
of the statistical procedures for calculating precision and bias is
given in Section 5.6. References, a Glossary of Terms, and recommended
formats for reporting data quality indicators are provided, in the
above order, in Sections 5.7 through 5.9.
-------
Section No. 5.2
Revision No. 0
Date: March 30. 1984
Page 1 of 6
5.2
STATISTICAL CONCEPTS
The principal indicators of data quality are bias and precision.
Bias is systematic error. Precision involves the closeness of
data values to each other. Accuracy involves closeness of measure-
ments to a reference value and incorporates both bias and precision.
To formalize these definitions. some basic statistical concepts are
presented and discussed in the ensuing paragraphs.
It is reiterated here that this chapter is concerned with the
error distributions of the measurement system and not the distribu-
tions of the chemical and/or physical measurements from a study project
or a monitoring program. The assessments of precision and bias, do,
however, apply to the measurements from a study project or monitori ng
program. All statistical concepts and analyses involve the use of data
quality assessments obtained for the specific purposes of estimating
data quality, i.e., bias, precision, and MOL.
5.2.1
Sample Statistics and Population Parameters
5.2.1.1 Sample Statistics--
If a quantity X (e.g., a concentration) is measured n times, it is
customary to refer to the values Xl, X2, . . . Xn so obtained as
a sample of size n of measurements of X. (It should be clear
throughout the text from the context whether the word "sample" is used
in this statistical sense or refers to, e.g., a chemical sample.)
Values obtained in the calculation of quantities which summarize data
of this kind are summary statistics or sample statistics.
More specifically, let Xl, X2, . . ., Xn be a set of independent
data quality assessments taken under fixed and prescribed experimental
conditions and regarded as a random sample from some population. For
many applications the Xi are considered measurements aimed at esti-
mating a reference or true value T. Some basic statistics for a sample
of size n are:
o the sample average X = (Xl + ... + Xn)/n
o
the sample bias B = X - T
Eq. 5-1
Eq. 5-2
o
the sample standard deviation s
=J n:1
n
L (X.-X}2
. 1 1
1=
Eq. 5-3
-------
Section No. !j.2
Revision No. 0
Date: March 30, 1984
Page 2 of 6
o the sample variance is s2
o
the sample coefficient of variation (alsQ called
relative standard deviation) CV = 100 siX
Eq. 5-4
o
the error of the ith measurement ei = Xi - T,
Eq. 5-5
Sample standard deviation and variance, sand s2, are measures
of precision with smaller values of s indicating better precision. If
the measurements Xi are widely dispersed, the values of (Xi-X)2 in Eq.
5-3 will tend to be large, giving a large value of s2 or s; whereas,
a small spread gives small values. This is illustrated in the follow-
ing diagram.
Case 1
x x x x x
I
X
x x
x
s 1 arge
Case 2
xxxxx I x x x
X
s small
In the above two cases, the measured values Xi are indicated by the
crosses, and the sample averages are the same. For the specific appli-
cation of measurement system error, it is assumed that results of
repeated, independent. measurements of the same sample, under the same
specified conditions, will be normally or near normally distributed
about their average value. There are several techniques for checking
for normality of a set of results including the use of histograms.
normal probability graphs (see Appendices C and D, respectively of
reference 13 in Section 5.7.5), and goodness-of-fit tests. The
assistance of a qualified statistician may be required to develop a
detailed protocol.
The sample size n and values of Band s are fundamental summary
statistics and should be reported in routine sampling situations.
These values may be calculated automatically using a statistical com-
puter (or calculator) program. The following example shows the
IImechanicsll of the calculation.
Example 5-1. Calculation of Band s: A reference chemical sample
at an assumed concentration, T = 50 concentration units (CU), was
analyzed four times to assess the bias and precision of the analysis
phase of the measurement system. The values Xl' X2, X3, X4 obtained
were 48, 55, 5U, and 45 CU. Values of Band s may be calculated as in
the following table:
-------
Section No. 5.2
Revision No. 0
Date: March 30. 1984
Page 3 of 6
-
Measurements X. X'-X
1 1
i = 1 48 -0.5
2 51 2.5
3 50 1.5
4 45 -3.5
(Xi-X)2
0.25
6.25
2.25
12.25
Average X = (X1+X2+X3+X4)/n
= (48+51+50+45)/4
= 194/4
X = 48.5 CU
4
Variance s2 = E (Xi-X)2/n-1
i=1
= (0.25+6.25+2.25+12.25)/3
s2 = 21/3 = 7.0
s = 2.6 CU
Bi as
B = X-T
= 48.5-50
B = -1.5 CU
Based on these results the estimated bi as of the measurement sys-
tem, under the above conditions, is -1.5 CU and the estimated preci-
sion, expressed as the standard deviation, is 2.6 CU. On a practical
basis the bias is not considered to be significant because the assumed
value T is contained within the range of the four measured values.
5.2.1.2 Population Parameters--
If a random variable X is measured many times, the calculated sam-
ple statistics will approach constant values referred to as popula-
tion parameters. For example, when n is very large, the sample
average X of n measurements approaches a value called the
population mean, or simply the mean, which is denoted by the
symbol~. In the absence of measurement bias, ~ is the true value of
the quantity being estimated. Thus, The appropriate estimator for the
population mean ~ based on n measurements is the sample average X. The
value of X generally will not coincide with ~ but does give the
best estimate based on the measurements available.
Similarly, if many measurements are made, the sample variance s2
approaches the population variance which is customarily denoted by
u2. Its square root, u, is the population standard deviation
(s.d.). Again, u and u2 are generally not known but may be estimated
-------
Section No. 5.2
Revision No. 0
Date: March 30, 1984
Page 4 of 6
by the sample statistics sand s2 (provided at least two measurements
are made since n ~ 2 is required to apply Equation ~-3).
The quantities ~. u, and u2 are called population parameters.
Another commonly used parameter is coefficient of variation (CV)
or relative standard deviation (r.s.d.). This is the ratio 100 u/~
which expresses the standard deviation as a percentage of the mean and
is sometimes useful when the standard deviation changes with levels
being measured. It is estimated by the ratio lUU siX, the
samp le CV.
The following table summarizes the population parameters
described above and the sample statistics used for their estima-
tion.
Population Sample
Parameter Symbol Statistic Symbol
-
mean ~ averaye X
standard deviation u standard deviation s
variance u2 variance s2
CV (or r.s.d.) IOU u/~ CV 1UU siX
It should be emphasized that there are situations with more struc-
ture than that of a simple random sample from a single population.
However, as previously noted, for the most part, this overview will be
concerned with one or more IIsimplell samples, rather than more struc-
tured situations.
5.2.2
Components of Data Quality
5.2.2.1
Precision--
Precision is a measure of the scatter of a group of measure-
ments, made at the same specified conditions, about their average
value. The sample standard deviation s and sample coefficient of
variation CV are used as indices of precision. The smaller the
standard deviation and coefficient of variation, the better the
precision. Precision is stated, in the units of measurement or as a
percentage of the measurement average, as a plus and minus spread
around the average measured value X.
-------
Section No. 5.2
Revision No. 0
Date: March 30, 1984
Page S of 6
In instances where precision estimates are obtained from analyses
of replicate pairs, the range R (maximum value - minimum value)
or relative range RR (10U R/X) is sometimes used as an index
of precision. For replicate pairs the relationship between the
range and standard deviation is s = R/ 12.
5.2.2.2
Bias--
Bias, as estimated with sample statistics, is the signed
difference between the average X of a set of measurements of a
standard and the "true" value of the standard T gi ven by
-
B=X-T.
Bias can be negative or positive and is expressed in the units of
measurement or as a percentage of the value of the standard. Percent-
age bias is given by
%B = 1UU (X - T)/T.
Bias is also estimated by average percent recovery P (see
Section 5.4.1 for a definition of percent recovery). The relationship
between percent bias and average percent recovery is:
%B = P - 100.
5.2.3
Components of Variance
For any measurement system there are many sources of variation or
error, some of which are sample collection, handling, shipping, stor-
age, preparation, and analysis. For each individual or groupings of
error sources within a measurement system, there are different classi-
fications of variation or precision. Different classifications are the
result of the different conditions and manner in which the data quality
assessments are made for estimating precision.
Intralaboratory precision is the variation associated with a
single laboratory or organization. Intralaboratory precision must be
further subclassified as short-term or long-term precision de-
pending on the conditions and manner in which the precision data are
obtained. Intralaboratory precision is usually referred to as
-------
Section No. 5.2
Revision No. U
Date: March 30, 1984
Page 6 of 6
repeatability. Interlaboratory precision or reproducibility is
the variation associated with two or more laboratories or organizations
using the same measurement method.
When precision information is used to assess system performance,
care should be taken that the appropriate measure and classification of
precision is used. When reporting precision estimates for environment-
al measurements, the conditions represented by the estimate (e.g.,
short- or long-term, single analyst, etc.) should be documented, and
the component or components of the measurement system included in the
estimate (e.g., the total system, sample preparation and analysis, or
analysis only) should be specified.
-------
Section No. 5.3
Revision No. a
Date: March 30, 1984
Page 1 of 14
5.3
PRECISION
5.3.1
Definitions
The most commonly used estimate of precision is the sample stan-
dard deviation, as defined in Section 5.2. Other precision measures
include the coefficient of variation (100 siX). range R (maxi-
mum value - minimum value), and relative range RR (100 R/X).
Collocated samples are independent samples collected in such a
manner that they are equally representative of the variable(s) of
interest at a given point in space and time. Examples of collocated
samples include: samples from two air quality analyzers sampling from
a common sample manifold or two water samples collected at essentially
the same time and from the same point in a lake.
A replicated sample is a sample that has been divided into two
or more portions, at some step in the measurement process. Each por-
tion is then carried through the remaining steps in the measurement
process.
A split sample is a sample divided into two portions, one of
which is sent to a different organization or laboratory and subjected
to the same environmental conditions and steps in the measurement pro-
cess as the one retained inhouse.
Collocated samples when collected, processed, and analyzed by
the same organization provide intralaboratory precision information
for the entire measurement system including sample acquisition, han-
dling, shipping, storage, preparation and analysis. Both samples can
be carried through the steps in the measurement process together pro-
viding an estimate of short-term precision for the entire measurement
system. Likewise, the two samples, if separated and processed at dif-
ferent times or by different people, and/or analyzed using different
instruments, provide an estimate of long-term precision of the entire
measurement system.
Collocated samples when collected, processed and analyzed by
different organizations provide interlaboratory precision informa-
tion for the entire measurement system.
-------
Section No. 5.3
Revision No. 0
Date: March 3U, 1984
Page 2 of 14
A replicated or split sample can be divided into portions (or
split) at different points in the sampliny and analysis process in
order to obtain precision information on the various components of the
measurement system. For example, a field replicated, or field
split sample, provides precision information about all steps after
sample acquisition including effects of storage, shipment, analysis,
and data processing; whereas, information on the intra- and interlabo-
ratory precision of sample preparation and analysis steps of the
measurement system is provided by samples subdivided once they are
received in the laboratory, i.e., laboratory replicated or laboratory
split samples, respectively. A sample divided into two portions just
prior to analysis, i.e., an analysis replicate, provides informa-
tion on the precision of the analytical instrumentation.
The replicated sample can provide short-term or long-term pre-
cision estimates by processing the two portions together or separating
them for processing at different times and under different conditions
as discussed above for collocated samples.
5.3.2
General Guidelines
The precision assessment should represent the variability of the
entire measurement system. Therefore, collocated samples are recom-
mended, when possible, as the preferred method of assessing precision
of the entire measurement system.
A sample subdivided in the field and preserved separately is used,
where possible, to assess the variability of sample handliny, preserva-
tion, and storage along with the variability of the analysis process.
If the nature of the matrix, sample acquisition procedure, or analyti-
ca 1 techni que prevent the assessment of the entire measurement system,
the replicated samples used to assess precision should be selected to
incorporate as much of the measurement system as possible.
Data quality assessments should be made at concentration levels
typical of the range observed in routine analyses using the same method
in the same laboratory. In situations where the typic~lly observed
measurement values are zero but a standard is set at some finite value,
repetitive measurements should be made of samples representative of the
standard and not zero. The design of the data quality assessment pro-
gram will depend upon such factors as the data quality needs, the
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1~84
Page 3 of 14
precision of the measurement system, and the size of the sample lot.
For large sample lots, a fixed frequency for replicate measurements
(such as one sample in ten or twenty) is recommended. For small sample
lots, more frequent repetition may be desirable to ensure that suffi-
cient data are available to assess precision. Alternatively, multiple
sample lots with a common matrix, analyzed by the same measurement sys-
tem, can be combined as discussed under continual precision assessments
(Section 5.6). If the environmental measurements normally produce a
high percentage of results below the MDL (see Section ~.5), samples for
replicate measurement should be selected from those containing measura-
ble levels of analyte. Where this is impractical, such as with complex
multi-analyte methods, sample replicates may be spiked at appropriate
concentration levels to ensure that sufficient data will be available
to assess precision.
Precision information can be dealt with in a variety of ways
dependiny on the specific situation of interest and the type of data
available.
Interpretation of precision data must always be based on a clear
knowledge of how the data were created. For example, the precision of
the entire measurement system, including sample acquisition, can only
be assessed by analyses of collocated samples. Precision data yener-
ated from multiple analyses of a standard only describe the stability
of the measurement device or instrument and only represent the ultimate
precision which could be achieved for a field sample if the sampling
activity, subsequent sample preparation steps, and the sample matrix
had no impact on final results.
Figure 5-1 graphically outlines these samples in a general sense,
but specific sampling and analysis situations may require additional
precision information and more extensive breakdown of precision evalua-
tion samples. If this is the case, a clear indication of what is being
done and why should be provided in data quality assessment documenta-
tion.
5.3.3
Calculation of the Summary Precision Statistics
Summary statistics provide an assessment of the precision of a
measurement system or component thereof for a project or time period
(i.e., for a sample lot or group of sample lots). They may be used to:
-------
Data Quality Assessment Sample Point of Origination
Sample Acquisition
I Collocated I
Samples
I Field I
. RePlicate,
Field
Split
Preparation
Analysis
I Rep ~r~at e I
Lab
Sp 1 it
Analysis
Replicate '-
Analysis
Sp 1 it
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Paye 4 of 14
Oata Interpretation
Best estimate of intra- or
interlaboratory precision of
the ent ire measu rement system.
Second best estimate of
intralaboratory precision
of measurement system from
sample acquisition throuyh
analysis.
Second best estimate of
interlaboratory precision of
the measurement system from
sample acquisition throuyh
analysis.
Best estimate of intralabora-
tory precision of sample pre-
paration and analysis.
Best estimate of interlabora-
tory precision of sample pre-
paration and analysis.
Estimate of intra laboratory
precision of analysis.
'-
Estimate of interlaboratory
precision of analysis.
F i gu re 5-1
Precision Evaluation Samples.
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Page 5 of 14
o
estimate precision at discrete concentration levels.
o
average estimated precision over applicable concentration
ranges.
o
provide the basis for a continual assessment of precision of
future measurements.
The summary statistics are developed from the basic statistics
gathered throughout the project or time period re~resented. Because
the precision of environmental measurement systems is often a function
of concentration (e.g., as concentration increases, standard deviation
increases), this relationship should be evaluated before selecting the
most appropriate form of the summary statistic. An evaluation of the
basic precision statistics as a function of concentration will usually
lead to one of three conclusions:
o Case 1:
standard deviation (or range) is independent of con-
centration (i.e., constant);
o Case 2:
standard deviation (or range) is directly proportional
to concentration, and coefficient of variation (or
relative range) is constant; or
o Case 3:
both standard deviation (or range) and coefficient of
variation (or relative range) vary with concentration.
For simplicity of use and interpretation, the relationship most
easily described should be selected for use, i.e., for Case 1 the
standard deviation (or range) is simplest to work with; whereas, for
Case 2, the coefficient of variation (or relative range) is simplest.
If the relationship of precision to concentration falls into Case 3,
regression analysis can be used to estimate the relationship between
standard deviation (or range) and concentration.
The decision as to which case is applicable can be based on ~lots
of precision versus concentration or by regressions of s (or R) or CV
(or RR) versus concentration by an approach summarized in Table ~-1 and
illustrated in the following examples.
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Page 6 of 14
TABLE 5-1.
RELATIONS BETWEEN PRECISION AND CONCENTRATION
Case Plot Plot of Slopea I ntercepta
of s vs X 100 siX vs X b1 bo
1. Constant s appears CV d~creases 0 >0
Standard constant for as X
Deviation all X increases
2. ConstaDt CV, s- increases as CV appears >0 0
100 siX X increases constant
for all X
3. Other s- increases as CV d~creases >0 >0
X increases as X
increases
a Based on the relationships of:
s = b1 X + bo, and
CV = 100 siX = 100 (b1 + bo/X)
Legend:
The slope b1 and intercept bo are parameter estimates from the
linear r~gression of standard deviation s on averaye estimated concen-
tration X. To decide among the cases, use one of the following two
methods.
Method 1: Regress s on X and base the decision on statistical
tests of the hypotheses b1 = O. bo = O. The assistance of a quali-
fied statistician may be required.
Method 2: Review plots of standard deviation vs X and 100 siX versus
X. If one plot appears more constant than the other, then elect
that case. If nonconstancy is evident from both plots, regress s on
X to get the precision relationship. The assistance of a qualified
statistician may be required.
Example 5-2 (Case 1. s is independent of concentration): Three
samples in a la-sample lot, whose concentrations approximated the range
of concentrations of all ten samples, were each analyzed in triplicate
to assess the precision of analysis. At estimated concentrations of
20, 32, and 40 CU, the standard deviations were calculated to be 2.2,
2.6, and 2.4 CU, respectively. Based on the plots in Figure 5-2, s
appears to be independent of concentration; whereas, CV definitely
-------
cv
12
8
8
18
S
2.8
2.8
2.4
2.2
2.8
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Paye 7 of 14
x
x
x
28
3B
48
x
x
x
x
18
28
3B
48
Figure 5-2.
x
Plots of CV (%) versus X (CU) and s (GU) versus X (CU)
for Example 5-2.
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Page 8 of 14
decreases with increasing concentration so
sis for all samples in the lot may be most
the pooled standard deviation s = 2.4 CU.
tion is calculated as:
that the precision of .analy-
conveniently summarized by
The pooled standard devia-
s = [(n1-1) sf+ (n2-1) s~+...+ (nk-1)s~ ] 1/2,
n1+ n2+ ... + nk-k
Eq. 5-6
and for this example
s = { [2(2.2)2 + 2(2.6)2 + 2(2.4)2J/(3 + 3 + 3 - 3)} 1/2
s = 2.4 CU.
The standard deviation of individual measurements in this 10- sam-
ple lot then is reported to be 2.4 CU.
Example 5-3 (Case 2, s is directly proportional to concentration
and CV is constant): For a study involving a 100-sam~le lot, 8 pairs
of field replicated samples were ~rocessed and analyzed to assess the
precision of the measurement system from sample acquisition throuyh
analysis. The results for each pair of samples (Xl and X2) are
tabulated in order of increasing Xl, below along with values for
replicate average, standard deviation, and coefficient of variation.
Average Standard Coefficient of
Xl X2 (X1+ X2)/2 Deviation Variation (%)
1.5 1.7 1.6U .14 9
1.7 1.6 1.65 .U7 4
2.U 2.1 2.05 .U7 3
2.4 2.1 2.25 .21 9
2.7 2.4 2.55 .21 8
3.9 4.3 4.10 .28 7
5.0 4.!:> 4.75 .35 7
5.2 4.7 4.95 .35 7
Average CV = 7%
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Page 9 of 14
An inspection of the data tabulation of the standard deviation and
average for each pair suggests that the standard deviation increased as
the average increased. Therefore, the coefficient of variation values
were tabulated. An inspection of the tabulation of CV versus av~rage
reveals no clear relationship, i.e., the CV can be treated as a con-
stant across the concentration range. These findings are confirmed by
the plots in Figure 5-3. Therefore, the average CV (7%) is .used as the
precision summary for this sample lot. The precision, expressed as a
standard deviation, of individual measurements in this example, at any
concentration, X, is estimated to be, and is reported as, s = U.07X.
Example 5-4 (Case 3, s increases with increasing concentration and
CV decreases with concentration): If a constant relationship does not
appear to exist between standard deviation and concentration or between
coefficient of variation and concentration, then it is necessary to use
a more com~lex a~proach, such as a linear regression equation, to
describe the relationship. A least-squares linear regression analysis
of the precision (i.e., s or CV) versus measured concentration results
in two coefficients, a slope and an intercept, which are used to repre-
sent the precision of the data set. The ranye and relative range are
sometimes used to estimate precision, particularly for replicate pairs,
because of their ease of computation and use.
For a study involving a lUU-sample lot, 10 collocated sample pairs
were collected for estimating the precision of the entire measurement
system. The results for each set of collocated samples are tabulated
below alony with values for the averaye, range, standard deviation,
relative range, and coefficient of variation of each pair of samples.
To simplify visual interpretation, the data have been ordered by
increasing values of concentration.
Average Standard Relative Coeffi ci ent of
Xl X2 (X1+X2)/2 Range Deviation Range Variation (%)
5.33 6.37 5.H5 1.04 0.735 17.8 12
10.1 H.65 9.38 1.45 1.U3 1!:>.!> 11
19.5 17.6 18.55 1.9 1.34 1U.2 7
1H.6 2U.5 19.!>!:> 1.9 1.34 9.H 7
32.8 36.1 34.4!:> 3.3 2.33 9.6 7
10H.!:> 1U2. 1U5.2 6.!> 4.6U 6.2 4
132. 124. 128.U 8.U 5.66 6.2 4
186. 197. 191.!:> 11. 7.7H 5.7 4
!:>01. 527. 514.U 26. 18.38 !:>.1 4
3517 3341 3429 176 124.43 5.1 4
-------
cv
18
8
2
8
S
8.35
8.28
8.21
8. t..
8.87
8.H
8
Figure 5-3.
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Paye 10 of 14
x
x
x
x
x
x
x
x
2
..
5
3
x
x x
x
x
x
x
x
x
2
3
5
..
x
-
Plots of CV (%) versus X (CU) and s (CU) versus X (CU)
for Example 5-3.
-------
Section No. 5.3
Revision No. 0
Date: March 30. 1984
Page 11 of 14
For organizations that prefer working with standard deviation and/
or coefficient of variation, plots of standard deviation versus average
concentration and coefficient of variation versus average concentration
are shown in Figure 5-4. These plots show that s is an increasing,
approximately linear function of concentration whereas CV is a decreas-
ing, nonlinear function of concentration. In this situation it is sim-
pler to use a linear regression equation to represent the standard
deviation over the concentration range. The calculated regression
equation is s = 0.036 X + 0.698. The individual measurements in
this example are estimated to have, at any concentration X, a standard
deviation of 0.036 X +0.698 CU.
For organizations that prefer working with range and relative
range, the tabulation shows a clear increase in range, and decrease in
relative range, with increasing concentration. A least-squares linear
regression of range as a function of the average concentration for the
data above yields the following regression equation R = 0.U51 X
+ U.987. For replicate pairs the standard deviation and range are
related by s = R/~
5.3.4
Reporting Precision
Because each data user must
required for his/her application,
standard deviation or alternative
determine the data reliability
the data reporter must provide a
measure of precision which applies to
measurement.
The data user should be provided with a narrative statement docu-
menting the conditions and manner in which the precision data were
obtained and the applicable component or components of the measurement
system. Statements for reporting precision estimates for cases 1,2,
and 3 as illustrated in examples 5-2, 5-3 and 5-4, respectively are
given as examples.
o Case 1 (Exam le 5-2. The estimate of intralaboratory, short-
term precision i.e., within lot, single analyst) for each con-
centration found in the sample lot is s = 2.4 CU, for concen-
trations in the range of 20 to 40 CU, based on triplicate
analyses of each of three samples. The standard deviation s
can be used to estimate a probability interval for the random
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Paye 12 of 14
x
8
x
Xx
x x
x
x
e
lee
2ee
see
X
..se
see
8ee
s
2e
x
15
Ie
x
x
5 x
x
#
e lee 2ee see ..ee see 80e
x
Figure 5-4.
Plots of CV (%) versus X (CU) and s (CU) versus X (CU)
for Example 5-4.
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1984
paye 13 of 14
error associated with an individual observation Xi as fol-
lows. The approximate 95% probability interval for_the differ-
ence in an observation Xi and the limiting averaye X (i .e.,
the value that would be obtained if the sample were analyzed
many times) is ~ 2 s. The approximate 95% probability interval
for errors (excluding bias) for individual measurements in this
sample lot is estimated to be ~ 2 s = ~ 2(2.4 CU) = ~ 4.8 CU.
o Case 2 (Example 5-3). The estimate of intralaboratory, short-
term precision for individual measurements in this sample lot
includiny steps in the measurement system from sample collec-
tion through analysis is s = 0.07X CU for concentrations in the
approximate ranye of 1.5 to 5.0 CU, based on the analyses of 8
field replicate pairs. For example, at X = 5.0 CU, the esti-
mated standard deviation is s = U.35 CU, the approximate 95%
probability interval for errors (excluding bias), for X = 5.0
CU, is ~ 2 s or ~ 0.7 CU.
o Case 3 (Example 5-4). The estimate of intralaboratory, short-
term precision for individual measurements in this sample lot
includiny the total measurement system is s = 0.036X + 0.698,
for sample concentrations in the approximate ranye of 5 to 3~00
CU. based on the results from 10 pairs of collocated samples.
The approximate 9~% probability interval for the errors (ex-
cludiny bias) in an individual measurement X is + 2 s = +
2(0.U36X + 0.698) CU. For example, at X = 1UU CU, s = 4~3 CU,
the approximate 95% probability interval for errors (excluding
bias) for X = IOU CU is ~ 8.6 CU.
5.3.5
Continual Precision Assessments
For organizations in which sample lots are routinely analyzed and
data are reported on a frequent basis, the basic precision statistics
from multiple lots of a given sample matrix may be combined to provide
an estimate of long-term precision and an improved estimate of short-
term precision as illustrated in Section 5.6.3. This assessment can
also be extended to include subsequent lots, unless test results for
these new lots indicate that method precision is significantly differ-
ent. This combining of data quality assessment results permits the
laboratory to provide a precision assessment derived from a substantial
amount of background data rather than from limited precision data pro-
duced in a small study.
This procedure also provides the basis for the use of control
charts to monitor the performance of the measurement system. The
-------
Section No. 5.3
Revision No. 0
Date: March 30, 1984
Page 14 of 14
procedure is based upon the availability of a precision assessment
(normally developed from prior performance of the system), the use of
control chart limits, and routine replicate pairs.
Historical data must first be combined as necessary to develop an
assessment of precision that defines the expected standard deviation of
replicates as a function of concentration. For each replicate pair in
the new sample lot, the observed standard deviation s is compared with
an upper control limit (UCL) for the expected standard deviation, s*,
calculated for the observed average sample concentration X. If
s ~ 3.27 s*, the established precision assessment can be applied to the
individual members of the new sample lot. If s > 3.27 s*, either the
established precision assessment is not applicable to the new data set,
or the measurement system is out of control. (The upper control limit
factor, for an s chart is UCL = B4 s* and for replicate pairs B4 =
3.27. Note that B4 should not be confused with bias B as used in
this chapter.) For further information on the use of control charts,
including the rationale for the 3.27 constant, see Reference 4 in
Section 5.7.5.
At least annually, and preferably after the accumulation of
results from 30 to 5U new replicate pairs, new assessments for preci-
sion must be calculated to reflect the current precision of the
measurement system. This may be done by either expansion or replace-
ment of the historical data base with the most current data.
Example 5-6: A laboratory is analyzing and reporting samples on a
continual basis. The regression equation for R versus X estimated from
historical data is R = 0.051 X + 0.987 CU. for replicate pairs. A
replicate pair yields measurements of 18.6 and 20.5 CU. The expected
range for the pair is calculated for the average concentration of 19.55
CU to be R = 0.051(19.55) + 0.987 = 1.984 CU. The control limit for
the range of the pair is calculated to be 3.27(1.984) CU = 6.4~ CU.
Since the observed range of 1.9 CU (2U.5 - 18.6 = 1.~) is less than the
6.49 CU control chart limit, the result is within expectations and
there is no reason to suspect that the historical precision assessment
is not applicable. A graphical presentation of R versus ~ may
be convenient for use in a laboratory analyzing many samples.
-------
Section No. 5.4
Revision No. 0
Date: March 30, 1984
Page 1 of 10
5.4 ASSESSMENT OF BIAS
5.4.1 Definitions
Bias - an estimate of the bias B is
average value X of a set of measurements
ence value of the standard T given by:
the difference between the
of a standard and the refer-
B = X - T
Alternative estimates of bias are percent bias
%B = 100 (X - T)/T,
and average percent recovery P
n
P = L
i=1
P , and
i
Pi = 100 (Ai - Bi)/T,
where Ai = the analytical result from the spiked sample and Bi =
the analytical result from separate analysis of the unspiked sample.
The relationship between percent bias and percent recovery is:
%B = P - 100
Reference material - A material of known or established concen-
tration that is used to calibrate or to assess the bias of a measure-
ment system. Depending on requirements, reference materials may be
used as prepared or may be diluted with inert matrix and used as blind
environmental samples.
Spiking material - A material of known or established concentra-
tion added to environmental samples and analyzed to assess the bias of
environmental measurements.
Target analyte spiking - Spiking with the analyte that is of
basic interest in the environmental sample.
Matrix spike - A sample created by adding known amounts of the
target analyte to a portion of the sample.
Field matrix spike - A sample created by spiking target analytes
into a portion of a sample in the field at the point of sample acquisi-
tion. This data quality assessment sample provides information on the
target analyte stability after collection and during transport, and
-------
Section No. 5.4
Revision No. 0
Date: March 30, 1984
Page 2 of 1U
storage, as well as on losses, etc., during sample preparation and on
errors of analysis.
Laboratory matrix spike - A sample created by spiking target
analytes into a portion of a sample when it is received in the labora-
tory. It provides bias information regarding sample preparation and
analysis and is the most common type of matrix spike. This type of
matrix spike does not necessarily reflect the behavior of the field-
collected target analyte, especially if the target analyte is not sta-
ble during shipping.
Analysis matrix spike - A sample created by spiking target ana-
lytes into a prepared portion of a sample just prior to analysis. It
only provides information on matrix effects encountered during analy-
sis, i.e., suppression or enhancement of instrument signal levels. It
is most often encountered with elemental analyses involving the various
forms of atomic spectroscopy and is often referred to as "standard
additions".
Non-target analyte spiking - Spiking of surrogate analytes into
the sample. A surrogate analyte is one which mimics the behavior
of target analytes in terms of stability, preparation losses, measure-
ment artifacts, etc., but does not interfere with target analyte
measurement. This approach is most frequently used with organic
compound determinations and is a compromise which is dependent on
the target compounds and surrogates involved. Surrogates, like
matrix spikes, can be added in the laboratory or in the field; results
are interpreted in a fashion similar to matrix spikes.
Internal standard spike - An analyte which has the same charac-
teristics as the surrogate, but is added to a sample just prior to
analysis. It provides a short term indication of instrument perfor-
mance, but it may also be an integral part of the analytical method in
a non-quality control sense, i.e., to normalize data for quantitation
purposes.
Figure 5-5 provides a graphic representation of target analyte
spiked samples in the preferred order starting at the top, based on
point of spiking in the sampling and analysis scheme.
-------
Sample
Acqu is it ion
Field
Mat ri x
Spike
Sample Spiking Point
Preparation
Analysis
Fi gure 5-5.
Lab
Matri x
Spike
Section No. 5.4
Revision No. 0
Date: March 30, 1984
Page 3 of lU
Data Interpretation
Provides a best case estimate
of bias based on recovery;
includes matrix effects asso-
ciated with sample preserva-
tion, shipping, preparation
and analysis.
Provides an estimate of bias
based on recovery. I ncor-
porates matrix effects asso-
ciated with sample preparation
and analysis only.
Analysis -- Provides an indication of
Matrix matrix effects associated with
Spike the analysis process only.
The Use of Target Analyte Spikes for Bias Estimation
-------
Section No. 5.4
Revision No. 0
Date: March 30. 1984
Page 4 of 10
5.4.2
Measurement of Bias
Bias assessments for environmental measurements are made using
spiking materials or reference materials, personnel, and equipment as
independent as possible from materials used in the calibration of the
measurement system. Where possible, bias assessments should be based
on analysis of spiked samples rather than reference materials so that
the effect of the matrix on recovery is incorporated into the assess-
ment. A documented spiking protocol and consistency in following that
protocol is an important element in obtaining meaningful data quality
estimates. Spikes should be added at different concentration levels to
cover the range of expected sample concentrations. For some measure-
ment systems (e.g., continuous analyzers used to measure pollutants in
ambient air), the spiking of samples is not practical, and assessments
are made using appropriate blind reference materials.
Ideally spiking materials or reference materials should be intro-
duced into samples at the collection site so that the bias assessment
includes any losses caused by sample handling, preservations, and
storage. If the matrix type or the measurement system prevents such
practices, bias assessments should be made for as large a portion of
the measurement system as possible.
A representative portion of the sample lot is selected for spiking
and the selected samples are analyzed before and after spiking in order
to measure recovery. The spiking frequency will depend upon the data
quality needs of the program, the bias and precision of the measurement
system, the size of the sample lot and other considerations. To pro-
perly assess the bias for a small sample lot, it may be necessary to
spike a relatively high percentage of the samples. However, where the
method performance for multiple lots of samples of similar matrix type
is expected to be equivalent, it may be possible to combine information
so that fewer spikes are required in each lot.
For certain multianalyte methods, such as EPA Method 608 for
organochlorine pesticides and PCBs in water, bias assessments are com-
plicated by mutual interference between certain analytes that prevent
all of the analytes being spiked into a sinyle sample. For such meth-
ods, lower spiking frequencies can be employed for analytes that are
seldom, or never, found. The use of spiked surrogate compounds for
multianalyte GC/MS procedures, while not ideal, may be the best
-------
Section No. 5.4
Revision No. 0
Date: March 30, 1984
Page 5 of 10
available procedure for assessment of bias. An added attraction is the
ability to obtain recovery data on every field sample at relatively low
costs. It is used, for example, to evaluate the applicability of
methodology and, indirectly, data quality assessments to individual
members of a sample lot. Such practices do not preclude the need to
assess bias by spiking with the analytes being measured or reported.
5.4.3
Calculation of Bias Statistics
The most widely used summary of bias is by linear regression of
bias on T; or, equivalently, regression of data quality assessment
results (Xi or X) on T (as illustrated in Section 5.6.3). For
the important special case of spiked samples as described above, the
following approach may also be useful.
A portion of the samples in the sample lot is spiked at multiple
concentration levels to determine individual measurements of percent
recovery. These recoveries are used to calculate summary bias statis-
tics for the entire sample lot. The summary statistics are used to
estimate the percent recovery for each individual measurement in the
lot. For each sample spike i, calculate the percent recovery Pi by,
p. = 100 (A' - B')/T'
1 1 1 l'
where:
Ai = the
Bi = the
the
Ti = the
analytical result
analytical result
unspiked sample,
known true value of the spike.
from the spiked sample,
from a separate analysis of
Average percent bias is calculated from average percent recovery
for the sample spikes by the relationship
% B = P-100.
If reference materials instead of spiked samples are analyzed to
assess bias, percent recovery is calculated by the equation above with
Bi equal to zero.
Upon completion of the project or time period, the bias assessment
for the data set of environmental measurements is calculated from the
individual percent recoveries Pi observed through the project period.
Unless a relationship between the percent recovery, or its variability,
and concentration can be established, all percent recovery measurements
-------
Section No. 5.4
Revision No. 0
Date: March 30, 1984
Page 6 of 10
may be combined for the bias assessment. Calculate the averaye percent
recovery P and the standard deviation of the percent recovery
(sp)' The meaning of the value for sp may be considerably different
from the precision assessment. For spiked samples it includes the
variability of the unspiked measurement plus the variability of the
spiked final measurement. In addition, the individual recoveries are
usually gathered over an extended time period, rather than over short
time intervals normally used for replicate measurements and therefore
may reflect the presence of many other variables.
Example 5-7: For a 100-sample lot, 10 sample portions were spiked
and analyzed along with unspiked portions. In this case the volume of
spike to the sample volume was so small that no volume correction was
required. The results of the analyses are tabulated below:
Sample Background Spike Result Recovery Percent
8i Ti Ai Ai - Bi Recovery, %
4.U 20.U 22.8 18.~ 94.0
7.9 20.U 26.2 18.3 91.5
4.5 20.0 25.4 20.9 104.5
1.3 20.0 21.2 19.9 99.5
17.3 ~O.O 66.7 49.4 94.8
26.3 10U.0 128.0 101. 7 101. 7
5.7 20.0 24.8 19.1 95.5
5.0 20.0 24.8 19.8 99.0
62.5 200.0 260.5 197.8 98.9
34.5 100.0 135.3 100.8 100.8
10
Average (P) =1: Pi/n = 980.2/10 = 98.0
i=l
6
Standard Deviation (sp) = [L: (Pi-p)2/n-1J1/2 = 4.U
i =1
The average percent recovery, P, and the standard deviation of
the percent recovery, sp, are P = 98.0%, sp = 4.0%.
The approximate 95% probability interval for percent recovery for
individual measurements in the sample lot is 98.0% ~ 2(4.0)%, i.e.,
-------
Section No. 5.4
Revision No. 0
Date: March 30, 1984
Page 7 of 10
from 90% to 106%. The percent recovery associated
measurement in this 100-sample lot is estimated to
106%, at the 95% probability level.
Example 5-8: In any application of statistics it is important to
identify underlying assumptions and check them with the data by graphic
display or statistical checks. Figure 5-6 is a plot of recovery versus
concentration for analyses of sulfate ion deposited on Teflon@ filters.
Evidently recovery is not constant, but is a slightly decreasing func-
tion of concentration. Therefore, a summary in terms of a single aver-
age percent recovery would be of questionable validity. This data is
examined in more detail using a linear regression approach in Section
5.6.3.
with any individual
be between 90% and
5.4.4
Reporting Bias
Each environmental measurement must be reported with an assessment
of bias. Bias should be expressed as a percent error interval of
- -
%B + 2 sp or as a percent recovery interval from P - 2 sp to P +
2 sp. (There are several ways to express bias and accuracy. It
should be noted here that expressing bias in this manner is not consis-
tent with the definition given in Subsection 5.4.1. Additional efforts
will be made to achieve consistency in the definition and use of bias
in future revisions.) Where reference materials are used as a matrix-
free check on laboratory performance as a supplement to sample spiking,
only the results of the sample spikes should be submitted to an envi-
ronmental data base.
The data user should be provided with a narrative statement
explaining the reported bias estimate along with tabulated percent
recovery intervals. The statement might read:
"Bias is expressed as a 95% probabil ity interval around the aver-
age percent recovery. A percent recovery interval of 90 to 106%,
for example, means that approximately 95% of the time when a spike
of the measured material is recovered, the observed percent re-
covery can be expected to lie between 90 and 106%."
5.4.5
Continual Bias Assessment
A~ with precision assessments, laboratories in which small sample
lots are routinely analyzed and data are reported on a frequent basis
-------
PERCENT
120
1 t 0 X
X
X X X
X
100 X X X
X X X X xM
X X X X
XX X ~ X
90 X X
X
X
80 X X
70
o
50
100
150
T
200
250
300
Figure 5-6.
Plot of Percent Recovery (Pi) versus Concentration (Ti) for Sulfate
Data Quality Assessment Results.
-a 0;0 V)
~~CDCD
\C.c-t"
-------
Section No. 5.4
Revision No. 0
Date: March 3U, 1984
Page 9 of 10
may combine the basic bias statistics of multiple small sample lots of
a given matrix into a single bias assessment for the combined sample
set. This assessment can also be extended to include subsequent small
sample lots, unless test results for these new lots indicate that
method bias is significantly different.
Combining bias data in this manner permits the laboratory to pro-
vide bias assessments derived from a substantial amount of background
data rather than from limited bias data produced in a small study. It
also can provide the basis for the use of control charts to monitor
measurement system bias over time.
Historical data must first be combined as necessary to develop an
assessment of bias which includes the determination of averaye percent
recovery (P) and the standard deviation of the percent recovery
(sp)' These estimates maybe used to develop control chart limits as
P ~ 3 sp for subsequent measurements. Each recovery measurement,
Pi, in a new sample lot must be compared with the control chart
limits. If each value for Pi falls within the control limits, the
historical percent recovery and analysis assessment can be applied to
all individual measurements of the new sample lot. If Pi falls
outside the control limits, either the historical assessment is not
applicable to the new data set or the laboratory operation is out of
statistical control.
At least annually, and preferably after no more than 30 to 50 new
recovery measurements have been taken, the control chart limits must be
recalculated to reflect the current percent recovery capabilities of
the measurement system. This may be done by either expansion or
replacement of the historical data base to include the most current
data.
Example 5-9: A laboratory is analyzing and reporting samples on a
continual basis. Historical data for the analysis of spiked samples
established that P = 98.0%, and sp = 4.0%; the control chart limits
are P + 3 s or 86 and 110%. A sample with a measured background
- p,
level (Bi) of 22.0 CU was spiked with the equivalent of 30.0 CU (T)
without appreciably changing the sample volume. The result for the
analysis of the spiked sample (Ai) was 49.2 CU. Therefore:
p.
1
=
100 49.2 - 22.0
30.0
= 90.7%.
-------
Section No. 5.4
Revision No. 0
Date: March 30, 1984
Page 10 of 10
Because Pi falls within the control chart limits of 86 and 110%,
the sample is consistent with the historical recovery and analysis
assessment. A graphical presentation of P ! 3 sp versus test number
may be convenient for use in a high volume laboratory.
-------
Sect ion No.5. 5
Revision No. 0
Date: March 30, 1984
Page 1 of 2
5.5
METHOD DETECTION LIMIT
5.5.1
Discussion
There have been many terms used to designate detection limits and
they have been defined in various ways. Lower Limits of Detection
(LLD), Minimum Detection Amount (MDA), Method Detection Limit (MOL),
Detection Sensitivity, and Limit of Detection (LOD) are some of the
terms used. Most authorities in the field agree that the smallest
detectable quantity, by whatever name, is related to the standard
deviation of sample analyses at or near zero analyte concentrations.
Since MOL is a basic performance characteristic of an analytical method
only its calculation, with example, is discussed.
5.5.2
MOL
Ideally each laboratory should establish and periodically reevalu-
ate its own MOL for each sample matrix type (for one time only matrix
types and multianalyte samples in difficult matrices. e.g., soils or
fish, this may be impractical) and for each environmental measurement
method. The MOL is determined for measurement systems by the analyses
of seven or more replicates of spiked matrix samples. As with preci-
sion and bias, the assessment of MOL should be based upon the perfor-
mance of the entire measurement system. The standard deviation of the
responses (sm), in concentration units, is used to calculate the MOL
as follows:
MOL = sm (t.99)
Eq. 5-7
where:
t.99 = "Student1s t value" appropriate for a one-tailed test at
the 99% confidence level and a standard deviation estimate
with n-1 degrees of freedom.
For example, if the MOL is determined using measurements from seven
appropriate samples, then use t.99 = 3.14 for n-1 = 6 degrees of
freedom. If the determination yielded a standard deviation of 0.15 CU,
the MOL is calculated (Equation 5-7) to be (3.14)(U.15 CU) = 0.47 CU.
-------
Section No. 5.5
Revision No. 0
Date: March 30, 1984
paye 2 of 2
The mechanics of calculating MOL are illustrated in the following
example.
Example 5-10. Eight samples, identical in appearance to routine
samples, spiked at a level of 0.U2 CU were blindly inserted throuyhout
a day of routine analyses to assess the system accuracy at low concen-
trations on that day.
The observed concentrations were 0.032, 0.016. 0.021, 0.U22,
U.024, 0.017, 0.U25. 0.019 CU. The standard deviation of these results
is sm = .OUS CU. Assuming on the basis of previous experience that
the true standard deviation is essentially constant for concentrations
this low, a measurement must exceed (tsm) in order to be siynificant-
ly greater than zero at the 0.01 level of significance, usiny a one-
tailed test. Here t denotes the upper percentile of Student's t-dis-
tribution with n-l = 7 degrees of freedom, so the MOL estimate using
equation 5-7 is (2.998) (.005 CU) = .01S CU.
-------
Section No. 5.6
Revision No. 0
Date: March 30, 19H4
Page 1 of 14
5.6
A CASE STUDY
To illustrate the application of the statistical concepts previ-
ous ly di scussed, data from an actual 1 aboratory measurement program are
presented and calculations of appropriate summary statistics are per-
formed. The example includes calculation of within-lot estimates of
precision and bias as well as between-lot summary statistics to illus-
trate a procedure for continual data quality assessment and/or estimat-
ing long-term precision of projects of long duration. The within-lot
sample sizes are 3 or smaller in all but one case. Within-lot averages
and standard deviations computed from such small samples tend to be
imprecise estimates of the population mean and standard deviation. The
example shows how improved estimates of these quantities can be ob-
tained by averaging over the lots.
The average within-lot standard deviation provides a better esti-
mate of short-term, intralaboratory, precision for the total data set.
Likewise, using data quality assessments between- or across-lots can
provide an estimate of between-lot variability, i.e., systematic error
from lot-to-lot as well as an estimate of total variability or long-
term precision over the subject time period.
The estimate of long-term precision is the appropriate measure to
use in describing the precision of the total data set. Estimates of
short-term (within-lot) precision and between-lot variability are use-
ful as part of a data quality assessment proyram in monitoring the
system's performance and can provide guidance for troubleshootiny by
indicating which component(s) of the measurement system is experiencing
larger than normal variability.
The example goes somewhat beyond the treatment presented in Sec-
tion 5.4, by employing linear regression to summarize relations between
bias and concentration. In this example, a lot is defined as all
measurements made within a given day, including routine analysis of
study samples and data quality assessment samples at one or more con-
centrati ons.
The example data are from a program for ion chromatography analy-
sis of sulfate deposits on Teflon@ filters. Data quality assessment
samples in this program consisted of sulfate solutions at three differ-
ent concentrations, known to the analyst. The ion chromatograph was
-------
Section No. 5.6
Revision No. 0
Date: March 30, 1984
Page 2 of 14
calibrated at the beginning of each day of operation and data quality
assessment samples were interspersed throughout the routine analyses as
a check that the system was in control, and to allow assessment of data
quality.
An additional data quality assessment was obtained from the regu-
lar analysis of samples provided by an external auditor. These samples
more closely approximated the actual study samples than did the inter-
nal data quality samples, and consisted of known (only to the auditor)
concentrations of sulfate deposited on Teflon@ membrane filters. Thus,
data quality estimates derived from these samples included components
of variance due to sample preparation (i .e., extraction) as well as
sample analysis.
The two types of
ters, respectively.
samples will be referred to as solutions and fil-
5.6.1
Within-lot Summaries
Let Xl, . . . ,Xn be data quality assessments associated with
a sinyle lot and made at equal or nearly equal concentration levels.
The within-lot summary statistics for n data quality assessments at a
fixed reference concentration T in the lot are:
-
X
= the average of the data quality assessments made at a fixed
concentration in the lot
Sw = the within-lot standard deviation of the n data quality
assessments made at a fixed concentration
Obviously, when only a single measurement is obtained at a certain
concentration for a given lot, the standard deviation cannot be calcu-
lated.
These within-lot summaries of data quality assessments are primar-
ily for internal use. They are useful intermediates in the calculation
of summary statistics based on longer time periods, such as six months
or a year (Section 5.6.2). Within-lot averages and standard deviations
for data quality assessments are also useful as variables to be moni-
tored with control charts. If data quality assessments for a lot are
taken at several known concentrations, regression statistics may also
provide a useful summary of the data quality of the lot (see Section
5.6.3). To fully utilize these statistics the assistance of a quali-
fied statistician may be required.
-------
Section No. 5.6
Revision No. 0
Date: March 30. 1984
Paye 3 of 14
Data quality assessments for the example sulfate analysis proyram
and associated within-lot statistics (i .e., n, X, Sw and T) are pre-
sented in Table 5-2 for three values of solution concentration, 0, 12U.
and 240 CU. As seen in the table, the within-lot sample size (i .e.,
the number of data quality assessment samples) ranyes from n=1 to n=6.
The within-lot average and standard deviation can vary widely with such
small sample sizes.
5.6.2
Between-Lot Summaries
This procedure allows for the calculation of summary statistics
for a project time period that spans several lots. The followiny sta-
tistics are used to report data quality over a yiven time period (e.y.,
quarter, year). They are obtained from within-lot statistics described
in the previous section, and are calculated for all data quality
assessments obtained under a particular set of fixed conditions, such
as matrix, concentration, and analysis procedure. For the example sul-
fate anaysis program, between-lot statistical summaries are calculated
for each (fixed) data quality assessment sample solution concentration.
Let
n.
J
= the number of data quality assessments
at a given concentration in the jth lot.
-
x.
J
= the average of the data quality
assessments at a yiven concentration for
the jth lot.
s.
J
= the standard deviation of the data
quality assessments at a yiven
concentration for the jth lot.
k
n = L nj
J=l
= total number of data quality assess-
ments for the reporting period, where k =
total number of lots.
Then,
The grand
weighted average is:
k
nj
L n
j=l
x =
X
J
Eq. 5-8
-------
Section No. 5.6
Revision No. 0
Date: March 30, 1984
Page 4 of 14
TABLE 5-2.
MEASUREMENTS AND WITHIN-LOT SUMMARY STATISTICS FOR
SULFATE DATA QUALITY ASSESSMENT SOLUTIONS
T = 0'.0' CUa T = 120' CUa T = 240' CUa
DATE aF MEASURE- - MEASURE- - MEASURE- -
aPERATI aN MENT n X Sw MENT n X Sw MENT n X Sw
7/16/80' 0'.0'0' 2 0'.88 1.24 119.8 1 119.8 - 241.0' 3 240'.4 1.4~
1.75 241.5
238.8
7/17/80' 0'.50' 2 0'.75 0'.35 239.8 6 239.2 0'.71
LaO' 239.5
238.0'
239.8
238.8
239.5
7/18/80' 2.25 1 2.25 - 118.8 1 118.8 - 241.0' 3 240'.4 1.44
241. 5
238.8
7/31/80' 0'.0'0' 1 0'.0'0' - 241. 2 3 240'.5 3. 7C
243.8
236.5
8/18/80' 124.3 3 120'. 9 2.98
119.0'
119.3
10'/6/80' 0'.0'0' 1 0'.0'0' - 240'.2 3 239.6 a. 7~
238.8
239.8
10'/27/80' 240'.5 3 235.8 5.0'
236.5
230'.5
10'/28/80' 123.5 3 120'.6 2.8/i
117.8
120'.5
10/29/80' 119.8 2 118.6 1.61
117.5
11/12/80' 127.2 3 121. 7 6.74
114.2
123.8
11/13/80' 130'.2 2 121.8 11.81
113.5
I I
a CU - concentration unit
-------
Section No. 5.6
Revision No. 0
Date: March 30, 1984
Page S of 14
The average within-lot standard deviation is:
s- =
w
k
L
j=1
(nj-1)s}
n-k
Eq. 5-9
The between-lot (or among lots) standard deviation is:
k
L
nj(Xj-X)2 2
k-1 - Sw
sa =
j=1
(n1 + ...+ nk)/k
Eq. 5-10
Total variation of single observations at the related concentra-
tion level over time is:
( 2 2 )1/2
St = sa + Sw
Eq. S-ll
Between-lot summary statistics for the sulfate sample data are
given in Table 5-3. Example calculations of between-lot summary sta-
tistics for Condition 1, where T = 240 CU, k = 6, and n = 21 are given
below:
-
X =
6 nj -
2: 2T (Xj)
j=1
3 6 3 )
= If (240.4) + 2T (239.2) + 2I (240.4 +
~1 (240.S) + ~1 (239.6) + ~1 (23S.8)
X = 239.3 CU
s- =
w
6 (nj-1) Sj2
L 21 - 6
j=1
= [2(1.44)2+5(O.71)2+2(1.44)~;2(3.70)2+2(O.72)2+2(5.03)2] 1/2
-------
TABLE 5-3.
SUMMARY STATISTICS FOR SULFATE DATA QUALITY ASSESSMENTS
SOLUTIONS AT FIXED CONDITIONS, CONCENTRATION KNUWN
Number of Lots I Average Within-Lot I
k Wi th Data Total Number n of Grand Average X Over Between-Lot Total
Condition True Value Qua 1 ity Samples at the k Lots of Measurements !!ias Standard Deviation Std. Dev. V ari abi I it}
T Assessments Condition Over All Lots at the Condi t i on X - T sl'l sa St
1 240 6 21 239.3 -0.7 2.45 1.03 2.66
2 120 7 15 120.6 .6 ~.78 0.00 5.78
3 0.0 !:i 7 0.79 .79 0.91 0.12 0.92
4
~
6
7
6
9
10
Lot Measurements at Fixed Condition Within-Lot Summaries
Number -. (Sinyle Concentration) ~(Single Concentration)
Time Period Summaries
(Sinyle Concentration)
1 Xu . . . X1n1 n1, Xl, sl
2 X21 . . . . . . . . X2n2 n2. X2' s2
k Xk1 . . . . . . Xknk nk> X k' sk
n " n1 + . . . + nk
yrand weiyhted averaye (see Eq. 5-6)
average within-lot standard deviation
(see Eq. 5-9)
between-lot standard deviation
(see Eq. 5-10)
total variability standard deviation
(see Eq. !:i-ll)
""0 0 ;;0 V1
QIQI CD CD
tCM-
-------
Section No. 5.6
Revision No. 0
Date: March 30, 1984
Page 7 of 14
Sw = 2.45 CU
6 nj(Xj-x)2 s~
L
6-1 w
sa = j=1
(n1 + ... + n6)/6
3(240.4-239.3)2 +
= [3(24000-23903)2 +
sa = 1.03 CU
St = J 5 ~ + S~ = 2066 CU
6(239.2-239.3)2 +
3(239.6-239.3)2 +
5
(3 + 6 + 3 + 3 + 3 + 3)/6
3(240.4-239.3)2+
3(235.8-239.3)2 -
2.452 J/2
The summary statistics should be reported for each set of sample
conditions over the appropriate time period using the format of Table
5-3. (The value sa = 0 at true value 12U resulted from a negative
value for s~ indicating that there was not significant
variation from lot to lot.)
5.6.3
Regression Summaries and Reporting Requirements for a Project
or Time Period
Any measure of bias (e.g., recovery) or precision can depend on
the concentration T. Even when T is unknown, dependence of precision
on T may be seen as dependence on the apparent (measured) concentration
X. Regression summaries of bias and precision over all conditions pro-
vide a useful complement to summary statistics for individual sets of
conditions. Further, in a case where all the reference values, T, of
data quality assessment samples for a given reporting period are dif-
ferent, then between-lot statistical summaries for specific conditions
in the Table 5-3 format would be equivalent to lists of the raw assess-
ment data. A much more useful summary involves a regression and plot
of measurement X on reference value T.
-------
Section No. 5.6
Revision No. °
Date: March 30, 19B4
Page B of 14
Table 5-4 contains summary regressions for the sulfate.analysis
data of Table 5-2. Associated data plots are shown in Figures 5-7
through 5-9.
Data from analysis of the external filter samples are tabulated in
Table 5-5. The summary regression data and plot of X versus T
are shown in Table ~-6 and Figure 5-10, respectively.
Regression summary statistics should be reported over the appro-
priate time period using the format of Tables 5-4 and 5-0.
The appropriate rows of each applicable table (i .e. summary sta-
tistics and regression summaries) should be completed for each measure-
ment system. If more than ten known concentrations are employed then
the ten most frequently used concentrations should be reported in the
format of Table 5-3. The summary statistics table can also be used to
report on individual lots.
If data quality assessment results are available for only one or
two concentrations then the regression summaries are inappropriate and
the summary statistics should be limited to individual conditions as in
Table 5-3. For three or more concentrations the applicable regression
summaries should be reported as well. For bias, either the within-lot
averages or grand average for the reporting period can be used as the
dependent variable. A similar option is left open for precision in the
second row of the summary regression report form (Tables 5-4 and 5-6).
The option selected should be indicated by underlining.
In those instances where there is no control over sample concen-
tration T, and thus, calculation of between-lot summary statistics is
not possible, a plot and regression of within-lot standard deviation
Sj on within-lot average Xj is recommended (bottom row in Tables
5-4 and 5-6). In such a case only the last row of the summary regres-
sion form is completed.
The second and fourth rows of the summary regression form (Tables
5-4 and 5-6) involve regression of within-lot (average) standard devia-
tion on either the reference value T or the replicate average Xj. If
T is known and only one of rows 2 and 4 is to be completed, then row 2
should be completed.
-------
TABLE 5-4.
SUMMARY REGRESSIONS FOR SULFATE ANALYSIS OF DATA FROM PREPARED SOLUTIONS,
REPORTING PERIOD JULY, 1980 THROUGH DECEMBER, 1980
Independent Number
Type of Data Sample Dependent Variable Variable of (u, v)
Quality Indicator Type va u pai rs slope intercept
Bias sul fate lot average, Xj or grand true value T 3 .99 0.98
ion in average ~
solutior
Precision I within-lot std. dev. Sj true value T 3 .006 2.28
or average within-lot std.
dev. s;
Precision I between-lot std. dev. sa true value T 3 .004 -0.08
Precisionb within-lot std. dev. Sj X.
(primarily for spl it J
samp 1 es or repeat ana lys is
of routine samples)
a Underlined variable is illustrated in this example.
b This row not applicable for this data set.
-0 0;;:0 V>
IlJllJroro
(Crt
-------
Section No. 5.6
Revision No. 0
Date: March 30, 1984
Page 10 of 14
*
.
8
-------
SW
e
5
4
3
2
o
o
Fi gu re 5-8.
Sa
1.5
I.
0.5
0.0
o
Figure 5-9.
Section No. 5.6
Revision No. 0
Date: March 30, 1984
Page 11 of 14
.
.
e0
120
T
180
240
Plot of Within-Lot Standard Deviation Sw (CU) versus T
(CU) for Sulfate Data Quality Assessment Results (Table 5-3).
*
.
e0
120
T
180
240
Plot of Between-Lot Standard Deviation sa (CU) versus T
(CU) for Sulfate Data Quality Assessment Results (Table 5-3).
-------
Section No. 5.6
Revision No. 0
Date: March 30, 1984
Page 12 of 14
TABLE 5-5. MEASUREMENT DATA FOR SULFATE FILTER SAMPLES
X T pa X T pa X T pa
152 190 80 225 222 101 98 102 96
265 272 97 79 83 95 116 114 102
50 54 93 157 169 93 199 201 99
132 141 94 59 63 94 159 169 94
35 34 103 127 127 IOU 267 292 91
223 265 84 147 154 95 158 154 103
34 34 IOU 227 231 98 29 27 107
177 222 80 82 83 99 112 114 98
110 102 108 192 222 86 228 250 91
178 190 94 54 54 100 194 201 97
257 265 97 266 272 98
. . . .
a P = Percent Recovery = 100 X/T (See Section 5.4 for definition)
-------
TABLE 5-6.
SUMMARY REGRESSIONS FOR SULFATE ANALYSIS OF FILTER DATA,
REPORTING PERIOD AUGUST, 198U TO DECEMBER, 1980
Independent Number
Type of Data Sample Dependent Variable Variable of (u. v)
Quality Indicator Type va u pairs slope intercept
-
X sulfate I ot aver~ge. X j or grand true value T 32 0.91 5.80
i on on average X
teflon
Bias filter -0.09 5.80
Precisionb within-lot std. dev. Sj true value T
or average within-lot std.
dev. Sw
Precisionb between-lot std. dev. sB true value T
Preci sionb within-lot std. dev. Sj X
(primarily for sp I it J
samples or repeat analysis
of rout i ne samples)
a Underlined variable is illustrated in this example.
b This row not applicable for this data set.
\J 0 :;:0 VI
QlQICDCD
to c-t < ()
CD CD -'. c-t
.. V) --I.
...... "".0
w 0 ~
3:~
OQl Z
-.,-,ZO
() 0 .
...... ~.
~ U1
WOo
o ~
......
\D
ex>
~
-------
Section No. 5.6
Revision No. 0
Date: March 30, 1~84
Page 14 of 14
x
*
*
*
* * * *
.
*
*
*
* .
*
*
* *
* .
*
.
.*
**
8
58
188
158
T
288
258
388
F i gu re 5-10.
Plot of X (CU) versus T (CU) for Sulfate Data
Quality Assessment Results Reported from August,
1980, to December, 1980.
-------
5.7
Section No. 5.7
Revision No. 0
Date: March 30, 1984
Page 1 of 6
SOURCES OF ADDITIONAL INFORMATION
5.7.1
5.7.2
Study Planning
1. IICompilation of Data Quality Information for Environmen-
tal Measurement Systems,1I QAMS , U.S. EPA, Office of
Research and Development, Washington, DC, 1983 (Draft).
2.
IIInterim Guidelines and Specifications for Preparing
Qua~ity Assurance Project Plans, IIQAMS-005/80. U.S. EPA,
Offlce of Research and Development, Washington, D.C.
20560, December, 1980.
3.
Natrella, M.G., Experimental Statistics, NBS Handbook 91,
U.S. Department of Commerce, National Bureau of Stan-
dards, 1966.
4.
Davies, O. L., The Design and Analysis of Industrial
Experiments, 2nd edition, Hafner Publishing Co., New
York, 1956.
5.
Cox, D.R., Planning of Experiments, Wiley, New York,
1958 .
6.
Box, G.E.P., W.G. Hunter and J.S. Hunter, Statistics for
Experimenters, Wiley, New York, 1978.
Youden, W.J., IIStatistical Aspects of Analytical Deter-
minations, IIJournal of Quality Technology, 4(1),1972,
pp. 45-49.
7.
8.
Elder, R.S., IIChoosing Cost-Effective QA/QC Programs for
Chemical Analysis,1I EPA Contract No. 68-03-2995, Radian
Corporation, Austin, Texas, 1981 (draft).
Sampling
1.
Environmental Monitoring and Support Laboratory, Handbook
for Sampling and Sample Preservation of Water and Waste-
water, EPA-600/4/82-029, U.S. EPA, Office of Research and
Development, Cincinnati, 1982.
Brumbaugh, M.A., IIPrinciples of Sampling in the Chemical
Field,1I Industrial Quality Control, January 1954, pp.
6-14.
2.
3. Kratochvil, B. and J.K. Taylor, IISampling for Chemical
Analysis,1I Analytical Chemistry, 53(8),1981, pp. 928A-
938A.
4.
Currie, L.A. and J.R. DeVoe, IISystematic Error in Chemi-
cal Analysisll, In: Validation , of the M~asureme~t Process,
ACS symposium Series 63, Amerlcan Chemlcal Soclety,
Washington, D.C., 1977, pp. 114-139.
-------
5.7.3
Section No. 5.7
Revision No. 0
Date: March 30, 1984
Page 2 of 6
Assessment of Precision
1.
Bennett, C.A. and N.L. Franklin, Statistical Analysis in
Chemistry and the Chemical Industry, Wiley, New York,
1954.
2.
Rhodes, R.C., "Components of Variation in Chemical Analy-
sis." In: Validation of the Measurement Process ACS
Symposium Series No. 63, American Chemical Society
Washington, D.C. 1977, pp. 176-19~. '
3.
Wilson, A.L., "The Performance Characteristics of Analy-
tical Methods-II," Talanta, 17, 1970, pp. 31-44.
Bicking,- C.A., "Precision in the Routine Performance of
Standard Tests," ASTM Standardi zat i on News, January 1979,
pp. 12-14.
4.
5.
Merten, D., L.A. Currie, J. Mandel, O. Suschny and G.
Wernimont, "Intercomparison, ~ual ity Control and Statis-
tics." In: Standard Reference Materials and Meaningful
Measurements, NBS Special Publication 408, u.S. Depart-
ment of Commerce, National Bureau of Standards, 1975, p.
805.
6.
Janardan, K.G. and D.J. Schaeffer, "Propagation of Random
Error in Estimating the Levels of Trace Organics in Envi-
ronmental Sources, "Analytical Chemistry, 51(7), 1979,
pp. 1024-1026.
7 .
Bicking, C.A., "Inter-Laboratory Round Robins for Deter-
mination of Routine Precision of Methods." In: Testing
Laboratory Performance, NBS Special Publication 591, U.S.
Department of Commerce, National Bureau of Standards,
1980, pp. 31-34.
8.
Wernimont, G., "Use of Control Charts in the Analytical
Laboratory, "Industri al and Engi neeri ng Chemi stry ,
18(10), 1946, pp. 587-592.
9.
Frazier, R.P., et al., "Establishing a Quality Control
Program for a State Envi ronmenta 1 Laboratory," Water and
Sewage Works, 121(5),1974, pp. 54-57.
Dorsey, N.E. and C. Eisenhart, "O~ Abs?lute Measure~ent."
In: Precision Measurement and Callbratlon, NBS Speclal
Publication 300, U.S. Department of Commerce, National
Bureau of Standards, 1969, pp. 49-55.
10.
11.
Suschny, O. and D.M. Richman, "Th~ Analytic~l Quality
Control Programme of the Internatlonal AtomlC Energy
Agency." In: Standard Reference Materi a 1 sand Meani ngful
Measurements, NBS Special Publication 408, U.S. Depart-
ment of Commerce, National Bureau of Standards, 1975, pp.
75-102.
-------
12.
5.7.4
Section No. 5.7
Revision No. 0
Date: March 30. 1984
Page 3 of 6
Taylor, J.R., An Introduction to Error Analysis -- The
S~udy o~ Uncertainties in Physical Measurements, Univer-
~lty SC1ences Books, Mills Valley, California, 1982 (This
1S a general text).
Assessment of Bias
1.
Uriano, G.A. and C.C. Gravatt, "The Role of Reference
Materials and Reference Methods in Chemical Analysis,"
CRC Critical Reviews in Analytical Chemistry, 6(4), 1977,
pp. 361-411.
2.
Uriano, G.A. and J.P. Cali, "Role of Reference Materials
and Reference Methods in the Measurement Process." In:
Validation of the Measurement Process, ACS Symposium
Series No. 63, American Chemical Society, Washington,
D.C., 1977, pp. 140-161.
3.
Skogerboe, R.K. and S.R. Koirtyohann, "Accuracy Assur~nce
~n the Analysis of Environmental Samples." In: Accuracy
1n Trace Analysis, Vol. 1, NBS Special Publication 422,
U.S. Department of Commerce, National Bureau of Stan-
dards, 1976, 1976, pp. 199-210.
4.
Watts, R.R., "Proficiency Testiny and Other Aspects of a
Comprehensive Quality Assurance Program." In: Optimizing
Chemical Laboratory Performance throuyh the Application
of QAOf u~lit1 ~~sura~ce ~r~~ci~les'VAAsS01c910ation 0~70f1f1~cial
~y ica emis s, r 1ny on, , 0, pp. u - ~.
Horwitz, W.L., R. Kamps and K.W. Boyer, "Quality Assur-
ance in the Analysis of Foods for Trace Constituents,"
Journal of the Association of Official Anal tical
Chemists, 63 6 , 1980, pp. 1344-1354.
5.
6.
Colby, B.N., "Development of Acceptance Criteria for the
Determination of Organic Pollutants at Medium Concentra-
tions in Soil, Sediments, and Water Samples," EPA Con-
tract No. 68-02-3656, Systems Science and Software,
LaJolla, CA, 1981.
7.
Bi ck i ng, C., S. 01 in and P. K i ny, P rocedu res for the
Evaluation of Environmental Monitoring Laboratories,
Tracor Jitco, Inc., EPA-600/4-78-017, U.S. EPA, Office of
Research and Development, Environmental Monitoring and
Support Laboratory, Ci nci nnati , 1978.
U.S. Department of the Army, "Quality Assurance Program
for U.S. Army Toxic and Hazardous Materials Agency,"
Aberdeen Proving Ground, MD., August 1980 (draft).
8.
9.
Freeberg, F.E., "Meani ngful Qual ity Assurance Program for
the Chemical Laboratory." In: Qptimizing Chemical
Laboratory Performance Thro~gh.the Appli~a~ion of Qu~lity
Assurance Principles, Assoc1at10n of Off1clal Analyt1cal
Chemists, Arlington, VA, 198U, pp. 13-23.
-------
5.7.!:>
11.
Section No. 5.7
Revision No. 0
Date: March 30, 1984
Page 4 of 6
10.
American Society for Testing and Materials "Standard
Practice for Determination of Precision and Bias of
Methods of Committee 0-19 on Water," ASTM Designation:
02777-77. In: 1977 Annual Book of ASTM Standards, Part
B., pp. 7-19.
11.
Frazier, R.P., et al., "Establishing a Quality Control
Program for a State Envi ronmenta 1 Laboratory." Water and
Sewaye Works, 121(5). 1974, pp. 54-57.
Use of Control Charts
1.
Shewhart, W.A., Economic Control of Manufacture Products,
Van Nostrand, New York, 1931.
McCully, K.A. and J.G. Lee, "Quality Assurance of Sample
Anal~sis in the Chemical Laboratory." In: Optimiziny
Chemlcal Laboratory Performance through the Application
of Quality Assurance Principles, Association of Official
Analytical Chemists, Arlington, VA, 1980, pp. 57-86.
2.
3.
Duncan, A.J., Quality Control and Industrial Statistics,
3rd edition, Richard D. Irwin, Inc., Homewood, IL, 1968.
4.
Grant, E.L. and R.S. Leavenworth, Statistical Quality
Control, 4th edition, McGraw-Hill, New York, 1972.
5.
Environmental Monitoring and Support Laboratory, Handbook
for Analytical Quality Control in Water and Wastewater
Laboratories, EPA-600/4-79-019, U.S. EPA, Office of
Research and Development, Cincinnati, 1979.
6.
Wernimont, G., "Use of Control charts in the Analytical
Laboratory, I ndustri a 1 and Engi neeri ng Chemi stry, 18( 10) ,
1946, pp. 587-592.
7 .
Bennett, C.A. and N.L. Franklin, Statistical Analysis in
Chemistry and the Chemical Industry, Wiley, New York,
1954.
8.
Eisenhart, C., "Realistic Evaluation of the Precision and
Accuracy of Instrument Calibration Systems." In: Preci-
sion Measurement and Calibration, NBS Special Publication
300, U.S. Department of Commerce, National Bureau of
Standards, 1969, pp. 21-47.
9.
Wernimont, G., "Statistical Control of the Measurement
Process." In: Validation of the Measurement Process, ACS
Washington, D.C., 1977, pp. 1-29.
Moore, P.G., "Normality in Quality Control Charts,"
Applied Statistics, 6(3), 1957, pp. 171-179.
10.
Morrison,J., "The Lognormal Distribution in Quality
Control," Applied Statistics, 7(3),1958, pp. 160-172.
-------
Section No. 1j.7
Revision No. 0
Date: March 30, 1984
Page Ij of 6
12.
I~lewicz, B. and R.H. Myers, IIComparison of Approxima-
tlons to the Percentage Points of the Sample Coefficient
of Variation,1I Technometrics, 12(1), 197U, pp. 166-170.
Environmental Monitoring and Support Laboratory, Quality
Assurance Handbook for Air Pollution Measurement Systems,
Volume I - Principles, EPA-600/9-76-0U5, U.S. EPA, Office
of Research and Development, Research Triangle Park, NC,
1976.
13.
14.
Grubbs, F.E. liThe Difference Control Chart with an Exam-
ple of Its Use,1I Industrial Quality Control, July, 1946,
pp. 22-25.
15.
Page, E.S., IICumulative Sum Charts," Technometrics, 3(1),
1961, pp. 1-9.
16.
Jackson, J.E., IIQuality Control Methods for Several
Related Variables, IITechnometrics, 1(4), 1959, pp.
359-377 .
17 .
Jackson, J.E. and R.H. Morris, IIAn Application of Multi-
variate Quality Control to Photographic Processing,1I
Journal of the American Statistical Association, 52,
1957, pp. 186-199.
18.
Montgomery, D.C. and H.M. Wadsworth, IISome Techniques for
Multivariate Quality Control Applications,1I ASQC Techni-
cal Conference Transactions, 1972.
19.
Frazier, R.P., J.A. Miller, J.F. Murray, M.P. Mauzy,
D.J. Schaeffer and A.F. Westerhold, IIEstablishing a
Quality Control Program for a State Environmental Labora-
tory,1I Water and Sewage Works, 121(5), 1974, pp. 54-57.
Hillier, F.S., IIX and R-Chart Control Limits Based on a
Small Number of Subgroups, IIJournal of Quality Technolo-
.9.t, 1(1),1969. pp. 17-26.
20.
5.7.6 Method Detection Limit
1.
Glaser J.A., D.L. Foerst, G.D. McKee, S.A. Quave, W.L.
Budde,'IITrace Analysis for Wastewaters,1I Environmental
Science and Technology, 15. 1981, pp. 1425-1435.
2.
Hubaux A. and G. Vos, IIDecision and Detection Limits for
Linear'Calibration Curves ,II Analytical Chemistry, 42,
1970, pp. 849-855.
IIGuidelines for Data Acquisition and Data Quality Evalua-
tion in Environmental Chemistry,1I Analytical Chemistry,
52, 1980, pp. 2242-2249.
3.
-------
Section No. 5.7
Revision No. 0
Date: March 30, 1984
Page 6 of 6
4.
Currie, L.A., "Limits for Qualitative Detection and Quan-
titative Determination - Application to Radiochemistry,"
Analytical Chemistry, 40, 1968, pp. 586-594.
5.
Ramirez-Munoz, J., "Qualitative and Quantitative Sensi-
tivity in Flame Photometry," Talanta, 13, 1966, pp. 87-
101.
6.
Parsons, M.L., "The Definition of Detection Limits,"'
Journal of Chemical Education, 46, 1969, pp. 290-292.
7 .
Ingle, J.D., Jr., "Sensitivity and Limit of Detection in
Quantitative Spectrometric Methods," Journal of Chemical
Education, 51, 1974, pp. 100-105.
8.
Wilson, A.L., "The Performance Characteristics of Analy-
tical Methods - III," Talanta, 20, 1973, pp. 725-732.
9.
Kaiser, H., "Guiding Concepts Relating to Trace Analy-
sis," Pure and Applied Chemistry, 34, 1973, pp. 35-61.
Liteanu, C. and I. Rica, "Statistical Theory and Methodo-
logy of Trace Analysis,1I John Wiley and Sons, 1980.
1U.
-------
Section No. 5.8
Revision No. 0
Date: March 30, 19H4
Paye 1 of 3
5.8
A GLOSSARY OF TERMS
Arithmetic mean
(~ for populations,
X for samples)
Bias
In a sample of n units, Xl,
X2 . . . . . X n, the sum of the
observed values in the sample
divided by the number of units in
the sample.
The difference between the popula-
tion mean and the true or reference
value, or as estimated from sample
statistics, the difference between
the sample average and the
reference value.
Coefficient of variation
A measure of relative dispersion.
It is equal to the standard devia-
tion divided by the mean and multi-
plied by 100 to give a percentaye
value.
Correlation Coefficient
A number between -1 and 1 that
indicates the deyree of linear
relationship between two sets of
numbe rs .
Error
The difference between an observed
value and its true value or the
probability interval that contains
the systematic and and random error
with 1-u confidence.
Lot, batch
A definite quantity of samples
collected under conditions that are
considered uniform.
Lot size (N)
The number of units in a particular
lot.
Mat ri x
The material in which the ana-
lyte(s) of primary interest is
embeded.
Method Detection Limit (MOL)
The lowest concentration of an
analyte that a measurement system
can "consistently detect" and/or
measure in replicated field
samples.
The particular value of a charac-
teristic and designated Xl, X2,
X3, and so on.
Observed value, observation,
or variate (X)
-------
Section No. 5.8
Revision No. 0
Date: March 30, 1984
Paye 2 of 3
Parameter
A constant or coefficient that
describes some characteristic of a
population (e.g., standard devia-
tion, mean, reyression coeffi-
cients).
Precision
Deyree of mutual agreement among
individual measurements made under
prescribed conditions.
Qua 1 ity
The totality of features and
characteristics of a product or
servi ce that .bear on its abi 1 i ty to
satisfy given needs.
Quality Assurance
A system of activities whose pur-
pose is to provide adequate confi-
dence that a product or service
will satisfy given needs.
(Juality Control
The operational techniques and the
activities which are aimed at main-
taining a product or a service at a
1 eve 1 of qua 1 ity that will sat is fy
given needs.
Range (R)
The difference between the largest
and smallest numbers in a set of n
numbers.
Range, Relative (RR)
The ranye divided by the mean of a
particular set of numbers.
Regression coefficients
The quantities describiny the slope
and intercept of a regression line.
Intercept (~o for populations, bo for samples),
Slope (~1 for populations, bi for samples).
Regression line or equation
The function that indicates the
regression relationship. For exam-
ple, X = ~o + PI T for popula-
tions, and X = bo + b1 T for
samples.
Sample (statistical)
A group of samples (chemical) taken
from a lot or batch of samples
(chemical).
Sample average
Same as arithmetic mean.
Sample size (n)
The number of units in a sample.
-------
Section No. 5.H
Revision No. 0
Date: March 30, 19H4
Paye 3 of 3
Standard deviation (u)
A measure of the dispersion about
the mean of the elements in a
population.
Standard deviation(s)
A measure of the dispersion about
the average of the elements in a
sample. An estimate of the stan-
dard deviation of a population.
Statistic
A constant or coefficient that
describes some characteristic of a
sample. Statistics are used to
estimate parameters of populations.
The totality, finite or infinite,
of a set of items, units, elements,
measurements, and the like, real or
conceptual, that is under consider-
ation.
Universe or population
-------
~EGRESSION SUMMARY STATISTICS FURM
QA/QC Independent Number
Type of Data Sample Dependent Variable Variable of (u, v)
Quality Indicator Type v u pai rs slope intercept
Bias lot average, Xj or grand true value T
average X (underline one)
Precision within-lot std. dev. Sj true value T
or average within-lot std.
dev. Sw (underline-one)
Precision between-lot std. dev. sa true value T
-
Precision within-lot std. dev. Sj Xj
(primarily for sp 1 it
samp 1 es or repeat ana lysi s
of routine samples)
(}'1
.
I.D
;0
rT1
("')
a
:3:
:3:
rT1
Z
a
rT1
a
..,.,
a
;0
~
-i
C/)
..,.,
a
;0
;0
rT1
""0
a
;0
-i
.......
z
CiJ
a
);>
-i
);>
.c;
c::
);>
r
.......
-i'
-<
""0 a ;0 C/)
PI PI CD CD
c.oC+
-------
BETWEEN-LOT SUMMARY STATISTICS FORM
-
Number Number of Lots Total Number n of Grand Averaye X Over Ihas
Condit ion True Value of k With QC Samp I es at the Lots of Measurements
T Lots Measurements Condition Over All Lots at the Condition X - T s!,/a sab StC
1
2
3
4
5
6
7
8
9
10
a Averaye within-lot standard deviation.
b Between-lot (among-lot) standard deviation.
""'00::0 V>
IlJllJfDfD
lCc-t
------- |