4>EPA United States Environmental Protection Agency ProUCL Version 5.1 User Guide Statistical Software for Environmental Applications for Data Sets with and without Nondetect Observations RESEARCH AND DEVELOPMENT ------- ------- ------- EPA/600/R-07/041 October 2015 www.epa.gov ProUCL Version 5.1 User Guide Statistical Software for Environmental Applications for Data Sets with and without Nondetect Observations Prepared for: Felicia Barnett, Director ORD Site Characterization and Monitoring Technical Support Center (SCMTSC) Superfund and Technology Liaison, Region 4 U.S. Environmental Protection Agency 61 Forsyth Street SW, Atlanta, GA 30303 Prepared by: Anita Singh, Ph.D. and Robert Maichle Lockheed Martin/SERAS IS&GS-CIVIL 2890 Woodbridge Ave Edison NJ 08837 U.S. Environmental Protection Agency Office of Research and Development Washington, DC 20460 Notice: Although this work was reviewed by EPA and approved for publication, it may not necessarily reflect official Agency policy. Mention of trade names and commercial products does not constitute endorsement or recommendation for use. 129cmb07 ------- NOTICE The United States Environmental Protection Agency (U.S. EPA) through its Office of Research and Development (ORD) funded and managed the research described in ProUCL Technical Guide and methods incorporated in the ProUCL software. It has been peer reviewed by the U.S. EPA and approved for publication. Mention of trade names or commercial products does not constitute endorsement or recommendation by the U.S. EPA for use. • All versions of the ProUCL software including the current version ProUCL 5.1 have been developed by Lockheed Martin, IS&GS - CIVIL under the Science, Engineering, Response and Analytical contract with the U.S. EPA and is made available through the U.S. EPA Technical Support Center (TSC) in Atlanta, Georgia (GA). • Use of any portion of ProUCL that does not comply with the ProUCL Technical Guide is not recommended. • ProUCL contains embedded licensed software. Any modification of the ProUCL source code may violate the embedded licensed software agreements and is expressly forbidden. • ProUCL software provided by the U.S. EPA was scanned with McAfee VirusScan version 4.5.1 SP1 and is certified free of viruses. With respect to ProUCL distributed software and documentation, neither the U.S. EPA nor any of their employees, assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed. Furthermore, software and documentation are supplied "as-is" without guarantee or warranty, expressed or implied, including without limitation, any warranty of merchantability or fitness for a specific purpose. ProUCL software is a statistical software package providing statistical methods described in various U.S. EPA guidance documents. ProUCL does not describe U.S. EPA policies and should not be considered to represent U.S. EPA policies. ------- Minimum Hardware Requirements ProUCL 5.1 will function but will run slowly and page a lot. • Intel Pentium 1.0 gigahertz (GHz) • 45 MB of hard drive space • 512 MB of memory (RAM) • CD-ROM drive or internet connection • Windows XP (with SP3), Vista (with SP1 or later), and Windows 7. ProUCL 5.1 will function but some titles and some Graphical User Interfaces (GUIs) will need to be scrolled. Definition without color will be marginal. • 800 by 600 Pixels • Basic Color is preferred Preferred Hardware Requirements • 1 GHz or faster Processor. • 1 gigabyte (GB) of memory (RAM) • 1024 by 768 Pixels or greater color display Software Requirements ProUCL 5.1 has been developed in the Microsoft .NET Framework 4.0 using the C# programming language. To properly run ProUCL 5.1 software, the computer using the program must have the .NET Framework 4.0 pre-installed. The downloadable .NET Framework 4.0 files can be obtained from one of the following websites: • http://msdn.microsoft.com/netframework/downloads/updates/default.aspx http://www.microsoft.com/en-us/download/details.aspx?id=17851 Quicker site for 32 Bit Operating systems • http://www.microsoft.com/en-us/download/details.aspx?id=24872 Use this site if you have a 64 Bit operating system ------- Installation Instructions when Downloading ProUCL 5.1 from the EPA Web Site • Download the file SETUP.EXE from the EPA Web site and save to a temporary location. • Run the SETUP.EXE program. This will create a ProUCL directory and two folders: 1) The USER GUIDE (this document), and 2) DATA (example data sets). • To run the program, use Windows Explorer to locate the ProUCL application file, and Double click on it, or use the RUN command from the start menu to locate the ProUCL.exe file, and run ProUCL.exe. • To uninstall the program, use Windows Explorer to locate and delete the ProUCL folder. Caution: If you have previous versions of the ProUCL, which were installed on your computer, you should remove or rename the directory in which earlier ProUCL versions are currently located. Installation Instructions when Copying ProUCL 5.1 from a CD • Create a folder named ProUCL 5.1 on a local hard drive of the machine you wish to install ProUCL 5.1. • Extract the zipped file ProUCL.zip to the folder you have just created. • Run ProUCL.exe Note: If you have extension turned off, the program will show with the name ProUCL in your directory and have an Icon with the label ProUCL. Creating a Shortcut for ProUCL 5.1 on Desktop • To create a shortcut of the ProUCL program on your desktop, go to your ProUCL directory and right click on the executable program and send it to desktop. A ProUCL icon will be displayed on your desktop. This shortcut will point to the ProUCL directory consisting of all files required to execute ProUCL 5.1. Caution: Because all files in your ProUCL directory are needed to execute the ProUCL software, one needs to generate a shortcut using the process described above. Simply dragging the ProUCL executable file from Window Explorer onto your desktop will not work successfully (an error message will appear) as all files needed to run the software are not available on your desktop. Your shortcut should point to the directory path with all required ProUCL files. ------- ProUCL 5.1 Software ProUCL version 5.1 (ProUCL 5.1), its earlier versions: ProUCL version 3.00.01, 4.00.02, 4.00.04, 4.00.05, 4.1.00, 4.1.01, and ProUCL 5.0.00, associated Facts Sheet, User Guides and Technical Guides (e.g., EPA 2010a, 2010b, 2013a, 2013b) can be downloaded from the following EPA website: http://www.epa.gov/osp/hstl/tsc/software.htm http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm Material for ProUCL webinars offered in March 2011, and relevant literature used in the development of various ProUCL versions can also be downloaded from the above EPA website. Contact Information for all Versions of ProUCL Since 1999, the ProUCL software has been developed under the direction of the Technical Support Center (TSC). As of November 2007, the direction of the TSC is transferred from Brian Schumacher to Felicia Barnett. Therefore, any comments or questions concerning all versions of ProUCL software should be addressed to: Felicia Barnett, Director ORD Site Characterization and Monitoring Technical Support Center (SCMTSC) Superfund and Technology Liaison, Region 4 U.S. Environmental Protection Agency 61 Forsyth Street SW, Atlanta, GA 30303-8960 barnett.felicia@epa.gov (404)562-8659 Fax: (404) 562-8439 ------- Getting Started The look and feel of ProUCL 5.1 is similar to that of ProUCL 5.0; and they share the same names for modules and drop-down menus. The functionality and the use of the methods and options available in ProUCL 5.1 have been illustrated using Screen shots of output screens generated by ProUCL 5.1. ProUCL 5.1 uses a pull-down menu structure, similar to a typical Windows program. For modules where no changes have been made in ProUCL since 2010 (e.g., Sample Sizes), screen shots as used in ProUCL 5.0 documents have been used in ProUCL 5.1 documents. Some of the screen shots generated using ProUCL 5.1 might have ProUCL 5.0 in their titles as those screen shots have not been re-generated and replaced. The screen shown below appears when the program is executed. Navigation Panel 4 Main Window Log Panel The above screen consists of three main window panels: • The MAIN WINDOW displays data sheets and outputs results from the procedure used. • The NAVIGATION PANEL displays the name of data sets and all generated outputs. o The navigation panel can hold up to 40 output files. In order to see more files (data files or generated output files), one can click on Widow Option. o In the NAVIGATION PANEL, ProUCL assigns self explanatory names to output files generated using the various modules of ProUCL. If the same module (e.g., Time Series Plot) is used many times, ProUCL identifies them by using letters a, b, c,...and so on as shown below. ------- Navigation Panel Name Well-mp-27jds REGRESSES Theil-Senjds Trend Test.gst Time Series .gst Time Series_a.gst Time Series_b.gst Time Series_c.gst Mann-Kendall :ds Trend Test_a.gst o The user may want to assign names of his choice to these output files when saving them using the "Save" or "Save As" Options. • The LOG PANEL displays transactions in green, warning messages in orange, and errors in red. For an example, when one attempts to run a procedure meant for left-censored data sets on a full-uncensored data set, ProUCL 5.1 will output a warning in orange in this panel. o Should both panels be unnecessary, you can choose Configure ^- Panel ON/OFF. The use of this option gives extra space to see and print out the statistics of interest. For example, one may want to turn off these panels when multiple variables (e.g., multiple quantile-quantile [Q-Q] plots) are analyzed and goodness-of-fit (GOF) statistics and other statistics may need to be captured for all of the selected variables. The following screen was generated using ProUCL 5.0. An identical screen will be generated using ProUCL 5.1 with title name as ProUCL 5.1- [WorkSheet.xls]. CL 5.0 - [WorkSheetxIs ,; File Nav Name WoikShe Edit | Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Configure Display ^ Cut CtrkX Copy Ctrl+C Paste Ctrl+V Header Name 5 6 Full Precision ^] Log Panel ** Navigation Panel Excel 2003 3 4 5 6 7 8 9 10 11 ------- EXECUTIVE SUMMARY The main objective of the ProUCL software funded by the United States Environmental Protection Agency (EPA) is to compute rigorous statistics to help decision makers and project teams in making good decisions at a polluted site which are cost-effective, and protective of human health and the environment. The ProUCL software is based upon the philosophy that rigorous statistical methods can be used to compute reliable estimates of population parameters and decision making statistics including: the upper confidence limit (UCL) of the mean, the upper tolerance limit (UTL), and the upper prediction limit (UPL) to help decision makers and project teams in making correct decisions. A few commonly used text book type methods (e.g., Central Limit Theorem [CLT], Student's t-UCL) alone cannot address all scenarios and situations occurring in environmental studies. Since many environmental decisions are based upon a 95 percent (%) UCL (UCL95) of the population mean, it is important to compute UCLs of practical merit. The use and applicability of a statistical method (e.g., student's t-UCL, CLT-UCL, adjusted gamma-UCL, Chebyshev UCL, bootstrap-t UCL) depend upon data size, data skewness, and data distribution. ProUCL computes decision statistics using several parametric and nonparametric methods covering a wide-range of data variability, distribution, skewness, and sample size. It is anticipated that the availability of the statistical methods in the ProUCL software covering a wide range of environmental data sets will help the decision makers in making more informative and correct decisions at Superfund and Resource Conservation and Recovery Act (RCRA) sites. It is noted that for moderately skewed to highly skewed environmental data sets, UCLs based on the CLT and the Student's t-statistic fail to provide the desired coverage (e.g., 0.95) to the population mean even when the sample sizes are as large as 100 or more. The sample size requirements associated with the CLT increases with skewness. It would be incorrect to state that a CLT or Student's statistic based UCLs are adequate to estimate Exposure Point Concentrations (EPC) terms based upon skewed data sets. These facts have been described in the published documents (Singh, Singh, and Engelhardt [1997, 1999]; Singh, Singh, and laci 2002; Singh and Singh 2003; and Singh et al. 2006) summarizing simulation experiments conducted on positively skewed data sets to evaluate the performances of the various UCL computation methods. The use of a parametric lognormal distribution on a lognormally distributed data set yields unstable impractically large UCLs values, especially when the standard deviation (sd) of the log- transformed data becomes greater than 1.0 and the data set is of small size less than (<) 30-50. Many environmental data sets can be modeled by a gamma as well as a lognormal distribution. The use of a gamma distribution on gamma distributed data sets tends to yield UCL values of practical merit. Therefore, the use of gamma distribution based decision statistics such as UCLs, UPLs, and UTLs should not be dismissed by stating that it is easier to use a lognormal model to compute these upper limits. The suggestions made in ProUCL are based upon the extensive experience of the developers in environmental statistical methods, published environmental literature, and procedures described in many EPA guidance documents. These suggestions are made to help the users in selecting the most appropriate UCL to estimate the EPC term which is routinely used in exposure assessment and risk management studies of the USEPA. The suggestions are based upon the findings of many simulation studies described in Singh, Singh, and Engelhardt (1997, 1999); Singh, Singh, and laci (2002); Singh and Singh (2003); and Singh et al. (2006). It should be pointed out that a typical simulation study does not (cannot) cover all real world data sets of various sizes and skewness from all distributions. When deemed necessary, the user may want to consult a statistician to select an appropriate upper limit to estimate the EPC term and other environmental parameters of interest. For an analyte (data set) with skewness (sd of logged data) near the end points of the skewness intervals presented in decision tables of Chapter 2 (e.g., Tables 2-9 ------- through 2-11), the user may select the most appropriate UCL based upon the site conceptual site model (CSM), expert site knowledge, toxicity of the analyte, and exposure risks associated with that analyte. The inclusion of outliers in the computation of the various decision statistics tends to yield inflated values of those decision statistics, which can lead to poor decisions. Often statistics that are computed for a data set which includes a few outliers tend to be inflated and represent those outliers rather than representing the main dominant population of interest (e.g., reference area). Identification of outliers, observations coming from population(s) other than the main dominant population is suggested, before computing the decision statistics needed to address project objectives. The project team may want to perform the statistical evaluations twice, once with outliers and once without outliers. This exercise will help the project team in computing reliable and defensible decision statistics which are needed to make cleanup and remediation decisions at polluted sites. The initial development during 1999-2000 and all subsequent upgrades and enhancements of the ProUCL software have been funded by U.S. EPA through its Office of Research and Development (ORD). Initially ProUCL was developed as a research tool for U.S. EPA scientists and researchers of the Technical Support Center (TSC) and ORD- National Exposure Research Laboratory (NERL), Las Vegas. Background evaluations, groundwater (GW) monitoring, exposure and risk management and cleanup decisions in support of the Comprehensive Environmental Recovery, Compensation, and Liability Act (CERCLA) and RCRA site projects of the U.S. EPA are often derived based upon test statistics such as the Shapiro-Wilk (S-W) test, t-test, Wilcoxon-Mann-Whitney (WMW) test, analysis of variance (ANOVA), and Mann-Kendall (MK) test and decision statistics including UCLs of the mean, UPLs, and UTLs. To address the statistical needs of the environmental projects of the USEPA, over the years ProUCL software has been upgraded and enhanced to include many graphical tools and statistical methods described in many EPA guidance documents including: EPA 1989a, 1989b, 1991, 1992a, 1992b, 2000 Multi-Agency Radiation Survey and Site Investigation Manual (MARSSIM), 2002a, 2002b, 2002c, 2006a, 2006b, and 2009. Several statistically rigorous methods (e.g., for data sets with nondetects [NDs]) not easily available in the existing guidance documents and in the environmental literature are also available in ProUCL 5.0/ProUCL 5.1. ProUCL 5.1/ProUCL 5.0 has graphical, estimation, and hypotheses testing methods for uncensored-full data sets and for left-censored data sets including ND observations with multiple detection limits (DLs) or reporting limits (RLs). In addition to computing general statistics, ProUCL 5.1 has goodness-of-fit (GOF) tests for normal, lognormal and gamma distributions, and parametric and nonparametric methods including bootstrap methods for skewed data sets for computation of decision making statistics such as UCLs of the mean (EPA 2002a), percentiles, UPLs for a pre-specified number of future observations (e.g., k with k=\, 2, 3,...), UPLs for mean of future k (>1) observations, and UTLs (e.g., EPA 1992b, 2002b, and 2009). Many positively skewed environmental data sets can be modeled by a lognormal as well as a gamma model. It is well-known that for moderately skewed to highly skewed data sets, the use of a lognormal distribution tends to yield inflated and unrealistically large values of the decision statistics especially when the sample size is small (e.g., <20-30). For gamma distributed skewed uncensored and left-censored data sets, ProUCL software computes decision statistics including UCLs, percentiles, UPLs for future k (>1) observations, UTLs, and upper simultaneous limits (USLs). For data sets with NDs, ProUCL has several estimation methods including the Kaplan-Meier (KM) method, regression on order statistics (ROS) methods and substitution methods (e.g., replacing NDs by DL, DL/2). ProUCL 5.1 can be used to compute upper limits which adjust for data skewness; specifically, for skewed data sets, ProUCL computes upper limits using KM estimates in gamma (lognormal) UCL and UTL equations provided the detected observations in the left-censored data set follow a gamma (lognormal) distribution. Some poor performing commonly used and cited methods such ------- as the DL/2 substitution method and H-statistic based UCL computation method have been retained in ProUCL 5.1 for historical reasons, and research and comparison purposes. The Sample Sizes module of ProUCL can be used to develop data quality objectives (DQOs) based sampling designs and to perform power evaluations needed to address statistical issues associated with a variety of site projects. ProUCL provides user-friendly options to enter the desired values for the decision parameters such as Type I and Type II error rates, and other DQOs used to determine the minimum sample sizes needed to address project objectives. The Sample Sizes module can compute DQO-based minimum sample sizes needed: to estimate the population mean; to perform single and two-sample hypotheses testing approaches; and in acceptance sampling to accept or reject a batch of discrete items such as a lot of drums containing hazardous waste. Both parametric (e.g., t-test) and nonparametric (e.g., Sign test, WMW test, test for proportions) sample size determination methods are available in ProUCL. ProUCL has exploratory graphical methods for both uncensored data sets and for left-censored data sets consisting of ND observations. Graphical methods in ProUCL include histograms, multiple quantile- quantile (Q-Q) plots, and side-by-side box plots. The use of graphical displays provides additional insight about the information contained in a data set that may not otherwise be revealed by the use of estimates (e.g., 95% upper limits) and test statistics (e.g., two-sample t-test, WMW test). In addition to providing information about the data distributions (e.g., normal or gamma), Q-Q plots are also useful in identifying outliers and the presence of mixture populations (e.g., data from several populations) potentially present in a data set. Side-by-side box plots and multiple Q-Q plots are useful to visually compare two or more data sets, such as: site-versus-background concentrations, surface-versus-subsurface concentrations, and constituent concentrations of several GW monitoring wells (MWs). ProUCL also has a couple of classical outlier test procedures, such as the Dixon test and the Rosner test which can be used on uncensored data sets as well as on left-censored data sets containing ND observations. ProUCL has parametric and nonparametric single-sample and two-sample hypotheses testing approaches for uncensored as well as left-censored data sets. Single-sample hypotheses tests: Student's t-test, Sign test, Wilcoxon Signed Rank test, and the Proportion test are used to compare site mean/median concentrations (or some other threshold such as an upper percentile) with some average cleanup standard, Cs (or a not-to-exceed compliance limit, Ao) to verify the attainment of cleanup levels (EPA, 1989a; 2000, 2006a) at remediated site areas of concern. Single-sample tests such as the Sign test and Proportion test, and upper limits including UTLs and UPLs are also used to perform intra-well comparisons. Several two- sample hypotheses tests as described in EPA guidance documents (e.g., 2002b, 2006b, 2009) are also available in the ProUCL software. The two-sample hypotheses testing approaches in ProUCL include: Student's t-test, WMW test, Gehan test and Tarone-Ware (T-W) test. The two-sample tests are used to compare concentrations of two populations such as site versus background, surface versus subsurface soils, and upgradient versus downgradient wells. The Oneway ANOVA module in ProUCL has both classical and nonparametric Kruskal-Wallis (K-W) tests. Oneway ANOVA is used to compare means (or medians) of multiple groups such as comparing mean concentrations of areas of concern and to perform inter-well comparisons. In GW monitoring applications, the ordinary least squares (OLS) regression model, trend tests, and time series plots are used to identify upwards or downwards trends potentially present in constituent concentrations identified in wells over a certain period of time. The Trend Analysis module performs the M-K trend test and Theil- Sen (T-S) trend test on data sets with missing values; and generates trend graphs displaying a parametric OLS regression line and nonparametric T-S trend line. The Time Series Plots option can be used to compare multiple time-series data sets. The use of the incremental sampling methodology (ISM) has been recommended by the Interstate Technology and Regulatory Council (ITRC 2012) for collecting ISM soil samples to compute mean ------- concentrations of the decision units (DUs) and sampling units (SUs) requiring characterization and remediation activities. At many polluted sites, a large amount of discrete onsite and/or offsite background data are already available which cannot be directly compared with newly collected ISM data. In order to provide a tool to compare the existing discrete background data with actual field onsite or background ISM data, a Monte Carlo Background Incremental Sample Simulator (BISS) module was incorporated in ProUCL 5.0 and retained in ProUCL 5.1 (currently blocked from general use) which may be used on a large existing discrete background data set. The BISS module simulates incremental sampling methodology based equivalent background incremental samples. The availability of a large discrete background data set collected from areas with geological conditions comparable to the DU(s) of interest is a pre-requisite for successful application of this module. For now, the BISS module has been blocked for use as this module is awaiting adequate guidance and instructions for its intended use on discrete background data sets. ProUCL software is a user-friendly freeware package providing statistical and graphical tools needed to address statistical issues described in many U.S. EPA guidance documents. ProUCL 5.0/ProUCL 5.1 can process many constituents (variables) simultaneously to: perform statistical tests (e.g., ANOVA and trend test statistics) and compute decision statistics including UCLs of mean, UPLs, and UTLs - a capability not available in several commercial software packages such as Minitab 16 and NADA for R (Helsel 2013). ProUCL also has the capability of processing data by group variables. Special care has been taken to make the software as user friendly as possible. For example, on the various GOF graphical displays, output sheets for GOF tests, OLS and ANOVA, in addition to critical values and/or p-values, the conclusion derived based upon those values is also displayed. ProUCL is easy to use and does not require any programming skills as needed when using commercial software packages and programs written in R script. Methods incorporated in ProUCL have been tested and verified extensively by the developers, researchers, scientists, and users. The results obtained by ProUCL are in agreement with the results obtained by using other software packages including Minitab, SAS®, and programs written in R Script. ProUCL 5.0/ProUCL 5.1 computes decision statistics (e.g., UPL, UTL) based upon the KM method in a straight forward manner without flipping the data and re-flipping the computed statistics for left-censored data sets; these operations are not easy for a typical user to understand and perform. This can become unnecessarily tedious when computing decision statistics for multiple variables/analytes. Moreover, unlike survival analysis, it is important to compute an accurate estimate of the sd which is needed to compute decision making statistics including UPLs and UTLs. For left-censored data sets, ProUCL computes a KM estimate of sd directly. These issues are elaborated by examples discussed in this User Guide and in the accompanying ProUCL 5.1 Technical Guide. ProUCL does not represent a policy software of the government. ProUCL has been developed on limited resources, and it does provide many statistical methods often used in environmental applications. The objective of the freely available user-friendly software, ProUCL is to provide statistical and graphical tools to address environmental issues of environmental site projects for all users including those users who cannot or may not want to program and/or do not have access to commercial software packages. Some users have criticized ProUCL and pointed out some deficiencies such as: it does not have geostatistical methods; it does not perform simulations; and does not offer programming interface for automation. Due to the limited scope of ProUCL, advanced methods have not been incorporated in ProUCL. For methods not available in ProUCL, users can use other statistical software packages such as SAS® (available to EPA personnel) and R script to address their computational needs. Contributions from scientists and researchers to enhance methods incorporated in ProUCL will be very much appreciated. Just like other government documents (e.g., U.S. EPA 2009), various versions of ProUCL (2007, 2009, 2011, 2013, 2016) also make some rule-of thumb type suggestions (e.g., minimum sample size 10 ------- requirement of 8-10) based upon professional judgment and experience of the developers. It is recommended that the users/project team/agencies make their own determinations about the rule-of- thumb type suggestions made in ProUCL before applying a statistical method. 11 ------- ACRONYMS and ABBREVIATIONS ACL A-D, AD AL AOC ANOVA Ao BC BCA BD BISS BTV CC,cc CERCLA CL CLT COPC Cs CSM Df DL DL/2 (t) DL/2 Estimates DOE DQOs DU EA EOF EM EPA Alternative compliance or concentration limit Anderson-Darling test Action limit Area(s) of concern Analysis of variance Not to exceed compliance limit or specified action level Box-Cox transformation Bias-corrected accelerated bootstrap method Binomial distribution Background Incremental Sample Simulator Background threshold value Confidence coefficient Comprehensive Environmental Recovery, Compensation, and Liability Act Compliance limit Central Limit Theorem Contaminant/constituent of potential concern Cleanup standards Conceptual site model Degrees of freedom Detection limit UCL based upon DL/2 method using Student's t-distribution cutoff value Estimates based upon data set with NDs replaced by 1/2 of the respective detection limits Department of Energy Data quality objectives Decision unit Exposure area Empirical distribution function Expectation maximization United States Environmental Protection Agency 12 ------- EPC GA GB GHz GROS GOF, G.O.F. GUI GW HA Ho H-UCL i.i.d. ISM ITRC k, K K K,k khat k star KM (%) KM (Chebyshev) KM(t) KM(z) K-M, KM K-S, KS K-W LCL LN,/« LCL LPL LROS Exposure point concentration Georgia Gigabyte Gigahertz Gamma ROS Goodness-of-fit Graphical user interface Groundwater Alternative hypothesis Null hypothesis UCL based upon Land's H-statistic Independently and identically distributed Incremental sampling methodology Interstate Technology & Regulatory Council Positive integer representing future or next k observations Shape parameter of a gamma distribution Number of nondetects in a data set MLE of the shape parameter of a gamma distribution Biased corrected MLE of the shape parameter of a gamma distribution UCL based upon Kaplan-Meier estimates using the percentile bootstrap method UCL based upon Kaplan-Meier estimates using the Chebyshev inequality UCL based upon Kaplan-Meier estimates using the Student's t-distribution critical value UCL based upon Kaplan-Meier estimates using critical value of a standard normal distribution Kaplan-Meier Kolmogorov-Smirnov Kruskal Wallis Lower confidence limit Lognormal distribution Lower confidence limit of mean Lower prediction limit LogROS; robust ROS 13 ------- LTL LSL M,m MARS SIM MCL MOD MDL MK, M-K ML MLE n N MVUE MW NARPM ND, nd, Nd NERL NRC OKG OLS ORD OSRTI OU PCA PDF, pdf .pdf PRO PROP /^-values QA QC Q-Q Lower tolerance limit Lower simultaneous limit Applied to incremental sampling: number in increments in an ISM sample Multi-Agency Radiation Survey and Site Investigation Manual Maximum concentration limit, maximum compliance limit Minimum detectable difference Method detection limit Mann-Kendall Maximum likelihood Maximum likelihood estimate Number of observations/measurements in a sample Number of observations/measurements in a population Minimum variance unbiased estimate Monitoring well National Association of Remedial Project Managers Nondetect National Exposure Research Laboratory Nuclear Regulatory Commission Orthogonalized Kettenring Gnanadesikan Ordinary least squares Office of Research and Development Office of Superfund Remediation and Technology Innovation Operating unit Principal component analysis Probability density function Files in Portable Document Format Preliminary remediation goals Proposed influence function Probability-values Quality assurance Quality Quantile-quantile 14 ------- R,r RAGS RCRA RL RMLE ROS RPM RSD RV S SCMTSC SD, Sd, sd SE SND SNV SSL SQL su s-w, sw T-S TSC TW, T-W UCL UCL95 UPL U.S. EPA, EPA UTL UTL95-95 USGS USL vs. WMW Applied to incremental sampling: number of replicates of ISM samples Risk Assessment Guidance for Superfund Resource Conservation and Recovery Act Reporting limit Restricted maximum likelihood estimate Regression on order statistics Remedial Project Manager Relative standard deviation Random variable Substantial difference Site Characterization and Monitoring Technical Support Center Standard deviation Standard error Standard Normal Distribution Standard Normal Variate Soil screening levels Sample quantitation limit Sampling unit Shapiro-Wilk Theil-Sen Technical Support Center Tarone-Ware Upper confidence limit 95% upper confidence limit Upper prediction limit United States Environmental Protection Agency Upper tolerance limit 95% upper tolerance limit with 95% coverage U.S. Geological Survey Upper simultaneous limit Versus Wilcoxon-Mann-Whitney 15 ------- WRS Wilcoxon Rank Sum WSR Wilcoxon Signed Rank Xp pth percentile of a distribution < Less than > Greater than > Greater than or equal to < Less than or equal to A Greek letter denoting the width of the gray region associated with hypothesis testing Z Greek letter representing the summation of several mathematical quantities, numbers % Percent a Type I error rate /? Type II error rate 0 Scale parameter of the gamma distribution Z Standard deviation of the log-transformed data A carat sign over a parameter, indicates that it represents a statistic/estimate computed using the sampled data 16 ------- GLOSSARY Anderson-Darling (A-D) test: The Anderson-Darling test assesses whether known data come from a specified distribution. In ProUCL the A-D test is used to test the null hypothesis that a sample data set, x\, ..., xn came from a gamma distributed population. Background Measurements: Measurements that are not site-related or impacted by site activities. Background sources can be naturally occurring or anthropogenic (man-made). Bias: The systematic or persistent distortion of a measured value from its true value (this can occur during sampling design, the sampling process, or laboratory analysis). Bootstrap Method: The bootstrap method is a computer-based method for assigning measures of accuracy to sample estimates. This technique allows estimation of the sample distribution of almost any statistic using only very simple methods. Bootstrap methods are generally superior to ANOVA for small data sets or where sample distributions are non-normal. Central Limit Theorem (CLT): The central limit theorem states that given a distribution with a mean, u, and variance, a2, the sampling distribution of the mean approaches a normal distribution with a mean (u) and a variance o2/N as N, the sample size, increases. Censored Data Sets: Data sets that contain one or more observations which are nondetects. Coefficient of Variation (CV): A dimensionless quantity used to measure the spread of data relative to the size of the numbers. For a normal distribution, the coefficient of variation is given by s/xBar. It is also known as the relative standard deviation (RSD). Confidence Coefficient (CC): The confidence coefficient (a number in the closed interval [0, 1]) associated with a confidence interval for a population parameter is the probability that the random interval constructed from a random sample (data set) contains the true value of the parameter. The confidence coefficient is related to the significance level of an associated hypothesis test by the equality: level of significance = 1 - confidence coefficient. Confidence Interval: Based upon the sampled data set, a confidence interval for a parameter is a random interval within which the unknown population parameter, such as the mean, or a future observation, x0, falls. Confidence Limit: The lower or an upper boundary of a confidence interval. For example, the 95% upper confidence limit (UCL) is given by the upper bound of the associated confidence interval. Coverage, Coverage Probability: The coverage probability (e.g., = 0.95) of an upper confidence limit (UCL) of the population mean represents the confidence coefficient associated with the UCL. Critical Value: The critical value for a hypothesis test is a threshold to which the value of the test statistic is compared to determine whether or not the null hypothesis is rejected. The critical value for any hypothesis test depends on the sample size, the significance level, a at which the test is carried out, and whether the test is one-sided or two-sided. 17 ------- Data Quality Objectives (DQOs): Qualitative and quantitative statements derived from the DQO process that clarify study technical and quality objectives, define the appropriate type of data, and specify tolerable levels of potential decision errors that will be used as the basis for establishing the quality and quantity of data needed to support decisions. Detection Limit: A measure of the capability of an analytical method to distinguish samples that do not contain a specific analyte from samples that contain low concentrations of the analyte. It is the lowest concentration or amount of the target analyte that can be determined to be different from zero by a single measurement at a stated level of probability. Detection limits are analyte and matrix-specific and may be laboratory-dependent. Empirical Distribution Function (EOF): In statistics, an empirical distribution function is a cumulative probability distribution function that concentrates probability \ln at each of the n numbers in a sample. Estimate: A numerical value computed using a random data set (sample), and is used to guess (estimate) the population parameter of interest (e.g., mean). For example, a sample mean represents an estimate of the unknown population mean. Expectation Maximization (EM): The EM algorithm is used to approximate a probability density function (PDF). EM is typically used to compute maximum likelihood estimates given incomplete samples. Exposure Point Concentration (EPC): The constituent concentration within an exposure unit to which the receptors are exposed. Estimates of the EPC represent the concentration term used in exposure assessment. Extreme Values: Values that are well-separated from the majority of the data set coming from the far/extreme tails of the data distribution. Goodness-of-Fit (GOF): In general, the level of agreement between an observed set of values and a set wholly or partly derived from a model of the data. Gray Region: A range of values of the population parameter of interest (such as mean constituent concentration) within which the consequences of making a decision error are relatively minor. The gray region is bounded on one side by the action level. The width of the gray region is denoted by the Greek letter delta, A, in this guidance. H-Statistic: Land's statistic used to compute UCL of mean of a lognormal population H-UCL: UCL based on Land's H-Statistic. Hypothesis: Hypothesis is a statement about the population parameter(s) that may be supported or rejected by examining the data set collected for this purpose. There are two hypotheses: a null hypothesis, (Ho), representing a testable presumption (often set up to be rejected based upon the sampled data), and an alternative hypothesis (HA), representing the logical opposite of the null hypothesis. Jackknife Method: A statistical procedure in which, in its simplest form, estimates are formed of a parameter based on a set of N observations by deleting each observation in turn to obtain, in addition to the usual estimate based on N observations, N estimates each based on N-l observations. 18 ------- Kolmogorov-Smirnov (KS) test: The Kolmogorov-Smirnov test is used to decide if a data set comes from a population with a specific distribution. The Kolmogorov-Smirnov test is based on the empirical distribution function (EDF). ProUCL uses the KS test to test the null hypothesis if a data set follows a gamma distribution. Left-censored Data Set: An observation is left-censored when it is below a certain value (detection limit) but it is unknown by how much; left-censored observations are also called nondetect (ND) observations. A data set consisting of left-censored observations is called a left-censored data set. In environmental applications trace concentrations of chemicals may indeed be present in an environmental sample (e.g., groundwater, soil, sediment) but cannot be detected and are reported as less than the detection limit of the analytical instrument or laboratory method used. Level of Significance (a): The error probability (also known as false positive error rate) tolerated of falsely rejecting the null hypothesis and accepting the alternative hypothesis. Lilliefors test: A goodness-of-fit test that tests for normality of large data sets when population mean and variance are unknown. Maximum Likelihood Estimates (MLE): MLE is a popular statistical method used to make inferences about parameters of the underlying probability distribution of a given data set. Mean: The sum of all the values of a set of measurements divided by the number of values in the set; a measure of central tendency. Median: The middle value for an ordered set of n values. It is represented by the central value when n is odd or by the average of the two most central values when n is even. The median is the 50th percentile. Minimum Detectable Difference (MOD): The MDD is the smallest difference in means that the statistical test can resolve. The MDD depends on sample-to-sample variability, the number of samples, and the power of the statistical test. Minimum Variance Unbiased Estimates (MVUE): A minimum variance unbiased estimator (MVUE or MVU estimator) is an unbiased estimator of parameters, whose variance is minimized for all values of the parameters. If an estimator is unbiased, then its mean squared error is equal to its variance. Nondetect (ND) values: Censored data values. Typically, in environmental applications, concentrations or measurements that are less than the analytical/instrument method detection limit or reporting limit. Nonparametric: A term describing statistical methods that do not assume a particular population probability distribution, and are therefore valid for data from any population with any probability distribution, which can remain unknown. Optimum: An interval is optimum if it possesses optimal properties as defined in the statistical literature. This may mean that it is the shortest interval providing the specified coverage (e.g., 0.95) to the population mean. For example, for normally distributed data sets, the UCL of the population mean based upon Student's t distribution is optimum. 19 ------- Outlier: Measurements (usually larger or smaller than the majority of the data values in a sample) that are not representative of the population from which they were drawn. The presence of outliers distorts most statistics if used in any calculations. Probability - Values (p-value): In statistical hypothesis testing, the p-value associated with an observed value, ^observed of some random variable T used as a test statistic is the probability that, given that the null hypothesis is true, T will assume a value as or more unfavorable to the null hypothesis as the observed value Observed. The null hypothesis is rejected for all levels of significance, a greater than or equal to the p- value. Parameter: A parameter is an unknown or known constant associated with the distribution used to model the population. Parametric: A term describing statistical methods that assume a probability distribution such as a normal, lognormal, or a gamma distribution. Population: The total collection of N objects, media, or people to be studied and from which a sample is to be drawn. It is the totality of items or units under consideration. Prediction Interval: The interval (based upon historical data, background data) within which a newly and independently obtained (often labeled as a future observation) site observation (e.g., onsite, compliance well) of the predicted variable (e.g., lead) falls with a given probability (or confidence coefficient). Probability of Type II (2) Error (P): The probability, referred to as (3 (beta), that the null hypothesis will not be rejected when in fact it is false (false negative). Probability of Type I (1) Error = Level of Significance (a): The probability, referred to as a (alpha), that the null hypothesis will be rejected when in fact it is true (false positive). pth Percentile or pth Quantile: The specific value, Xp of a distribution that partitions a data set of measurements in such a way that the p percent (a number between 0 and 100) of the measurements fall at or below this value, and (100-p) percent of the measurements exceed this value, Xp. Quality Assurance (QA): An integrated system of management activities involving planning, implementation, assessment, reporting, and quality improvement to ensure that a process, item, or service is of the type and quality needed and expected by the client. Quality Assurance Project Plan: A formal document describing, in comprehensive detail, the necessary QA, quality control (QC), and other technical activities that must be implemented to ensure that the results of the work performed will satisfy the stated performance criteria. Quantile Plot: A graph that displays the entire distribution of a data set, ranging from the lowest to the highest value. The vertical axis represents the measured concentrations, and the horizontal axis is used to plot the percentiles/quantiles of the distribution. Range: The numerical difference between the minimum and maximum of a set of values. 20 ------- Regression on Order Statistics (ROS): A regression line is fit to the normal scores of the order statistics for the uncensored observations and is used to fill in values imputed from the straight line for the observations below the detection limit. Resampling: The repeated process of obtaining representative samples and/or measurements of a population of interest. Reliable UCL: see Stable UCL. Robustness: Robustness is used to compare statistical tests. A robust test is the one with good performance (that is not unduly affected by outliers and underlying assumptions) for a wide variety of data distributions. Resistant Estimate: A test/estimate which is not affected by outliers is called a resistant test/estimate Sample: Represents a random sample (data set) obtained from the population of interest (e.g., a site area, a reference area, or a monitoring well). The sample is supposed to be a representative sample of the population under study. The sample is used to draw inferences about the population parameter(s). Shapiro-Wilk (SW) test: Shapiro-Wilk test is a goodness-of-fit test that tests the null hypothesis that a sample data set, x\,..., x» came from a normally distributed population. Skewness: A measure of asymmetry of the distribution of the parameter under study (e.g., lead concentrations). It can also be measured in terms of the standard deviation of log-transformed data. The greater the standard deviation, the greater is the skewness. Stable UCL: The UCL of a population mean is a stable UCL if it represents a number of practical merit (e.g., a realistic value which can actually occur at a site), which also has some physical meaning. That is, a stable UCL represents a realistic number (e.g., constituent concentration) that can occur in practice. Also, a stable UCL provides the specified (at least approximately, as much as possible, as close as possible to the specified value) coverage (e.g., -0.95) to the population mean. Standard Deviation (sd, sd, SD): A measure of variation (or spread) from an average value of the sample data values. Standard Error (SE): A measure of an estimate's variability (or precision). The greater the standard error in relation to the size of the estimate, the less reliable is the estimate. Standard errors are needed to construct confidence intervals for the parameters of interests such as the population mean and population percentiles. Substitution Method: The substitution method is a method for handling NDs in a data set, where the ND is replaced by a defined value such as 0, DL/2 or DL prior to statistical calculations or graphical analyses. This method has been included in ProUCL 5.1 for historical comparative purposes but is not recommended for use. The bias introduced by applying the substitution method cannot be quantified with any certainty. ProUCL 5.1 will provide a warning when this option is chosen. Uncensored Data Set: A data set without any censored (nondetects) observations. 21 ------- Unreliable UCL, Unstable UCL, Unrealistic UCL: The UCL of a population mean is unstable, unrealistic, or unreliable if it is orders of magnitude higher than the other UCLs of a population mean. It represents an unpractically large value that cannot be achieved in practice. For example, the use of Land's H-statistic often results in an impractically large inflated UCL value. Some other UCLs, such as the bootstrap-t UCL and Hall's UCL, can be inflated by outliers resulting in an impractically large and unstable value. All such impractically large UCL values are called unstable, unrealistic, unreliable, or inflated UCLs. Upper Confidence Limit (UCL): The upper boundary (or limit) of a confidence interval of a parameter of interest such as the population mean. Upper Prediction Limit (UPL): The upper boundary of a prediction interval for an independently obtained observation (or an independent future observation). Upper Tolerance Limit (UTL): A confidence limit on a percentile of the population rather than a confidence limit on the mean. For example, a 95% one-sided UTL for 95% coverage represents the value below which 95% of the population values are expected to fall with 95 % confidence. In other words, a 95% UTL with coverage coefficient 95% represents a 95% UCL for the 95th percentile. Upper Simultaneous Limit (USL): The upper boundary of the largest value. xBar: arithmetic average of computed using the sampled data values 22 ------- ACKNOWLEDGEMENTS We wish to express our gratitude and thanks to our friends and colleagues who have contributed during the development of past versions of ProUCL and to all of the many people who reviewed, tested, and gave helpful suggestions throughout the development of the ProUCL software package. We wish to especially acknowledge EPA scientists including Deana Crumbling, Nancy Rios-Jafolla, Tim Frederick, Dr. Maliha Nash, Kira Lynch, and Marc Stiffleman; James Durant of ATSDR, Dr. Steve Roberts of University of Florida, Dr. Elise A. Striz of the National Regulatory Commission (NRC), and Drs. Phillip Goodrum and John Samuelian of Integral Consulting Inc. for testing and reviewing ProUCL 5.0 and its associated guidance documents, and for providing helpful comments and suggestions. We also wish to thank Dr. D. Beal of Leidos for reviewing ProUCL 5.0. Special thanks go to Ms. Donna Getty and Mr. Richard Leuser of Lockheed Martin for providing a thorough technical and editorial review of ProUCL 5.1 and also ProUCL 5.0 User Guide and Technical Guide. A special note of thanks is due to Ms. Felicia Barnett of EPA ORD Site Characterization and Monitoring Technical Support Center (SCMTSC), without whose assistance the development of the ProUCL 5.1 software and associated guidance documents would not have been possible. Finally, we wish to dedicate the ProUCL 5.1 (and ProUCL 5.0) software package to our friend and colleague, John M. Nocerino who had contributed significantly in the development of ProUCL and Scout software packages. 23 ------- Table of Contents NOTICE 1 Minimum Hardware Requirements 2 Software Requirements 2 Installation Instructions when Downloading ProUCL 5.1 from the EPA Web Site 3 ProUCL5.1 4 Contact Information for all Versions of ProUCL 4 EXECUTIVE SUM MARY 7 GLOSSARY 17 ACKNOWLEDGEMENTS 23 Table of Contents 24 INTRODUCTION OVERVIEW OF ProUCL VERSION 5.1 SOFTWARE 29 The Need for ProUCL Software 34 ProUCL 5.1 Capabilities 37 ProUCL 5.1 Technical Guide 44 Chapter 1 Guidance on the Use of Statistical Methods in ProUCL Software 45 1.1 Background Data Sets 45 1.2 Site Data Sets 46 1.3 Discrete Samples or Composite Samples? 47 1.4 Upper Limits and Their Use 48 1.5 Point-by-Point Comparison of Site Observations with BTVs, Compliance Limits and Other Threshold Values 50 1.6 Hypothesis Testing Approaches and Their Use 51 1.6.1 Single Sample Hypotheses (Pre-established BTVs and Not-to-Exceed Values are Known) 51 1.6.2 Two-Sample Hypotheses (BTVs and Not-to-Exceed Values are Unknown) 52 1.7 Minimum Sample Size Requirements and Power Evaluations 53 1.7.1 Why a data set of minimum size, n = 8-10? 54 1.7.2 Sample Sizes for Bootstrap Methods 55 1.8 Statistical Analyses by a Group ID 56 1.9 Statistical Analyses for Many Constituents/Variables 56 1.10 Use of Maximum Detected Value as Estimates of Upper Limits 56 1.10.1 Use of Maximum Detected Value to Estimate BTVs and Not-to-Exceed Values 57 1.10.2 Use of Maximum Detected Value to Estimate EPC Terms 57 1.11 Samples with Nondetect Observations 58 1.11.1 Avoid the Use of the DL/2 Substitution Method to Compute UCL95 58 1.11.2 ProUCL Does Not Distinguish between Detection Limits, Reporting limits, or Method Detection Limits 59 1.12 Samples with Low Frequency of Detection 59 1.13.1 Identification of COPCs 60 1.13.2 Identification of Non-Compliance Monitoring Wells 60 24 ------- 1.13.3 Verification of the Attainment of Cleanup Standards, Cs 60 1.13.4 Using BTVs (Upper Limits) to Identify Hot Spots 61 1.14 Some General Issues, Suggestions and Recommendations made by ProUCL 61 1.14.1 Handling of Field Duplicates 61 1.14.2 ProUCL Recommendation about ROS Method and Substitution (DL/2) Method 61 1.14.3 Unhandled Exceptions and Crashes in ProUCL 61 1.15 The Unofficial User Guide to ProUCL4 (Helsel and Gilroy 2012) 62 1.16 Box and Whisker Plots 69 Chapter 2 Entering and Manipulating Data 74 2.1 Creating a New Data Set 74 2.2 Opening an Existing Data Set 74 2.3 Input File Format 75 2.4 Number Precision 76 2.6 Saving Files 78 2.7 Editing 79 2.8 Handling Nondetect Observations and Generating Files with Nondetects 79 2.9 Caution 80 2.10 Summary Statistics for Data Sets with Nondetect Observations 81 2.11 Warning Messages and Recommendations for Data Sets with an Insufficient Amount of Data 82 2.12 Handling Missing Values 84 2.13 User Graphic Display Modification 86 2.13.1 Graphics Tool Bar 86 2.13.2 Drop-Down Menu Graphics Tools 86 Chapter 3 Select Variables Screen 88 3.1 Select Variables Screen 88 3.1.1 Graphs by Groups 90 Chapter 4 General Statistics 93 4.1 General Statistics for Full Data Sets without NDs 93 4.2 General Statistics with NDs 95 Chapter 5 Imputing Nondetects Using ROS Methods 97 Chapter 6 Graphical Methods (Graph) 99 6.1 Box Plot 101 6.2 Histogram 106 6.3 Q-Q Plots 107 6.4 Multiple Q-Q Plots 109 6.4.1 Multiple Q-Q plots (Uncensored data sets) 109 6.5 Multiple Box Plots 110 6.5.1 Multiple Box plots (Uncensored data sets) 110 Chapter 7 Classical Outlier Tests 112 7.1 Outlier Test for Full Data Set 113 7.2 Outlier Test for Data Sets with NDs 114 Chapter 8 Goodness-of-Fit (GOF) Tests for Uncensored and Left-Censored Data Sets.119 8.1 Goodness-of-Fit test in ProUCL 119 8.2 Goodness-of-Fit Tests for Uncensored Full Data Sets 122 25 ------- 8.2.1 GOF Tests for Normal and Lognormal Distribution 123 8.2.2 GOF Tests for Gamma Distribution 125 8.3 Goodness-of-Fit Tests Excluding NDs 127 8.3.1 Normal and Lognormal Options 127 8.3.2 Gamma Distribution Option 131 8.4 Goodness-of-Fit Tests with ROS Methods 133 8.4.1 Normal or Lognormal Distribution (Log-ROS Estimates) 133 8.4.2 Gamma Distribution (Gamma-ROS Estimates) 135 8.5 Goodness-of-Fit Tests with DL/2 Estimates 137 8.5.1Normal or Lognormal Distribution (DL/2 Estimates) 137 8.6 Goodness-of-Fit Test Statistics 137 Chapter 9 Single-Sample and Two-Sample Hypotheses Testing Approaches 141 9.1 Single-Sample Hypotheses Tests 141 9.1.1 Single-Sample Hypothesis Testing for Full Data without Nondetects 142 9.1.1.1 Single-Sample t-Test 143 9.1.1.2 Single-Sample Proportion Test 144 9.1.1.3 Single-Sample Sign Test 146 9.1.1.4 Single-Sample Wilcoxon Signed Rank (WSR) Test 149 9.1.2 Single-Sample Hypothesis Testing for Data Sets with Nondetects 151 9.1.2.1 Single Proportion Test on Data Sets with NDs 151 9.1.2.2 Single-Sample Sign Test with NDs 154 9.1.2.3 Single-Sample Wilcoxon SignedRank TestwithNDs 155 9.2 Two-Sample Hypotheses Testing Approaches 157 9.2.1 Two-Sample Hypothesis Tests for Full Data 158 9.2.1.1 Two-Sample t-Test without NDs 160 9.2.1.2 Two-Sample Wilcoxon-Mann-Whitney (WMW) Test without NDs 163 9.2.2 Two-Sample Hypothesis Testing for Data Sets with Nondetects 165 9.2.2.1 Two-Sample Wilcoxon-Mann-Whitney Test with Nondetects 166 9.2.2.2 Two-Sample Gehan Test for Data Sets with Nondetects 168 9.2.2.3 Two-Sample Tarone-Ware Test for Data Sets with Nondetects.. Ill Chapter 10 Computing Upper Limits to Estimate Background Threshold Values Based Upon Full Uncensored Data Sets and Left-Censored Data Sets with Nondetects 175 10.1 Background Statistics for Full Data Sets without Nondetects 176 10.1.1 Normal or Lognormal Distribution 177 10.1.2 Gamma Distribution 179 10.1.3 Nonparametric Methods 182 10.1.4 All Statistics Option 184 10.2 Background Statistics with NDs 186 10.2.1 Normal or Lognormal Distribution 187 10.2.2 Gamma Distribution 190 10.2.3 Nonparametric Methods (with NDs) 193 10.2.4 All Statistics Option 194 Chapter 11 Computing Upper Confidence Limits (UCLs) of Mean Based Upon Full- Uncensored Data Sets and Left-Censored Data Sets with Nondetects 200 11.1 UCLs for Full (w/o NDs) Data Sets 202 11.1.1 Normal Distribution (Full Data Sets without NDs) 202 26 ------- 11.1.2 Gamma, Lognormal, Nonparametric, All Statistics Option (Full Data without NDs) 204 11.2 UCL for Left-Censored Data Sets with NDs 208 Chapter 12 Sample Sizes Based Upon User Specified Data Quality Objectives (DQOs) and Power Assessment 212 12.1 Estimation of Mean 214 12.2 Sample Sizes for Single-Sample Hypothesis Tests 215 12.2.1 Sample Size for Single-Sample t-Test 215 12.2.2 Sample Size for Single-Sample Proportion Test 216 12.2.3 Sample Size for Single-Sample Sign Test 217 12.2.4 Sample Size for Single-Sample Wilcoxon Signed Rank Test 219 12.3 Sample Sizes for Two-Sample Hypothesis Tests 220 12.3.1 Sample Size for Two-Sample t-Test 220 12.3.2 Sample Size for Two-Sample Wilcoxon Mann-Whitney Test 221 12.4 Sample Sizes for Acceptance Sampling 223 Chapter 13 Analysis of Variance 224 13.1 Classical Oneway ANOVA 224 13.2 Nonparametric ANOVA 226 Chapter 14 Ordinary Least Squares of Regression and Trend Analysis 228 14.1 Simple Linear Regression 228 14.2 Mann-Kendall Test 232 14.3 Theil - Sen Test 235 14.4 Time Series Plots 238 Chapter 15 Background Incremental Sample Simulator (BISS) Simulating BISS Data from a Large Discrete Background Data 244 Chapter 16 Windows 246 Chapter 17 Handling the Output Screens and Graphs 247 17.1 Copying and Saving Graphs 247 17.2 Printing Graphs 248 17.3 Making Changes in Graphs using Tools and Properties 250 17.4 Printing Non-graphical Outputs 250 17.5 Saving Output Screens as Excel Files 251 Chapter 18 Summary and Recommendations to Compute a 95% UCL for Full Uncensored and Left-Censored Data Sets with NDs 253 18.1 Computing UCL95s of the Mean Based Upon Uncensored Full Data Sets 253 18.2 Computing UCLs Based Upon Left-Censored Data Sets with Nondetects 254 REFERENCES 256 27 ------- 28 ------- INTRODUCTION OVERVIEW OF ProUCL VERSION 5.1 SOFTWARE The main objective of the ProUCL software funded by the U.S.EPA is to compute rigorous decision statistics to help the decision makers in making reliable decisions which are cost-effective, and protective of human health and the environment. The development of ProUCL software is based upon the philosophy that rigorous statistical methods can be used to compute representative estimates of population parameters (e.g., site mean, background percentiles) and accurate decision making statistics (including the upper confidence limit [UCL] of the mean, upper tolerance limit [UTL], and upper prediction limit [UPL]) which will assist decision makers and project teams in making sound decisions. The use and applicability of a statistical method (e.g., student's t-UCL, Central Limit Theorem (CLT)-UCL, adjusted gamma-UCL, Chebyshev UCL, bootstrap-t UCL) depend upon data size, data variability, data skewness, and data distribution. ProUCL computes decision statistics using several parametric and nonparametric methods covering a wide-range of data variability, skewness, and sample size. A couple of text book methods described in most of the statistical text books (e.g., Hogg and Craig, 1995) based upon the Student's t-statistic and the CLT alone cannot address all scenarios and situations commonly occurring in environmental studies. It is incorrect to assume that Student's t-statistic and/or CLT based UCLs of mean will provide the desired coverage (e.g., 0.95) to the population mean irrespective of the skewness of the data set/population under consideration. These issues have been discussed in detail in Chapters 2 and 4 of the accompanying ProUCL 5.1 Technical Guide. Several examples are provided in the Technical Guide which elaborate on these issues. The use of a parametric lognormal distribution on a lognormally distributed data set tends to yield unstable impractically large UCL values, especially when the standard deviation of the log-transformed data is greater than 1.0 and the data set is of small size such as less than 30-50 (Hardin and Gilbert 1993; Singh, Singh, and Engelhardt 1997). Many environmental data sets can be modeled by a gamma as well as a lognormal distribution. Generally, the use of a gamma distribution on gamma distributed data sets yields UCL values of practical merit (Singh, Singh, and laci 2002). Therefore, the use of gamma distribution based decision statistics such as UCLs, UPL, and UTLs cannot be dismissed just because it is easier to use a lognormal model to compute these upper limits. The two distributions do not behave in a similar manner. The advantages of computing the gamma distribution-based decision statistics are discussed in Chapters 2 through 5 of the ProUCL Technical Guide. Since many environmental decisions are made based upon a 95% UCL of the population mean, it is important to compute reliable UCLs and other decision making statistics of practical merit. In an effort to compute stable UCLs of the population mean and other decision making statistics, in addition to computing the Student's t statistic and the CLT based statistics (e.g., UCLs, UPLs), significant effort has been made to incorporate rigorous statistical methods for computing UCLs (and other limits) in the ProUCL software, covering a wide-range of data skewness and sample sizes (e.g., Singh, Singh, and Engelhardt, 1997; Singh, Singh, and laci, 2002; and Singh, Singh, 2003). It is anticipated that the availability of the statistical methods in the ProUCL software, which can be applied to a wide range of environmental data sets, will help decision makers in making more informative, practical and sound decisions. It is noted that even for skewed data sets, practitioners tend to use the CLT or Student's t-statistic based UCLs of mean for "large" sample sizes of 25-30 (rule-of-thumb to use CLT). However, this rule-of- thumb does not apply for moderately to highly skewed data sets, specifically when a (standard deviation 29 ------- of the log-transformed data) starts exceeding 1. The large sample size requirement associated with the use of the CLT depends upon the skewness of the data distribution under consideration. The large sample requirement associated with CLT for the sample mean to follow an approximate normal distribution increases with the data skewness; and for highly skewed data sets, even samples of size greater than (>)100 may not be large enough for the sample mean to follow an approximate normal distribution. For moderately skewed to highly skewed environmental data sets, as expected, UCLs based on the CLT and the Student's t-statistic fail to provide the desired coverage of the population mean even when the sample sizes are as large as 100 or more. These facts have been verified in the published simulation experiments conducted on positively skewed data sets (e.g., Singh, Singh, and Engelhardt, 1997; Singh, Singh, and laci, 2002; and Singh and Singh, 2003); some graphs showing the simulation results are provided in Appendix B of the ProUCL 5.1 Technical Guide. The initial development and all subsequent upgrades and enhancements of the ProUCL software have been funded by the U.S. EPA through its Office of Research and Development (ORD). Initially ProUCL was developed as a research tool for scientists and researchers of the Technical Support Center and ORD- NERL, Las Vegas. During 1999-2001, the initial intent and objectives of developing the ProUCL software (Version 1.0 and Version 2.0) were to provide a statistical research tool for EPA scientists which can be used to compute theoretically sound 95% upper confidence limits (UCL95s) of the mean routinely used in exposure assessment, risk management and cleanup decisions made at various CERCLA and RCRA sites (EPA 1992a, 2002a). During 2002, the peer-reviewed ProUCL version 2.1 (with Chebyshev inequality based UCLs) was released for public use. Several researchers have developed rigorous parametric and nonparametric statistical methods (e.g., Johnson 1978; Grice and Bain 1980; Efron [1981 1982]; Efron and Tibshirani 1993; Hall [1988, 1992]; Sutton 1993; Chen 1995; Singh, Singh, and Engelhardt 1997; Singh, Singh, and laci 2002] to compute upper limits (e.g., UCLs) which adjust for data skewness. Since Student's t-UCL, CLT-UCL, and percentile bootstrap UCL fail to provide the desired coverage to the population mean of skewed distributions, several parametric (e.g., gamma distribution based) and nonparametric (e.g., bias-corrected accelerated [BCA] bootstrap and bootstrap-t, Chebyshev UCL) UCL computation methods which adjust for data skewness were incorporated in ProUCL versions 3.0 and 3.00.02 during 2003-2004. ProUCL version 3.00.02 also had graphical Q-Q plots and GOF tests for normal, lognormal, and gamma distributions; capabilities to statistically analyze multiple variables simultaneously were also incorporated in ProUCL 3.00.02 (EPA 2004). It is important to compute decision statistics (e.g., UCLs, UTLs) which are cost-effective and protective of human health and the environment (balancing between Type I and Type II errors), therefore, one cannot dismiss the use of the better [better than t-UCL, CLT-UCL, ROS and KM percentile bootstrap UCL, KM-UCL (t)] performing UCL computation methods including gamma UCLs and the various bootstrap UCLs which adjust for data skewness. During 2004-2007, ProUCL was upgraded to versions 4.00.02, and 4.00.04. These upgrades included exploratory graphical (e.g., Q-Q plots, box plots) and statistical (e.g., maximum likelihood estimation [MLE], KM, and ROS) methods for left-censored data sets consisting of nondetect (NDs) observations with multiple DLs or RLs. For uncensored and left- censored data sets, these upgrades provide statistical methods to compute upper limits: percentiles, UPLs and UTLs needed to estimate site-specific background level constituent concentrations or background threshold values (BTVs). To address statistical needs of background evaluation projects (e.g., EPA 2000, 2002b), several single-sample and two-sample hypotheses testing approaches were also included in these ProUCL upgrades. During 2008-2010, ProUCL was upgraded to ProUCL 4.00.05. The upgraded ProUCL was enhanced by including methods to compute gamma distribution based UPLs and UTLs (Krishnamoorthy, Mathew, and Mukherjee 2008). The Sample Size module to compute DQOs-based minimum sample sizes, needed to 30 ------- address statistical issues associated with environmental projects (e.g., EPA 2000,2002c, 2006a, 2006b), was also incorporated in ProUCL 4.00.05. During 2009-2011, ProUCL 4.00.05 was upgraded to ProUCL 4.1 and 4.1.01. ProUCL 4.1 (2010) and 4.1.01 (2011) retain all capabilities of the previous versions of ProUCL software. Two new modules: Oneway ANOVA and Trend Analysis were included in ProUCL 4.1. The Oneway ANOVA module has both parametric and nonparametric ANOVA tests to perform inter-well comparisons. The Trend Analysis module can be used to determine potential upward or downward trends present in constituent concentrations identified in GW monitoring wells (MWs). The Trend Analysis module can compute Mann-Kendall (MK) and Theil-Sen (T-S) trend statistics to determine upward or downward trends potentially present in analyte concentrations. ProUCL 4.1 also has the OLS Regression module. In ProUCL 4.1, some modifications were made in decision tables which are used to make suggestions regarding the use of UCL95 for estimating EPCs. Specifically, based upon experience, developers of ProUCL re-iterated that the use of a lognormal distribution for estimating EPCs and BTVs should be avoided, as the use of the lognormal distribution tends to yield unrealistic and unstable values of decision making statistics including UCLs, UPLs, and UTLs. This is especially true when the sample size is <20- 30 and the data set is moderately to highly skewed. During March 2011, webinars were presented describing the capabilities and use of the methods available in ProUCL 4.1, which can be downloaded from the EPA ProUCL website. ProUCL version 5.0.00 (EPA 2013, 2014) represents an upgrade of ProUCL 4.1.01 (EPA June 2011) which represents an upgrade of ProUCL 4.1.00 (EPA 2010). For uncensored and left-censored data sets, ProUCL 5.0.00 (ProUCL 5.0) contains all statistical and graphical methods that were available in the previous versions of the ProUCL software package except for some poor performing and restricted (e.g., can be used only when a single detection limit is present) estimation methods such as the MLE and winsorization methods for left-censored data sets. ProUCL has GOF tests for normal, lognormal, and gamma distributions for uncensored and left-censored data sets with NDs. ProUCL 5.0 has the extended version of the Shapiro-Wilk (S-W) test to perform normal and lognormal GOF tests for data sets of sizes up to 2000 (Royston [1982, 1982a]). In addition to normal and lognormal distribution- based decision statistics, ProUCL software computes UCLs, UPLs, and UTLs based upon the gamma distribution. Several enhancements were made in the UCLs/EPCs and Upper Limits/BTVs modules of the ProUCL 5.0 software. A new statistic, an upper simultaneous limit (USL) (Singh and Nocerino 2002; Wilks 1963) has been incorporated in the Upper Limits/BTVs module of ProUCL 5.0 for data sets consisting of NDs with multiple DLs. A two-sample hypothesis test, the Tarone-Ware (T-W; Tarone and Ware, 1978) test has also been incorporated in ProUCL 5.0. Nonparametric tolerance limits have been enhanced, and for specific values of confidence coefficients, coverage probability, and sample size, ProUCL 5.0 outputs the confidence coefficient (CC) actually achieved by a UTL. The Trend Analysis and OLS Regression modules can handle missing events when computing trend test statistics and generating trend graphs. Some new methods using KM estimates in gamma (and lognormal) distribution-based UCL, UPL, and UTL equations have been incorporated to compute the decision statistics for data sets consisting of nondetect observations. To facilitate the computation of UCLs from ISM based samples (ITRC 2012); the minimum sample size requirement has been lowered to 3, so that one can compute the UCL95 based upon ISM data sets of sizes >3. All known bugs, typographical errors, and discrepancies found by the developers and users of the ProUCL software package were addressed in ProUCL version 5.0.00. Specifically, a discrepancy found in the estimate of mean based upon the KM method was fixed in ProUCL 5.0. Some changes were made in the decision logic used in the Goodness of Fit and UCLs/EPCs modules. In practice, based upon a ------- given data set, it is well known that the two statistical tests (e.g., T-S and OLS trend tests) can lead to different conclusions. To streamline the decision logic associated with the computation of the various UCLs, the decision tables in ProUCL 5.0 were updated. Specifically, for each distribution if at least one of the two GOF tests (e.g., Shapiro-Wilk or Lilliefors test for normality) determines that the hypothesized distribution holds, then ProUCL concludes that the data set follows the hypothesized distribution, and decision statistics are computed accordingly. Additionally, for gamma distributed data sets, ProUCL 5.0 suggests the use of the: adjusted gamma UCL for samples of sizes < 50 (instead of 40 suggested in previous versions); and approximate gamma UCL for samples of sizes >50. Also, for samples of larger sizes (e.g., with n > 100) and small values of the gamma shape parameter, k (e.g., k < 0.1), significant discrepancies were found in the critical values of the two gamma GOF test statistics (Anderson-Darling [A-D] and Kolmogorov Smirnov [K-S] tests) obtained using the two gamma deviate generation algorithms: Whitaker (1974) and Marsaglia and Tsang (2000). For values of k < 0.2, the critical values of the two gamma GOF tests: A-D and K-S tests have been updated using the currently available more accurate gamma deviate generation algorithm due to Marsaglia and Tsang's (2000); more details about the implementation of their algorithm can be found in Kroese, Taimre, and Botev (2011). For values of the shape parameter, k=0.025, 0.05, 0.1, and 0.2, the critical value tables for these two tests were updated by incorporating the newly generated critical values for the three significance levels: 0.05, 0.1, and 0.01. The updated tables are provided in Appendix A of the ProUCL 5.0/ProUCL 5.1 Technical Guide. It should be noted that for k=0.2, the older and the newly generated critical values are in general agreement; therefore, critical values for k=0.2 were not replaced in tables summarized in Appendix A of the ProUCL Technical Guide. ProUCL 5.0 also has a new Background Incremental Sample Simulator (BISS) module (temporarily blocked for general public use) which can be used on a large existing discrete background data set to simulate background incremental samples. The availability of a large discrete data set collected from areas with geological formations and conditions comparable to the DUs (background or onsite) of interest is a requirement for successful application of this module. The simulated BISS data can be compared with the actual field ISM (ITRC 2012) data collected from the various DUs using other modules of ProUCL 5.0. The values of the BISS data are not directly available to users; however, the simulated BISS data can be accessed by the various modules of ProUCL 5.0 to perform desired statistical evaluations. For example, the simulated background BISS data can be merged with the actual field ISM data after comparing the two data sets using a two-sample t-test; the simulated BISS or the merged data can be used to compute a UCL of the mean or a UTL. Note: The ISM methodology used to develop the BISS module is a relatively new approach; methods incorporated in this BISS module requires further investigation. For now, the BISS module has been blocked for use in ProUCL 5.0/ProUCL 5.1 as this module is awaiting adequate guidance and instructions for its intended use on discrete background data sets. ProUCL 5.0 is a user-friendly freeware package providing statistical and graphical tools needed to address statistical issues described in several EPA guidance documents. Considerable effort was made to provide a detailed technical guide to help practitioners understand the statistical methods needed to address the statistical needs of their environmental projects. ProUCL generates detailed output sheets and graphical displays for each method which can be used to educate students learning environmental statistical methods. Like previous versions, ProUCL 5.0 can process many variables simultaneously to compute various tests (e.g., ANOVA and trend test statistics) and decision statistics including UCL of the mean, UPLs, and UTLs, a capability not available in other software packages such as Minitab 16 and NADA for R (Helsel 2013). Without the availability of this option, the user has to compute decision and 32 ------- test statistics for one variable at a time which becomes cumbersome when dealing with a large number of variables. ProUCL 5.0 also has the capability of processing data by groups. ProUCL 5.0 is easy to use; it does not require any programming skills as needed when using programs written in R Script. Deficiencies Identified in ProUCL 5.0: For ProUCL to be compatible with Microsoft Office 8 and provide Excel-compatible Spreadsheet functionality (e.g., ability to input/output *.xlsx files), ProUCL 5.0 used FarPoint Spread 5 for .NET; and for graphics, ProUCL 5.0 used the development software package, ChartFx 7. The look and feel of ProUCL 5.0 is quite different from its previous versions; all main menu options were re-arranged. However, the use of upgraded development softwares resulted in some problems. Specifically, it takes an unacceptably long time to save large ProUCL 5.0 generated output files using FarPoint Spread 5. Also the use of ChartFx 7 caused some problems in properly labeling axes for histograms. Additionally some unhandled exceptions and crashes were noted by users. The unhandled exceptions were mainly noted for "bad" data sets including data sets not following ProUCL input format; data sets with not enough observations; and data sets with not enough detects. ProUCL 5.1: ProUCL 5.1 represents an upgrade of ProUCL 5.0 to address deficiencies identified in ProUCL 5.0. ProUCL 5.1 retains all capabilities of ProUCL 5.0 as described above. All modules in ProUCL 5.1, and their look and feel is the same as in ProUCL 5.0. In this document, any statement made about the capabilities of ProUCL 5.0 also apply to ProUCL version 5.1; and to save time, not all screen shots used in ProUCL 5.0 manuals have been replaced in the ProUCL 5.1 User Guide and Technical Guide. Upgrades in ProUCL 5.1 (not available in earlier versions) have been labeled as New in ProUCL 5.1 in this document. All known bugs, crashes, and unhandled exceptions (e.g., on bad data sets) found in ProUCL 5.0 have been addressed in ProUCL 5.1. In ProUCL 5.1, some enhancements have been made in the Trend Analysis option of the Statistical Test module of ProUCL 5.1. ProUCL 5.1 computes and outputs residuals for the non-parametric T-S trend line which may be helpful to compute a prediction band around the T-S trend line. In addition to generating Q-Q plots based upon detected observations, the Goodness of Fit Tests option of the Statistical Tests module of ProUCL 5.1 generates censored probability plots for data sets with NDs. Some changes have been made in the decision table used to make suggestions for UCL selection based upon a gamma distribution. New licensing agreements were obtained for the development softwares: FarPoint and ChartFx. Due to deficiencies present in the development software, ProUCL 5.1 generated large output files still take a long time to be saved. However, there is a quick work around to this problem, instead of saving the output sheet using ProUCL, one can copy the output spreadsheet and save the copied output sheet using Excel. This operation can be carried out instantly. Also, ChartFx 7.0 has some deficiencies, and labeling along the x-axis on a histogram is still not as desirable as one would like it to be. Some tools have been added in ProUCL 5.1, and relevant statistics (e.g., start point, midpoint, and end point) of a histogram bar can be displayed by hovering the cursor on that bar. Software ProUCL version 5.1, its earlier versions: ProUCL version 3.00.02, 4.00.02, 4.00.04, 4.1.00, 4.1.01 and ProUCL 5.0, associated Facts Sheet, User Guides and Technical Guides (e.g., EPA [2004, 2007, 2009a, 2009b, 2010a, 2010b, 2013a, 2013b]) can be downloaded from the EPA website: http://www.epa.gov/osp/hstl/tsc/software.htm http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm 33 ------- The Need for ProUCL Software EPA guidance documents (e.g., EPA [1989a, 1989b, 1992a, 1992b, 1994, 1996, 2000, 2002a, 2002b, 2002c, 2006a, 2006b, 2009a, and 2009b]) describe statistical methods including: DQOs-based sample size determination procedures, methods to compute decision statistics: UCL95, UPL, and UTLs, parametric and nonparametric hypotheses testing approaches, Oneway ANOVA, OLS regression, and trend determination approaches. Specifically, EPA guidance documents (2000, 2002c, 2006a, 2006b) describe DQOs-based parametric and nonparametric minimum sample size determination procedures needed: to compute decision statistics (e.g., UCL95); to perform site versus background comparisons (e.g., t-test, proportion test, WMW test); and to determine the number of discrete items (e.g., drums filled with hazardous material) that need to be sampled to meet the DQOs (e.g., specified proportion, p0 of defective items, allowable error margin in an estimate of mean). Statistical methods are used to compute test statistics (e.g., S-W test, t-test, WMW test, T-S trend statistic) and decision statistics (e.g., 95% UCL, 95% UPL, UTL95-95) needed to address statistical issues associated with CERCLA and RCRA site projects. For example, exposure and risk management and cleanup decisions in support of EPA projects are often made based upon the mean concentrations of the contaminants/constituents of potential concern (COPCs). Site-specific BTVs are used in site versus background evaluation studies. A UCL95 is used to estimate the EPC terms (EPA 1992a, 2002a); and upper limits such as upper percentiles, UPLs, or UTLs are used to estimate BTVs or not-to-exceed values (EPA 1992b, 2002b, and 2009). The estimated BTVs are used to address several objectives: to identify the COPCs; to identify the site areas of concern (AOCs); to perform intra-well comparisons to identify MWs not meeting specified standards; and to compare onsite constituent concentrations with site-specific background level constituent concentrations. Oneway ANOVA is used to perform inter-well comparisons and OLS regression and trend tests are often used to determine potential trends present in constituent concentrations identified in GW monitoring wells (MWs). Most of the methods described in this paragraph are available in the ProUCL 5.1 (ProUCL 5.0) software package. It is noted that not much guidance is available in the guidance documents cited above to compute rigorous UCLs, UPLs, and UTLs for moderately to highly skewed uncensored and left-censored data sets containing NDs with multiple DLs, a common occurrence in environmental data sets. Several parametric and nonparametric methods are available in the statistical literature (Singh, Singh, and Engelhardt 1997; Singh, Singh, and laci 2002; Krishnamoorthy et al. 2008; Singh, Maichle, and Lee, 2006) to compute UCLs and other upper limits which adjust for data skewness. During the years, as new methods became available to address statistical issues related to environmental projects, those methods were incorporated in ProUCL software so that environmental scientists and decision makers can make more accurate and informed decisions. Until 2006, not much guidance was provided on how to compute UCL95s of the mean and other upper limits (e.g., UPLs and UTLs) based upon data sets containing NDs with multiple DLs. For data sets with NDs, Singh, Maichle, and Lee (2006) conducted an extensive simulation study to compare the performances of the various estimation methods (in terms of bias in the mean estimate) and UCL computation methods (in terms of coverage provided by a UCL). They demonstrated that the nonparametric KM method performs well in terms of bias in estimates of mean. They also concluded that UCLs computed using the Student's t-statistic and percentile bootstrap method using the KM estimates do not provide the desired coverage to the population mean of skewed data sets. They demonstrated that depending upon sample size and data skewness, UCLs computed using KM estimates, the BCA bootstrap method (mildly skewed data sets), the bootstrap-t method, and the Chebyshev inequality (moderately to highly skewed data sets) provide better coverage (closer to the specified 95% coverage) to the population mean than other UCL computation methods. Based upon their findings, during 2006-2007, several UCL and other upper limits computation methods based upon KM and ROS estimates were incorporated in the ProUCL 4.0 software. It is noted that since the inclusion of the KM method in ProUCL 4.0 (2007), the 34 ------- use of the KM method based upper limits has become popular in many environmental applications to estimate EPC terms and BTVs. The KM method is also described in the latest version of the unified RCRA guidance document (U.S. EPA 2009). It is not easy to justify distributional assumptions of data sets consisting of both detects and NDs with multiple DLs. Therefore, based upon the published literature and experience, parametric UCL (and other upper limits) computation methods such as the MLE method (Cohen 1991) and the expectation maximization (EM) method (Gleit 1985) for normal and lognormal distributions were not included ProUCL 5.0 (and ProUCL 5.1) even though these methods were available in earlier versions of ProUCL. Additionally, the winsorization method (Gilbert 1987) available in an earlier version of ProUCL has also been excluded from ProUCL 5.0 (ProUCL 5.1) due to its poor performance. During 2015, some researchers (e.g., from New Mexico State University, Las Cruces, NM) suggested that the EM method performs better than some of the methods available in ProUCL 5.0, especially the gamma ROS (GROS) method; a method which can be used on left-censored data sets with multiple DLs. The literature has articles dealing with MLE and EM methods for data sets with a single censoring point (DL). Further research needs to be conducted on methods for computing reliable estimates of the mean, sd, and upper limits based upon parametric MLE and EM methods for data sets with NDs and multiple DLs. As always, it is the desire of the developers of ProUCL to incorporate the best available methods in ProUCL. The developers of ProUCL welcome/encourage other researchers to share their findings about the EM method showing that EM method performs better than methods already available in ProUCL 5.0/ProUCL 5.1 for data sets with single/multiple censoring points. The developers of ProUCL have been enhancing the ProUCL software with better performing methods as those methods become available. Efforts will be made to incorporate contributed code (with acknowledgement) for superior methods in future versions of ProUCL. ProUCL software is also used for teaching environmental statistics courses therefore, in addition to statistical and graphical methods routinely used to address statistical needs of environmental projects, some poor performing methods such as the substitution DL/2 method and Land's (1975) H-statistic based UCL computation method have been retained in ProUCL version 5.1 for research and comparison purposes. Methods incorporated in ProUCL 5.1 and in its earlier versions have been tested and verified extensively by the developers, researchers, scientists, and users. Specifically, the results obtained by ProUCL 5.1 are in agreement with the results obtained by using other software packages including Minitab, SAS®, and programs available in R-Script (not all methods are available in these software packages). Additionally, like ProUCL 5.0, ProUCL 5.1 outputs several intermediate results (e.g., khat and biased corrected kstar estimates of the gamma shape parameter, k, and critical values (e.g., tolerance factor, K, used to compute UTLs; critical value, d2max, used to compute USL) needed to compute decision statistics of interest, which may help interested users to verify statistical results computed by the ProUCL software. Whenever applicable, ProUCL provides warning messages and based upon professional experience and findings of simulation studies, makes suggestions to help a typical user in selecting the most appropriate decision statistic (e.g., UCL). Note: The availability of intermediate results and critical values can be used to compute lower limits and two-sided intervals which are not as yet available in the ProUCL software. For left-censored data sets, ProUCL 5.1 computes decision statistics (e.g., UCL, UPL, and UTL) based upon KM estimates computed in a straight forward manner without flipping the data and re-flipping the decision statistics; these operations are not easy for a typical user to understand and perform and can become quite tedious when multiple analytes need to be processed. Moreover, in environmental applications it is important to compute accurate estimates of sd which are needed to compute decision making statistics including UPLs and UTLs. Decision statistics (UPL, UTL) based upon a KM estimate 35 ------- of the of sd and computed using indirect methods can be different from the statistics computed using an estimate of sd obtained using the KM method directly, especially when one is dealing with a skewed data set or when using a log-transformation. These issues are elaborated by examples discussed in the accompanying ProUCL 5.1 Technical Guide. For uncensored data sets, researchers (e.g., Johnson 1978; Chen 1995; Efron and Tibshirani 1993; Hall [1988, 1992], and additional references found in Chapters 2 and 3) developed parametric (e.g., gamma distribution based) and nonparametric (bootstrap-t and Hall's bootstrap method, modified-t) methods for computation of decision statistics which adjust for data skewness. For uncensored positively skewed data sets, Singh, Singh, and laci (2002) and Singh and Singh (2003) performed simulation experiments to compare the performances (in terms of coverage probabilities) of the various UCL computation methods described in the literature. They demonstrated that for skewed data sets, UCLs based upon Student's t statistic, central limit theorem (CLT), and percentile bootstrap method tend to underestimate the population mean (EPC). It is reasonable to state that the findings of the simulation studies performed on uncensored skewed data sets comparing the performances of the various UCL computation methods can be extended to skewed left-censored data sets. Based upon the findings of those studies performed on uncensored data sets and also using the findings summarized in Singh, Maichle, and Lee (2006), it was concluded that t-statistic, CLT, and the percentile bootstrap method based UCLs computed using KM estimates (and also ROS estimates) underestimate the population mean of moderately skewed to highly skewed data sets. Interested users may want to verify these statements by performing simulation experiments or other forms of rigorous testing. Like uncensored skewed data sets, for left-censored data sets, ProUCL 5.1 offers several parametric and nonparametric methods for computing UCLs and other limits which adjust for data skewness. Due to the lack of research and methods, in earlier versions of the ProUCL software (e.g., ProUCL 4.00.02, ProUCL 4.0), KM estimates were used in the normal distribution based equations for computing the various upper limits for left-censored data sets. However, normal distribution based upper limits (e.g., t-UCL) using KM estimates (or any other estimates such as ROS estimates) fail to provide the specified coverage (e.g., 0.95) of the parameters (e.g., mean, percentiles) of populations with skewed distributions (Singh, Singh, laci 2002; Johnson 1978; Chen 1995). For skewed data sets, ProUCL 5.0/ProUCL 5.1 computes UCLs applying KM estimates in UCL equations for skewed data sets (e.g., gamma and lognormal); therefore, some changes have been made in the decision tables of ProUCL 5.0/ProUCL 5.1 for computing UCL95s. Also, the nonparametric UCL computation methods (e.g., percentile bootstrap) do not provide the desired coverage to the population means of skewed distributions (e.g., Hall [1988, 1992], Efron and Tibshirani, 1993). For example, the use of t-UCL or the percentile bootstrap UCL method on robust ROS estimates or on KM estimates underestimates the population mean for moderately skewed to highly skewed data sets. Chapters 3 and 5 of the ProUCL Technical Guide describe parametric and nonparametric KM methods for computing upper limits (and available in ProUCL 5.0/ ProUCL 5.1) which adjust for data skewness. The KM method yields good estimates of the population mean and std (Singh, Maichle, and Lee 2006); however upper limits computed using the KM or ROS estimates in normal equations or in the percentile bootstrap method do not account for skewness present in the data set. Appropriate UCL computation methods which account for data skewness should be used on KM or ROS estimates. For left-censored data sets, ProUCL 5.0/ProUCL 5.1 compute upper limits using KM estimates in gamma (lognormal) UCL, UPL, and UTL equations (e.g., also suggested in U.S. EPA 2009) provided the detected observations in the left-censored data set follow a gamma (lognormal) distribution. Recently, the use of the ISM methodology has been recommended (ITRC 2012) for collecting soil samples with the purpose of estimating mean concentrations of DUs requiring analysis of human and 36 ------- ecological risk and exposure. ProUCL can be used to compute UCLs based upon ISM data as described and recommended in the ITRC ISM Technical and Regulatory Guide (2012). At many sites, large amounts of discrete background data are already available which are not directly comparable to the actual field ISM data (onsite or background). To compare the existing discrete background data with field ISM data, the BISS module (blocked for general use in ProUCL version 5.1 awaiting guidance and instructions for its intended use) of ProUCL 5.1 can be used on a large (e.g., consisting of at least 30 observations) existing discrete background data set. The BISS module simulates the incremental sampling methodology based equivalent incremental background samples; and each simulated BISS sample represents an estimate of the mean of the population represented by the discrete background data set. The availability of a large discrete background data set collected from areas with geological conditions comparable to the DU(s) of interest (onsite DUs) is a requirement for successful application of this module. The user cannot see the simulated BISS data; however, the simulated BISS data can be accessed by other modules of ProUCL 5.0 (ProUCL 5.1) for performing desired statistical evaluations. For example, the simulated BISS data can be merged with the actual field ISM background data after comparing the two data sets using a two-sample t-test. The actual field ISM or the merged ISM and BISS data can be accessed by modules of ProUCL to compute a UCL of the mean or a UTL. ProUCL 5.1 Capabilities Assumptions: Like most statistical methods, statistical methods for computing upper limits (e.g., UCLs, UPLs, UTLs) are also based upon certain assumptions including the availability of a randomly collected data set consisting of independently and identically distributed (i.i.d) observations representing the population (e.g., site area, reference area) under investigation. A UCL of the mean (of a population) and BTV estimates (UPL, UTL) should be computed using a randomly collected (simple random or systematic random) data set representing a single statistical population (e.g., site population or background population). When multiple populations (e.g., background and site data mixed together) are present in a data set, the recommendation is to separate them first by using the population partitioning techniques (e.g., Singh, Singh, and Flatman 1994) prior to computing the appropriate decision statistics (e.g., 95% UCLs). Regardless of how the populations are separated, decision statistics should be computed separately for each identified population. The topic of population partitioning and the extraction of a valid site-specific background data set from a broader mixture data set potentially consisting of both onsite and offsite data are beyond the scope of ProUCL 5.0/ProUCL 5.1. Parametric estimation and hypotheses testing methods (e.g., t-test, UCLs, UTLs) are based upon distributional (e.g., normal distribution, gamma) assumptions. ProUCL includes GOF tests for determining if a data set follows a normal, a gamma, or a lognormal distribution. Multiple Constituents/Variables: Environmental scientists need to evaluate many constituents in their decision making processes including exposure and risk assessment, background evaluations, and site versus background comparisons. ProUCL can process multiple constituents/variables simultaneously in a user-friendly manner; an option not available in other freeware or commercial software packages such as NADA for R (Helsel 2013). This option is very useful when one has to process many variables/analytes and compute decision statistics (e.g., UCLs, UPLs, and UTLs) and/or test statistics (e.g., ANOVA test, trend test) for those variables/analytes. Analysis by a Group Variable: ProUCL also has the capability of processing data by groups. A valid group column should be included in the data file. The analyses of data categorized by a group ID variable such as: 1) Surface versus (vs.) Subsurface; 2) AOC1 vs. AOC2; 3) Site vs. Background; and 4) Upgradient vs. Downgradient MWs are common in many environmental applications. ProUCL offers this option for data sets with and without nondetects. The Group option provides a way to perform statistical 37 ------- tests and methods including graphical displays separately for each of the group (samples from different populations) that may be present in a data set. For example, the same data set may consist of analytical data from multiple groups or populations representing site, background, two or more AOCs, surface soil, subsurface soil, and GW. By using this option, the graphical displays (e.g., box plots, Q-Q plots, histograms) and statistics (including computation of background statistics, UCLs, ANOVA test, trend test and OLS regression statistics) can be easily computed separately for each group in the data set. Exploratory Graphical Displays for Uncensored and Left-Censored Data Sets: Graphical methods included in the Graphs module of ProUCL include: Q-Q plots (data in same column), multiple Q-Q plots (data in different columns), box plots, multiple box plots (data in different columns), and histograms. These graphs can also be generated for data sets containing ND observations. Additionally, the OLS Regression and Trend Analysis module can be used to generate graphs displaying parametric OLS regression lines with confidence and prediction intervals around the regression and nonparametric Theil- Sen trend lines. The Trend Analysis module can generate trend graphs for data sets without a sampling event variable, and also generates time series graphs for data sets with a sampling event (time) variable. Like ProUCL 5.0, ProUCL 5.1 accepts only numerical values for the event variable. Graphical displays of a data set are useful for gaining added insight regarding a data set that may not otherwise be clear by looking at test statistics such as T-S test or MK statistics. Unlike test statistics (e.g., t-test, MK test, AD test) and decision statistics (e.g., UCL, UTL), graphical displays do not get influenced by outliers and ND observations. It is suggested that the final decisions be made based upon statistical results as well as graphical displays. Side-by-side box plots or multiple Q-Q plots are useful to graphically compare concentrations of two or more groups (e.g., several monitoring wells). The GOF module of ProUCL generates Q-Q plots for normal, gamma, and lognormal distributions based upon uncensored as well as left-censored data sets with NDs. All relevant information such as the test statistics, critical values and probability-values (p- values), when available are also displayed on the GOF Q-Q plots. In addition to providing information about the data distribution, a normal Q-Q plot in the original raw scale also helps to identify outliers and multiple populations that may be present in a data set. On a Q-Q plot, observations well-separated from the majority of the data may represent potential outliers coming from a population different from the main dominant population (e.g., background population). In a Q-Q plot, jumps and breaks of significant magnitude suggest the presence of observations coming from multiple populations (onsite and offsite areas). ProUCL can also be used to display box plots with horizontal lines displayed/superimposed at pre-specified compliance limits (CLs) or computed upper limits (e.g., UPL, UTL). This kind of graph provides a visual comparison of site data with compliance limits and/or BTV estimates. Outlier Tests: ProUCL also provides a couple of classical outlier test procedures (EPA 2006b, 2009), the Dixon test and the Rosner test. The details of these outlier tests are described in Chapter 7. These outlier tests often suffer from "masking effects" in the presence of multiple outliers. It is suggested that the classical outlier procedures should always be accompanied by graphical displays including box plots and Q-Q plots. Description and use of the robust and masking-resistant outlier procedures (Rousseeuw and Leroy 1987; Singh and Nocerino 1995) are beyond the scope of ProUCL 5.1. Interested users are encouraged to try the Scout 2008 software package (EPA 2009d) for robust outlier identification methods especially when dealing with multivariate data sets consisting of observations for several variables/analytes/constituents. Outliers represent observations coming from populations different from the main dominant population represented by the majority of the data set. Outliers distort most statistics (e.g., mean, UCLs, UPLs, test statistics) of interest. Therefore, it is desirable to compute decisions statistics based upon data sets 38 ------- representing the main population and not to compute distorted statistics by accommodating a few low probability outliers (e.g., by using a lognormal distribution). Moreover, it should be noted that even though outliers might have minimal influence on hypotheses testing statistics based upon ranks (e.g., WMW test), outliers do distort several nonparametric statistics including bootstrap methods such as bootstrap-t and Hall's bootstrap UCLs and other nonparametric UPLs and UTLs computed using higher order statistics. Goodness-of-Fit Tests: In addition to computing simple summary statistics for data sets with and without NDs, ProUCL 5.1 includes GOF tests for normal, lognormal and gamma distributions. To test for normality (lognormality) of a data set, ProUCL includes the Lilliefors test and the extended S-W test for samples of sizes up to 2000 (Royston 1982, 1982a). For the gamma distribution, two GOF tests: the A-D test (Anderson and Darling 1954) and K-S test (Schneider 1978) are available in ProUCL. For samples of larger sizes (e.g., with n > 100) and small values of the gamma shape parameter, k (e.g., k < 0.1), significant discrepancies were found in the critical values of the two gamma GOF test statistics (A-D and K-S tests) obtained using the two gamma deviate generation algorithms: Whitaker (1974) and Marsaglia and Tsang (2000). In ProUCL 5.0 (and ProUCL 5.1), for values of k < 0.2, the critical values of the two gamma GOF tests: A-D and K-S tests have been updated using the currently available more efficient gamma deviate generation algorithm due to Marsaglia and Tsang's (2000); more details about the implementation of their algorithm can be found in Kroese, Taimre, and Botev (2011). For these two GOF and values of the shape parameter, k=0.025, 0.05, 0.1, and 0.2, critical value tables have been updated by incorporating the newly generated critical values for three levels of significance: 0.05, 0.1, and 0.01. The updated tables are provided in Appendix A of the ProUCL Technical Guide. It was noted that for k=0.2, the older (generated in 2002) and the newly generated critical values are in general agreement; therefore, critical values for k=0.2 were not replaced in tables summarized in Appendix A. ProUCL also generates GOF Q-Q plots for normal, lognormal, and gamma distributions displaying all relevant statistics including GOF test statistics. GOF tests for data sets with and without NDs are described in Chapters 2 and 3 of the ProUCL Technical Guide. For data sets containing NDs, it is not easy to verify the distributional assumptions correctly, especially when the data set consists of a large percentage of NDs with multiple DLs and NDs exceeding some detected values. Historically, decisions about distributions of data sets with NDs are based upon GOF test statistics computed using the data obtained: without NDs; replacing NDs by 0, DL, or DL/2; using imputed NDs based upon a ROS (e.g., lognormal ROS) method. For data sets with NDs, ProUCL 5.1 can perform GOF tests using the methods listed above. ProUCL 5.1 can also generate censored probability plots (Q-Q plots) which are very similar to Q-Q plots generated using detected data. Using the Imputed NDs using ROS Methods option of the Stats/Sample Sizes module of ProUCL 5.0, additional columns can be generated for storing imputed (estimated) values for NDs based upon normal ROS, gamma ROS, and lognormal ROS (also known as robust ROS) methods. Sample Size Determination and Power Evaluation: The Sample Sizes module in ProUCL can be used to develop DQO-based sampling designs needed to address statistical issues associated with environmental projects. ProUCL 5.1 provides user-friendly options for entering the desired/pre-specified values for decision parameters (e.g., Type I and Type II error rates) and other DQOs used to determine minimum sample sizes for statistical applications including: estimation of the mean, single and two-sample hypothesis testing approaches, and acceptance sampling for discrete items (e.g., drums containing hazardous waste). Both parametric (e.g., t-test) and nonparametric (e.g., Sign test, WRS test) sample size determination methods as described in EPA (2000, 2002c, 2006a, 2006b) guidance documents are available in ProUCL 5.1. ProUCL also has the sample size determination option for acceptance sampling of lots of discrete objects such as a lot (batch, set) of drums containing hazardous waste (e.g., RCRA 39 ------- applications, EPA 2002c). When the sample size for an application (e.g., verification of cleanup level) is not computed using the DQOs-based sampling design process, the Sample Size module can be used to assess the power of the test statistic used in retrospect. The mathematical details of the Sample Sizes module are given in Chapter 8 of the ProUCL Technical Guide. Bootstrap Methods: Bootstrap methods are computer intensive nonparametric methods which can be used to compute decision statistics of interest when a data set does not follow a known distribution, or when it is difficult to analytically derive the distributions of statistics of interest. It is well-known that for moderately skewed to highly skewed data sets, UCLs based upon standard bootstrap and the percentile bootstrap methods do not perform well (e.g., Efron [1981, 1982]; Efron and Tibshirani 1993; Hall [1988,1992]; Singh, Singh, and laci 2002; Singh and Singh 2003, Singh, Maichle and Lee 2006) as the interval estimates based upon these bootstrap methods fail to provide the specified coverage to the population mean (e.g., UCL95 does not provide adequate 95% coverage of population mean). For skewed data sets, Efron and Tibshirani (1993) and Hall (1988, 1992) considered other bootstrap methods such as the BCA, bootstrap-t and Hall's bootstrap methods. For skewed data sets, bootstrap-t and Hall's bootstrap (meant to adjust for skewness) methods perform better (e.g., in terms of coverage for the population mean) than the other bootstrap methods. However, it has been noted (e.g., Efron and Tibshirani 1993, Singh, Singh, and laci 2002) that these two bootstrap methods tend to yield erratic and inflated UCL values (orders of magnitude higher than other UCLs) in the presence of outliers. Similar behavior of the bootstrap-t UCL and Hall's bootstrap UCL methods is observed for data sets consisting of NDs and outliers. For nonparametric uncensored and left-censored data sets with NDs, depending upon data variability and skewness, ProUCL recommends the use of BCA bootstrap, bootstrap-t, or Chebyshev inequality based methods for computing decision statistics. Due to the reasons described above, whenever applicable, ProUCL 5.0/ProUCL 5.1 provides cautionary notes and warning messages regarding the use of bootstrap-t and Halls bootstrap UCL methods. Hypotheses Testing Approaches: ProUCL software has both single-sample (e.g., Student's t-test, sign test, proportion test, WSR test) and two-sample (Student's t-test, WMW test, Gehan test, and T-W test) parametric and nonparametric hypotheses testing approaches. Hypotheses testing approaches in ProUCL can handle both full-uncensored data sets and left-censored data sets with NDs. Most of the hypotheses tests also report associated p-values. For some hypotheses tests (e.g., WMW test, WSR test, proportion test), large sample /"-values based upon the normal approximation are computed using continuity correction factors. The mathematical details of the various single-sample and two-sample hypotheses testing approaches are described in Chapter 6 the ProUCL Technical Guide. • Single-Sample Tests: Parametric (Student's t-test) and nonparametric (Sign test, WSR test, tests for proportions and percentiles) hypotheses testing approaches are available in ProUCL. Single-sample hypotheses tests are used when environmental parameters such as the cleanup standard, action level, or compliance limits are known, and the objective is to compare site concentrations with those known threshold values. A t-test (or a sign test) may be used to verify the attainment of cleanup levels in an AOC after a remediation activity has taken place or a test for proportion may be used to verify if the proportion of exceedances of an action level (Ao or a CL) by sample observations collected from an AOC (or a MW) exceeds a certain specified proportion (e.g., 1%, 5%, 10%). The differences between these tests should be noted and understood. A t-test or a Wilcoxon Signed Rank (WSR) test are used to compare the measures of location and central tendencies (e.g., mean, median) of a site area (e.g., AOC) to a cleanup standard, Cs, or action level also representing a measure of central tendency (e.g., mean, median); whereas, a proportion test determines if the proportion of site observations from an AOC exceeding a compliance limit (CL) exceeds a specified 40 ------- proportion, Po (e.g., 5%, 10%). The percentile test compares a specified percentile (e.g., 95th) of the site data to a pre-specified upper threshold (e.g., action level). • Two-Sample Tests: Hypotheses tests (Student's t-test, WMW test, Gehan test, T-W test) are used to perform site versus background comparisons, compare concentrations of two or more AOCs, or to compare concentrations of GW collected from MWs. As cited in the literature, some of the hypotheses testing approaches (e.g., nonparametric two-sample WMW) deal with a single detection limit scenario. When using the WMW test on a data set with multiple detection limits, all observations (detects and NDs) below the largest detection limit need to be considered as NDs (Gilbert 1987). This in turn tends to reduce the power and increase uncertainty associated with test. As mentioned before, it is always desirable to supplement the test statistics and conclusions with graphical displays such as multiple Q-Q plots and side-by-side box plots. The Gehan test or T-We (new in ProUCL 5.1) should be used in cases where multiple detection limits are present. Note about Quantile Test: For smaller data sets, the Quantile test as described in U.S. EPA documents (U.S. EPA [1994, 2006b]; Hollander and Wolfe, 1999) is available in ProUCL 4.1 (see ProUCL 4.1 Technical Guide). In the past, some users incorrectly used this test for larger data sets. Due to lack of resources, this test has not been expanded for data sets of all sizes. Therefore, to avoid confusion and its misuse for larger data sets, the Quantile test was not included in ProUCL 5.0 and ProUCL 5.1. Computation of Upper Limits including UCLs. UPLs. UTLs. and USLs: ProUCL software has parametric and nonparametric methods including bootstrap and Chebyshev inequality based methods to compute decision making statistics such as UCLs of the mean (EPA 2002a), percentiles, UPLs for future k (>1) observations, UTLs (U.S. EPA [1992b and 2009]) and upper simultaneous limits (USLs) (Singh and Nocerino [1995, 2002]) based upon uncensored full data sets and left-censored data sets containing NDs with multiple DLs. Methods incorporated in ProUCL cover a wide range of skewed data distributions with and without NDs. In addition to normal and lognormal distributions based upper limits, ProUCL 5.0 can compute parametric UCLs, percentiles, UPLs for future k (>1) observations, UTLs, and USLs based upon gamma distributed data sets. For data sets with NDs, ProUCL has several estimation methods including the Kaplan-Meier (KM) method (1958), ROS methods (Helsel 2005) and substitution methods such as replacing NDs with the DL or DL/2 (Gilbert 1987; U.S. EPA 2006b). Substitution method and other poor performing methods (e.g., H-UCL for lognormal distribution) have been retained, as requested by U.S. EPA scientists, in ProUCL 5.0/ProUCL 5.1 for research and comparison purposes. One may not interpret the availability of these poor performing methods in ProUCL as recommended methods by ProUCL or by the U.SEP A for computing decision statistics. Computation of UCLs Based upon Uncensored Data Sets without NDs: Parametric UCL computation methods in ProUCL for uncensored data sets include: Student's t-UCL, Approximate gamma UCL (using chi-square approximation), Adjusted gamma UCL (adjusted for level significance), Land's H-UCL, and Chebyshev inequality-based UCL (using minimum variance unbiased estimates (MVUEs) of parameters of a lognormal distribution). Nonparametric UCL computation methods for data sets without NDs include: CLT-based UCL, Modified-t-statistic-based UCL (adjusted for skewness), Adjusted-CLT-based UCL (adjusted for skewness), Chebyshev inequality-based UCL (using sample mean and standard deviation), Jackknife method-based UCL, UCL based upon standard bootstrap, UCL based upon percentile bootstrap, UCL based upon BCA bootstrap, UCL based upon bootstrap-t, and UCL based upon Hall's bootstrap method. The details of UCL computation methods for uncensored data sets are summarized in Chapter 2 of the ProUCL Technical Guide. 41 ------- Computations of UPLs, UTLs. and USLs Based upon Uncensored Data Sets without NDs: For uncensored data sets without NDs, ProUCL can compute parametric percentiles, UPLs for k (k>l) future observations, UPLs for mean of k (>1) future observations, UTLs, and USLs based upon the normal, gamma, and lognormal distributions. Nonparametric upper limits are typically based upon order statistics of a data set. Depending upon the size of the data set, the higher order statistics (maximum, second largest, third largest, and so on) are used to compute these upper limits (e.g., UTLs). Depending upon the sample size, specified CC and coverage probability, ProUCL 5.1 outputs the actual CC achieved by a nonparametric UTL. The details of the parametric and nonparametric computation methods for UPLs, UTLs, and USLs are described in Chapter 3 of the ProUCL Technical Guide. Computation of UCLs. UPLs. UTLs. and USLs Based upon Left-Censored Data Sets with NDs: For data sets with NDs, ProUCL computes UCLs, UPLs, UTLs, and USLs based upon the mean and sd computed using lognormal ROS (LROS, robust ROS), Gamma ROS (GROS), KM, and DL/2 substitution methods. To adjust for skewness in non-normally distributed data sets, ProUCL uses bootstrap methods and Chebyshev inequality when computing UCLs and other limits using estimates of the mean and sd obtained using the methods (details in Chapters 4 and 5) listed above. ProUCL 5.1 (new in ProUCL 5.0) uses parametric methods on KM (and ROS) estimates, provided detected observations in the left-censored data set follow a parametric distribution. For example, if the detected data follow a gamma distribution, ProUCL uses KM estimates in gamma distribution-based equations when computing UCLs, UTLs, and other upper limits. When detected data do not follow a discernible distribution, depending upon size and skewness of detected data, ProUCL recommends the use of Kaplan-Meier (1958) estimates in bootstrap methods and the Chebyshev inequality for computing nonparametric decision statistics (e.g., UCL95, UPL, UTL) of interest. ProUCL computes KM estimates directly using left-censored data sets without flipping data and requiring re-flipping of decision statistics. The KM method incorporated in ProUCL computes both sd and standard error (SE) of the mean. As mentioned earlier, for historical reasons and for comparison and research purposes, the DL/2 substitution method and H-UCL based upon LROS method have been retained in ProUCL 5.0/ProUCL 5.1. The inclusion of the substitution and LROS methods in ProUCL should not be inferred as an endorsement of those methods by ProUCL software and its developers. The details of the UCL computation methods for data sets with NDs are given in Chapter 4 and the detail description of the various other upper limits: UPLs, UTLs, and USLs for data sets with NDs are given in Chapter 5 of the ProUCL Technical Guide. Oneway ANOVA. OLS Regression and Trend Analysis: The Oneway ANOVA module has both classical and nonparametric K-W ANOVA tests as described in EPA guidance documents (e.g., EPA [2006b, 2009]). Oneway ANOVA is used to compare means (or medians) of multiple groups such as comparing mean concentrations of several areas of concern or performing inter-well comparisons of COPC concentrations at several MWs. The OLS Regression option computes the classical OLS regression line and generates graphs displaying the OLS line, confidence bands and prediction bands around the regression line. All statistics of interest including slope, intercept, and correlation coefficient are displayed on the OLS line graph. The Trend Analysis module has two nonparametric trend tests: the M-K trend test and T-S trend test. Using this option, one can generate trend graphs and time-series graphs displaying a T-S trend line and all other statistics of interest with associated />-values. In addition to slope and intercept, the T-S test in ProUCL 5.1 computes and outputs residuals based upon the computed nonparametric T-S line. In GW monitoring applications, OLS regression, trend tests, and time series plots are often used to identify trends (e.g., upwards, downwards) in constituent concentrations of GW monitoring wells over a certain period of time (U.S. EPA 2009). The details of Oneway ANOVA are given in Chapter 9 and OLS regression line and Trend tests methods are described in Chapter 10 of the ProUCL Technical Guide. 42 ------- BISS Module: At many sites, a large amount of discrete onsite and background data are already available which are not directly comparable to actual field ISM data. In order to provide a tool to compare the existing discrete data with ISM data, the BISS module of ProUCL 5.0 may be used on a large existing discrete data set. The ISM methodology used to develop the BISS module is a relatively new approach; methods incorporated in this BISS module require further investigation. For now, the BISS module has been blocked for use in ProUCL 5.0/ProUCL 5.1 as this module is awaiting adequate guidance for its intended use on discrete background data sets. Recommendations and Suggestions in ProUCL: Until 2006, not much guidance was available on how to compute a UCL95 of the mean and other upper limits (e.g., UPLs and UTLs) for skewed left-censored data sets containing NDs with multiple DLs, a common occurrence in environmental data sets. For uncensored positively skewed data sets, Singh, Singh, and laci (2002) and Singh and Singh (2003) performed extensive simulation experiments to compare the performances (in terms of coverage probabilities) of several UCL computation methods described in the statistical and environmental literature. They noted that the optimal choice of a decision statistic (e.g., UCL95) depends upon the sample size, data distribution and data skewness. They incorporated the results of their findings in ProUCL 3.1 and higher versions to select the most appropriate UCL to estimate the EPC term. For data sets with NDs, Singh, Maichle, and Lee (2006) conducted a similar simulation study to compare the performances of the various estimation methods (in terms of bias in the mean estimate); and some UCL computation methods (in terms of coverage provided by a UCL). They demonstrated that the KM estimation method performs well in terms of bias in estimates of the mean; and for skewed data sets, the t- statistic, CLT, and the percentile bootstrap method based UCLs computed using KM estimates (and ROS estimates) underestimate the population mean. From these findings summarized in Singh, Singh, and laci (2002) and Singh, Maichle, and Lee (2006), it is natural to state and assume the findings of the simulation studies performed on uncensored skewed data sets comparing performances of the various UCL computation methods can be extended to skewed left-censored data sets. Like uncensored data sets without NDs, for data sets with NDs, there is no one single best UCL (and other upper limits such as UTL, UPL) which can be used to estimate an EPC (and background threshold values) for all data sets of varying sizes, distribution, and skewness. The optimal choice of a decision statistic depends upon the size, distribution, and skewness of detected observations. For data sets with and without NDs, ProUCL computes decision statistics including UCLs, UPLs, and UTLs using several parametric and nonparametric methods covering a wide-range of sample size, data variability and skewness. Using the results and findings summarized in the literature cited above, and based upon the sample size, data distribution, and data skewness, modules of ProUCL make suggestions about using the most appropriate decision statistic(s) to estimate population parameter(s) of interest (e.g., EPC). The suggestions made in ProUCL are based upon the extensive professional applied and theoretical experience of the developers in environmental statistical methods, published literature, results of simulation studies conducted by the developers of ProUCL and procedures described in many U.S. EPA guidance documents. These suggestions are made to help the users in selecting the most appropriate UCL to estimate an EPC which is routinely used in exposure assessment and risk management studies of the U.S. EPA. It should be pointed out that a typical simulation study cannot cover all data sets of various sizes and skewness from all types of distributions. For an analyte (data set) with skewness (sd of logged data) near the end points of the skewness intervals described in decision tables of Chapter 2 (e.g., Tables 2-9 through 2-11) of the ProUCL Technical Guide, the user/project team may select the most appropriate UCL based upon the site CSM, expert site knowledge, toxicity of the analyte, and exposure risks associated with that analyte. The project team should make the final decision regarding using or not using 43 ------- the suggestions/recommendations made by ProUCL. If deemed necessary, the project team may want to consult a statistician. Even though, ProUCL software has been developed using limited government funding, ProUCL 5.1 provides many statistical and graphical methods described in U.S. EPA documents for data sets with and without NDs. However, one may not compare the availability of methods in ProUCL 5.1 with methods available in the commercial software packages such as SAS® and Minitab 16. For example, trend tests correcting for seasonal/spatial variations and geostatistical methods are not available in the ProUCL software. For those methods, the user is referred to commercial software packages such as SAS®. As mentioned earlier, is the developers of ProUCL recommended supplementing test results (e.g., two- sample test) with graphical displays (e.g., Q-Q plots, side-by-side box plots) especially when data sets contain NDs and outliers. With the inclusion of the BISS, Oneway ANOVA, OLS Regression Trend and the user-friendly DQOs based Sample Size modules, ProUCL represents a comprehensive software package equipped with statistical methods and graphical tools needed to address many environmental sampling and statistical needs as described in the various CERCLA (U.S. EPA 1989a, 1992a, 2002a, 2002b, 2006a, 2006b), MARSSIM (U.S. EPA 2000), and RCRA (U.S. EPA 1989b, 1992b, 2002c, 2009) guidance documents. Finally, the users of ProUCL are cautioned about the use of methods and suggestions described in some recent environmental literature. For example, many decision statistics (e.g., UCLs, UPLs, UTLs,) computed using the methods (e.g., percentile bootstrap, statistics using KM estimates and t-critical values) described in Helsel (2005, 2012) will fail to provide the desired coverage for environmental parameters of interest (mean, upper percentile) of moderately skewed to highly skewed populations and conclusions derived based upon those decisions statistics may lead to incorrect conclusions which may not be cost-effective or protective of human health and the environment. Note about ProUCL 5.1: ProUCL 5.1 represents an upgrade of ProUCL 5.0 to address deficiencies identified in ProUCL 5.0. ProUCL 5.1 retains all capabilities of ProUCL 5.0 as described above. All modules in ProUCL 5.1, and their look and feel is the same as in ProUCL 5.0. In this document, any statement made about the capabilities of ProUCL 5.0 also apply to ProUCL version 5.1; and to save time, not all screen shots used in ProUCL 5.0 manuals have been replaced in the ProUCL 5.1 User Guide and Technical Guide. Upgrades in ProUCL 5.1 (not available in earlier versions) have been labeled as New in ProUCL 5.1 in this document. ProUCL 5.1 Technical Guide In addition to this User Guide, a Technical Guide also accompanies the ProUCL 5.1 software, providing details of using the statistical and graphical methods incorporated in ProUCL 5.1. Most of the mathematical algorithms and formulae (with references) used in the development of ProUCL 5.1 are described in the associated Technical Guide. 44 ------- Chapter 1 Guidance on the Use of Statistical Methods in ProUCL Software Decisions based upon statistics computed using discrete data sets of small sizes (e.g., < 6) cannot be considered reliable enough to make decisions that affect human health and the environment. For example, a background data set of size < 6 is not large enough to characterize a background population, compute BTV estimates, or to perform background versus site comparisons. Several U.S. EPA guidance documents (e.g., EPA 2000, 2006a, 2006b) detail DQOs and minimum sample size requirements needed to address statistical issues associated with different environmental applications. In order to obtain reliable statistical results, an adequate amount of data should be collected using project-specified DQOs (i.e., CC, decision error rates). The Sample Sizes module of ProUCL computes minimum sample sizes based on DQOs specified by the user and described in many guidance documents. In some cases, it may not be possible (e.g., due to resource constraints) to collect the calculated number of samples needed to meet the project-specific DQOs. Under these circumstances one can use the Sample Sizes module to assess the power of the test statistic resulting from the reduced number of samples which were collected. Based upon professional experience, the developers of ProUCL 4 software and its later versions have been making some rule-of-thumb suggestions regarding minimum sample size requirements needed to perform statistical evaluations such as: estimation of environmental parameters of interest (i.e., EPCs and BTVs), comparing site data with background data or with some pre-established screening levels (e.g., action levels [ALs], compliance limits [CLs]). Those rule-of thumb suggestions are described later in Section 1.7 of this chapter. It is noted that those minimum sample requirements have been adopted by some other guidance documents including the RCRA Guidance Document (EPA 2009). This chapter also describes the differences between the various statistical upper limits including upper confidence limits (UCLs) of the mean, upper prediction limits (UPLs) for future observations, and upper tolerance intervals (UTLs) often used to estimate the environmental parameters of interest including EPC terms and BTVs. The use of a statistical method depends upon the environmental parameter(s) being estimated or compared. The measures of central tendency (e.g., means, medians, or their UCLs) are used to compare site mean concentrations with a cleanup standard, Cs, also representing some central tendency measure of a reference area or some other known threshold representing a measure of central tendency. The upper threshold values, such as the CLs, alternative concentration limits (ACL), or not-to-exceed values, are used when individual point-by-point observations are compared with those threshold values. Depending upon whether the environmental parameters (e.g., BTVs, not-to-exceed value, or EPC term) are known or unknown, different statistical methods with different data requirements are needed to compare site concentrations with pre-established (known) or estimated (unknown) standards and BTVs. Several upper limits, and single and two sample hypotheses testing approaches, for both full-uncensored and left-censored data sets are available in the ProUCL software package for performing the comparisons described above. 1.1 Background Data Sets Based upon the CSM and regional and expert knowledge about the site, the project team selects background or reference areas. Depending upon the site activities and the pollutants, the background area can be site-specific or a general reference area with conditions comparable to the site before contamination due to site related activities. An appropriate random sample of independent observations 45 ------- (i.i.d) should be collected from the background area. A defensible background data set represents a "single" environmental population possibly without any outliers. In a background data set, in addition to reporting and/or laboratory errors, statistical outliers may also be present. A few elevated statistical outliers present in a background data set may actually represent potentially contaminated locations belonging to an impacted site area and/or possibly from other sources; those elevated outliers may not be coming from the background population under evaluation. Since the presence of outliers in a data set tends to yield distorted (poor and misleading) values of the decision making statistics (e.g., UCLs, UPLs and UTLs), elevated outliers should not be included in background data sets and estimation of BTVs. The objective here is to compute background statistics based upon a data set which represents the main background population, and does not accommodate the few low probability high outliers (e.g., coming from extreme tails of the data distribution) that may also be present in the sampled data. The occurrence of elevated outliers is common when background samples are collected from various onsite areas (e.g., large Federal Facilities). The proper disposition of outliers, to include or not include them in statistical computations, should be decided by the project team. The project team may want to compute decision statistics with and without the outliers to evaluate the influence of outliers on the decision making statistics. A couple of classical outlier tests (Dixon and Rosner tests) are available in ProUCL. Since both of these classical tests suffer from masking effects (e.g., some extreme outliers may mask the occurrence of other intermediate outliers), it is suggested that these classical outlier tests be supplemented with graphical displays such as a box plot and a Q-Q plot on a raw scale. The use of exploratory graphical displays helps in determining the number of outliers potentially present in a data set. The use of graphical displays also helps in identifying extreme high outliers as well as intermediate and mild outliers. The use of robust and masking-resistant outlier identification procedures (Singh and Nocerino, 1995, Rousseeuw and Leroy, 1987) is recommended when multiple outliers are present in a data set. Those methods are beyond the scope of ProUCL 5.1. However, several robust outlier identification methods are available in the Scout 2008 version 1.0 software package (EPA 2009d, An appropriate background data set of a reasonable size (preferably computed using the DQOs processes) is needed for the data set to be representative of background conditions and to compute upper limits (e.g., estimates of BTVs) and compare site and background data sets using hypotheses testing approaches. A background data set should have a minimum of 10 observations, however more observations is preferable. 1.2 Site Data Sets A data set collected from a site population (e.g., AOC, exposure area [EA], DU, group of MWs) should be representative of the population under investigation. Depending upon the areas under investigation, different soil depths and soil types may be considered as representing different statistical populations. In such cases, background versus site comparisons may have to be conducted separately for each of those sub-populations (e.g., surface and sub-surface layers of an AOC, clay and sandy site areas). These issues, such as comparing depths and soil types, should also be considered in the planning stages when developing sampling designs. Specifically, the availability of an adequate amount of representative data is required from each of those site sub-populations/strata defined by sample depths, soil types, and other characteristics. Site data collection requirements depend upon the objective(s) of the study. Specifically, in background versus site comparisons, site data are needed to perform: 46 ------- point-by-point onsite comparisons with pre-established ALs or estimated BTVs. Typically, this approach is used when only a small number (e.g., < 6) of onsite observations are compared with a BTV or some other not-to-exceed value. If many onsite values need to be compared with a BTV, the recommended upper limit to use is the UTL or upper simultaneous limit (USL) to control the false positive error rate (Type I Error Rate). More details can be found in Chapter 3 of the Technical Guide. Alternatively, one can use hypothesis testing approaches (Chapter 6 of ProUCL Technical Guide) provided enough observations (at least 10, more are preferred) are available. single-sample hypotheses tests to compare site data with a pre-established cleanup standards, Cs (e.g., representing a measure of central tendency); proportion test to compare site proportion of exceedances of an AL with a pre-specified allowable proportion, Po. These hypotheses testing approaches are used on site data when enough site observations are available. Specifically, when at least 10 (more are desirable) site observations are available; it is preferable to use hypotheses testing approaches to compare site observations with specified threshold values. The use of hypotheses testing approaches can control both types of error rates (Type 1 and Type 2) more efficiently than the point-by-point individual observation comparisons. This is especially true as the number of point-by-point comparisons increases. This issue is illustrated by the following table summarizing the probabilities of exceedances (false positive error rate) of a BTV (e.g., 95th percentile) by onsite observations, even when the site and background populations have comparable distributions. The probabilities of these chance exceedances increase as the site sample size increases. Sample Size 1 2 5 8 10 12 64 Probability of Exceedance 0.05 0.10 0.23 0.34 0.40 0.46 0.96 • two-sample hypotheses tests to compare site data distribution with background data distribution to determine if the site concentrations are comparable to background concentrations. An adequate amount of data needs to be made available from the site as well as the background populations. It is preferable to collect at least 10 observations from each population under comparison. Notes: From a mathematical point of view, one can perform hypothesis tests on data sets consisting of only 3-4 data values; however, the reliability of the test statistics (and the conclusions derived) thus obtained is questionable. In these situations it is suggested to supplement the test statistics decisions with graphical displays. 1.3 Discrete Samples or Composite Samples? ProUCL can be used for discrete sample data sets, as well as on composite sample data sets. However, in a data set (background or site), samples should be either all discrete or all composite. In general, both discrete and composite site samples may be used for individual point-by-point site comparisons with a threshold value, and for single and two-sample hypotheses testing applications. 47 ------- • When using a single-sample hypothesis testing approach, site data can be obtained by collecting all discrete or all composite samples. The hypothesis testing approach is used when many (> 10) site observations are available. Details of the single-sample hypothesis approaches are widely available in EPA guidance documents (MARSSIM 2000, EPA 1989a, 2006b). Several single- sample hypotheses testing procedures available in ProUCL are described in Chapter 6 of the ProUCL 5.1 Technical Guide. • If a two-sample hypothesis testing approach is used to perform site versus background comparisons, then samples from both of the populations should be either all discrete samples, or all composite samples. The two-sample hypothesis testing approaches are used when many (e.g., at least 10) site, as well as background, observations are available. For better results with higher statistical power, the availability of more observations perhaps based upon an appropriate DQOs process (EPA 2006a) is desirable. Several two-sample hypotheses tests available in ProUCL 5.1 are described in Chapter 6 of the ProUCL 5.1 Technical Guide. 1.4 Upper Limits and Their Use The computation and use of statistical limits depend upon their applications and the parameters (e.g., EPC term, BTVs) they are supposed to be estimating. Depending upon the objective of the study, a pre- specified cleanup standard, Cs, can be viewed as representing: 1) an average (or median) constituent concentration, //o; or 2) a not-to-exceed upper threshold concentration value, Ao. These two threshold values, /uo, and A0, represent two significantly different parameters, and different statistical methods and limits are used to compare the site data with these two very different threshold values. Statistical limits, such as a UCL of the population mean, a UPL for an independently obtained "single" observation, or independently obtained "k" observations (also called future k observations, next k observations, or k different observations), upper percentiles, and UTLs are often used to estimate the environmental parameters: EPC (fio) and a BTV (Ao). A new upper limit, USL was included in ProUCL 5.0 which may be used to estimate a BTV based upon a well-established background data set representing a single statistical population without any outliers. It is important to understand and note the differences between the uses and numerical values of these statistical limits so that they can be properly used. The differences between UCLs and UPLs (or upper percentiles), and UCLs and UTLs should be clearly understood. A UCL with a 95% confidence limit (UCL95) of the mean represents an estimate of the population mean (measure of the central tendency), whereas a UPL95, a UTL95%-95% (UTL95-95), and an upper 95th percentile represent estimates of a threshold from the upper tail of the population distribution such as the 95th percentile. Here, UPL95 represents a 95% upper prediction limit, and UTL95-95 represents a 95% confidence limit of the 95th percentile. For mildly skewed to moderately skewed data sets, the numerical values of these limits tend to follow the order given as follows. Sample Mean < UCL95 of Mean < Upper 95th Percentile < UPL95 of a Single Observation < UTL95-95 Example 1-1. Consider a real data set collected from a Superfund site. The data set has several inorganic COPCs, including aluminum (Al), arsenic (As), chromium (Cr), iron (Fe), lead (Pb), manganese (Mn), thallium (Tl) and vanadium (V). Iron concentrations follow a normal distribution. This data set has been used in several examples throughout the two ProUCL guidance documents (Technical Guide and User Guide), therefore it is provided as follows. 48 ------- Aluminum 6280 3830 3900 5130 9310 15300 9730 7840 10400 16200 6350 10700 15400 12500 2850 9040 2700 1710 3430 6790 11600 4110 7230 4610 Arsenic 1.3 1.2 2 1.2 3.2 5.9 2.3 1.9 2.9 3.7 1.8 2.3 2.4 2.2 1.1 3.7 1.1 1 1.5 2.6 2.4 1.1 2.1 0.66 Chromium 8.7 8.1 11 5.1 12 20 12 11 13 20 9.8 14 17 15 8.4 14 4.5 3 4 11 16.4 7.6 35.5 6.1 Iron 4600 4330 13000 4300 11300 18700 10000 8900 12400 18200 7340 10900 14400 11800 4090 15300 6030 3060 4470 9230 Lead 16 6.4 4.9 8.3 18 14 12 8.7 11 12 14 14 19 21 16 25 20 11 6.3 13 98.5 53.3 109 8.3 Manganese 39 30 10 92 530 140 440 130 120 70 60 110 340 85 41 66 21 8.6 19 140 72.5 27.2 118 22.5 Thallium 0.0835 0.068 0.155 0.0665 0.071 0.427 0.352 0.228 0.068 0.456 0.067 0.0695 0.07 0.214 0.0665 0.4355 0.0675 0.066 0.067 0.068 0.13 0.068 0.095 0.07 Vanadium 12 8.4 11 9 22 32 19 17 21 32 15 21 28 25 8 24 11 7.2 8.1 16 Several upper limits for iron are summarized as follows, and it be seen that they follow the order (in magnitude) as described above. Table 1-1. Computation of Upper Limits for Iron (Normally Distributed) Mean 9618 Median 9615 Min 3060 Max 18700 UCL95 11478 UPL95 for a Single Observation 18145 UPL95 for 4 Observations 21618 UTL95-95 21149 95% Upper Percentile 17534 For highly skewed data sets, these limits may not follow the order described above. This is especially true when the upper limits are computed based upon a lognormal distribution (Singh, Singh, and Engelhardt 1997). It is well known that a lognormal distribution based H-UCL95 (Land's UCL95) often yields unstable and impractically large UCL values. An H-UCL95 often becomes larger than UPL95 and even larger than a UTL 95%-95% and the largest sample value. This is especially true when dealing with skewed data sets of smaller sizes. Moreover, it should also be noted that in some cases, a H-UCL95 becomes smaller than the sample mean, especially when the data are mildly skewed and the sample size is large (e.g., > 50, 100). 49 ------- There is a great deal of confusion about the appropriate use of these upper limits. A brief discussion about the differences between the applications and uses of the statistical limits described above is provided as follows. • A UCL represents an average value that is compared with a threshold value also representing an average value (pre-established or estimated), such as a mean Cs. For example, a site 95% UCL exceeding a Cs, may lead to the conclusion that the cleanup standard, Cs has not been attained by the average site area concentration. It should also be noted that UCLs of means are typically computed from the site data set. • A UCL represents a "collective" measure of central tendency, and it is not appropriate to compare individual site observations with a UCL. Depending upon data availability, single or two-sample hypotheses testing approaches are used to compare a site average or a site median with a specified or pre-established cleanup standard (single-sample hypothesis), or with the background population average or median (two-sample hypothesis). • A UPL, an upper percentile, or a UTL represents an upper limit to be used for point-by-point individual site observation comparisons. UPLs and UTLs are computed based upon background data sets, and point-by-point onsite observations are compared with those limits. A site observation exceeding a background UTL may lead to the conclusion that the constituent is present at the site at levels greater than the background concentrations level. • When enough (e.g., at least 10) site observations are available, it is preferable to use hypotheses testing approaches. Specifically, single-sample hypotheses testing (comparing site to a specified threshold) approaches should be used to perform site versus a known threshold comparison; and two- sample hypotheses testing (provided enough background data are also available) approaches should be used to perform site versus background comparison. Several parametric and nonparametric single and two-sample hypotheses testing approaches are available in ProUCL 5.0/ProUCL 5.1. It is re-emphasized that only averages should be compared with averages or UCLs, and individual site observations should be compared with UPLs, upper percentiles, UTLs, or USLs. For example, the comparison of a 95% UCL of one population (e.g., site) with a 90% or 95% upper percentile of another population (e.g., background) cannot be considered fair and reasonable as these limits (e.g., UCL and UPL) estimate and represent different parameters. 1.5 Point-by-Point Comparison of Site Observations with BTVs, Compliance Limits and Other Threshold Values The point-by-point observation comparison method is used when a small number (e.g., < 6) of site observations are compared with pre-established or estimated BTVs, screening levels, or preliminary remediation goals (PRGs). Typically, a single exceedance of the BTV by an onsite (or a monitoring well) observation may be considered an indication of the presence of contamination at the site area under investigation. The conclusion of an exceedance by a site value is sometimes confirmed by re-sampling (taking a few more collocated samples) at the site location (or a monitoring well) exhibiting constituent concentrations in excess of the BTV. If all collocated sample observations (or all sample observations collected during the same time period) from the same site location (or well) exceed the BTV or PRO, then it may be concluded that the location (well) requires further investigation (e.g., continuing treatment and monitoring) and possibly cleanup. 50 ------- When BTV constituent concentrations are not known or pre-established, one has to collect or extract a background data set of an appropriate size that can be considered representative of the site background. Statistical upper limits are computed using the background data set thus obtained, which are used as estimates of BTVs. To compute reasonably reliable estimates of BTVs, a minimum of 10 background observations should be collected, perhaps using an appropriate DQOs process as described in EPA (2000, 2006a). Several statistical limits listed above are used to estimate BTVs based upon a defensible (free of outliers, representing the background population) background data set of an adequate size. The point-by-point comparison method is also useful when quick turnaround comparisons are required in real time. Specifically, when decisions have to be made in real time by a sampling/screening crew, or when only a few site samples are available, then individual point-by-point site concentrations are compared either with pre-established cleanup goals or with estimated BTVs. The sampling crew can use these comparisons to: 1) screen and identify the COPCs, 2) identify the potentially polluted site AOCs, or 3) continue or stop remediation or excavation at an onsite area of concern. If a larger number of samples (e.g., >10) are available from the AOC, then the use of hypotheses testing approaches (both single-sample and a two-sample) is preferred. The use of hypothesis testing approaches tends to control the error rates more tightly and efficiently than the individual point-by-point site comparisons. 1.6 Hypothesis Testing Approaches and Their Use Both single-sample and two-sample hypotheses testing approaches are used to make cleanup decisions at polluted sites, and also to compare constituent concentrations of two (e.g., site versus background) or more populations (e.g., MWs). 1.6.1 Single Sample Hypotheses (Pre-established BTVs and Not-to-Exceed Values are Known) When pre-established BTVs are used such as the U.S. Geological Survey (USGS) background values (Shacklette and Boerngen 1984), or thresholds obtained from similar sites, there is no need to extract, establish, or collect a background data set. When the BTVs and cleanup standards are known, one-sample hypotheses are used to compare site data (provided enough site data are available) with known and pre- established threshold values. It is suggested that the project team determine (e.g., using DQOs) or decide (depending upon resources) the number of site observations that should be collected and compared with the "pre-established" standards before coming to a conclusion about the status (clean or polluted) of the site AOCs. As mentioned earlier, when the number of available site samples is < 6, one might perform point-by-point site observation comparisons with a BTV; and when enough site observations (at least 10) are available, it is desirable to use single-sample hypothesis testing approaches. Depending upon the parameter (juo, Ao), represented by the known threshold value, one can use single-sample hypotheses tests for population mean or median (t-test, sign test), or use single-sample tests for proportions and percentiles. The details of the single-sample hypotheses testing approaches can be found in EPA (2006b) guidance document and in Chapter 6 of ProUCL Technical Guide. One-Sample t-Test: This test is used to compare the site mean, /u, with some specified cleanup standard, Cs, where the Cs represents an average threshold value, /no. The Student's t-test (or a UCL of the mean) is used (assuming normality of site data set or when sample size is large, such as larger than 30, 50) to verify the attainment of cleanup levels at a polluted site after some remediation activities. 51 ------- One-Sample Sign Test or Wilcoxon Signed Rank (WSR) Test: These tests are nonparametric tests and can also handle ND observations, provided the detection limits of all NDs fall below the specified threshold value, Cs. These tests are used to compare the site location (e.g., median, mean) with some specified Cs representing a similar location measure. One-Sample Proportion Test or Percentile Test: When a specified cleanup standard, Ao, such as a PRG or a BTV represents an upper threshold value of a constituent concentration distribution rather than the mean threshold value, juo, then a test for proportion or a test for percentile (equivalently UTL 95-95 UTL 95-90) may be used to compare site proportion (or site percentile) with the specified threshold or action level, Ao. 1.6.2 Two-Sample Hypotheses (BTVs and Not-to-Exceed Values are Unknown) When BTVs, not-to-exceed values, and other cleanup standards are not available, then site data are compared directly with the background data. In such cases, two-sample hypothesis testing approaches are used to perform site versus background comparisons. Note that this approach can be used to compare concentrations of any two populations including two different site areas or two different monitoring wells (MWs). In order to use and perform a two-sample hypothesis testing approach, enough data should be available from each of the two populations. Site and background data requirements (e.g., based upon DQOs) for performing two-sample hypothesis test approaches are described in EPA (2000, 2002b, 2006a, 2006b) and also in Chapter 6 of the ProUCL 5.1 Technical Guide. While collecting site and background data, for better representation of populations under investigation, one may also want to account for the size of the background area (and site area for site samples) in sample size determination. That is, a larger number (>15-20) of representative background (and site) samples should be collected from larger background (and site) areas; every effort should be made to collect as many samples as determined by the DQOs-based sample sizes. The two-sample (or more) hypotheses approaches are used when the site parameters (e.g., mean, shape, distribution) are being compared with the background parameters (e.g., mean, shape, distribution). The two-sample hypotheses testing approach is also used when the cleanup standards or screening levels are not known a priori. Specifically, in environmental applications, two-sample hypotheses testing approaches are used to compare average or median constituent concentrations of two or more populations. To derive reliable conclusions with higher statistical power based upon hypothesis testing approaches, an adequate amount of data (e.g., minimum of 10 samples) should be collected from all of the populations under investigation. The two-sample hypotheses testing approaches incorporated in ProUCL 5.1 are listed as follows: 1. Student t-test (with equal and unequal variances) - Parametric test assumes normality 2. Wilcoxon-Mann-Whitney (WMW) test - Nonparametric test handles data with NDs with one DL - assumes two populations have comparable shapes and variability 3. Gehan test - Nonparametric test handles data sets with NDs and multiple DLs - assumes comparable shapes and variability 4. Tarone-Ware (T-W) test - Nonparametric test handles data sets with NDs and multiple DLs - assumes comparable shapes and variability The Gehan and T-W tests are meant to be used on left-censored data sets with multiple DLs. For best results, the samples collected from the two (or more) populations should all be of the same type obtained using similar analytical methods and apparatus; the collected site and background samples should all be discrete or all composite (obtained using the same design and pattern), and be collected from the same 52 ------- medium (soil) at similar depths (e.g., all surface samples or all subsurface samples) and time (e.g., during the same quarter in groundwater applications) using comparable (preferably same) analytical methods. Good sample collection methods and sampling strategies are given in EPA (1996, 2003) guidance documents. Note: ProUCL 5.1 (and previous versions) has been developed using limited government funding. ProUCL 5.1 is equipped with statistical and graphical methods needed to address many environmental sampling and statistical issues as described in the various CERCLA, MARSSIM, and RCRA documents cited earlier. However, one may not compare the availability of methods in ProUCL 5.1 with methods incorporated in commercial software packages such as SAS® and Minitab 16. Not all methods available in the statistical literature are available in ProUCL. 1.7 Minimum Sample Size Requirements and Power Evaluations Due to resource limitations, it is not be possible (nor needed) to sample the entire population (e.g., background area, site area, AOCs, EAs) under study. Statistics is used to draw inference(s) about the populations (clean, dirty) and their known or unknown statistical parameters (e.g., mean, variance, upper threshold values) based upon much smaller data sets (samples) collected from those populations. To determine and establish BTVs and site specific screening levels, defensible data set(s) of appropriate size(s) representing the background population (e.g., site-specific, general reference area, or historical data) need to be collected. The project team and site experts should decide what represents a site population and what represents a background population. The project team should determine the population area and boundaries based upon all current and intended future uses, and the objectives of data collection. Using the collected site and background data sets, statistical methods supplemented with graphical displays are used to perform site versus background comparisons. The test results and statistics obtained by performing such site versus background comparisons are used to determine if the site and background level constituent concentrations are comparable; or if the site concentrations exceed the background threshold concentration level; or if an adequate amount of remediation approaching the BTV or some cleanup level has been performed at polluted site AOCs. To perform these statistical tests, determine the number of samples that need to be collected from the populations (e.g., site and background) under investigation using appropriate DQOs processes (EPA 2000, 2006a, 2006b). ProUCL has the Sample Sizes module which can be used to develop DQOs based sampling designs needed to address statistical issues associated with polluted sites projects. ProUCL provides user-friendly options to enter the desired/pre-specified values of decision parameters (e.g., Type I and Type II error rates) to determine minimum sample sizes for the selected statistical applications including: estimation of mean, single and two-sample hypothesis testing approaches, and acceptance sampling. Sample size determination methods are available for the sampling of continuous characteristics (e.g., lead or Radium 226), as well as for attributes (e.g., proportion of occurrences exceeding a specified threshold). Both parametric (e.g., t-tests) and nonparametric (e.g., Sign test, test for proportions, WRS test) sample size determination methods are available in ProUCL 5.1 and in its earlier versions (e.g., ProUCL 4.1). ProUCL also has sample size determination methods for acceptance sampling of lots of discrete objects such as a batch of drums containing hazardous waste (e.g., RCRA applications, U.S. EPA 2002c). However, due to budgetary or logistical constraints, it may not be possible to collect the same number of samples as determined by applying a DQO process. For example, the data might have already been collected (as often is the case) without using a DQO process, or due to resource constraints, it may not have been possible to collect as many samples as determined by using a DQO-based sample size formula. 53 ------- In practice, the project team and the decision makers tend not to collect enough background samples. It is suggested to collect at least 10 background observations before using statistical methods to perform background evaluations based upon data collected using discrete samples. The minimum sample size recommendations described here are useful when resources are limited, and it may not be possible to collect as many background and site samples as computed using DQOs based sample size determination formulae. In case data are collected without using a DQO process, the Sample Sizes module can be used to assess the power of the test statistic in retrospect. Specifically, one can use the standard deviation of the computed test statistic (EPA 2006b) and compute the sample size needed to meet the desired DQOs. If the computed sample size is greater than the size of the data set used, the project team may want to collect additional samples to meet the desired DQOs. Note: From a mathematical point of view, the statistical methods incorporated in ProUCL and described in this guidance document for estimating EPC terms and BTVs, and comparing site versus background concentrations can be performed on small site and background data sets (e.g., of sizes as small as 3). However, those statistics may not be considered representative and reliable enough to make important cleanup and remediation decisions which will potentially impact human health and the environment. ProUCL provides messages when the number of detects is <4-5, and suggests collecting at least 8-10 observations. Based upon professional judgment, as a rule-of-thumb, ProUCL guidance documents recommend collecting a minimum of 10 observations when data sets of a size determined by a DQOs process (EPA 2006) cannot be collected. This however, should not be interpreted as the general recommendation and every effort should be made to collect DQOs based number of samples. Some recent guidance documents (e.g., EPA 2009) have also adopted this rule-of-thumb and suggest collecting a minimum of about 8-10 samples in the circumstance that data cannot be collected using a DQO-based process. However, the project team needs to make these determinations based upon their comfort level and knowledge of site conditions. • To allow users to compute decision statistics using data from ISM (ITRC, 2012) samples, ProUCL 5.1 will compute decision statistics (e.g., UCLs, UPLs, UTLs) based upon samples of sizes as small as 3. The user is referred to the ITRC ISM Technical Regulatory Guide (2012) to determine which UCL (e.g., Student's t-UCL or Chebyshev UCL) should be used to estimate the EPC term. 1.7.1 Why a data set of minimum size, n = 8-10? Typically, the computation of parametric upper limits (UPL, UTL, UCL) depends upon three values: the sample mean, sample variability (standard deviation) and a critical value. A critical value depends upon sample size, data distribution, and confidence level. For samples of small size (< 8-10), the critical values are large and unstable, and upper limits (e.g., UTLs, UCLs) based upon a data set with fewer than 8-10 observations are mainly driven by those critical values. The differences in the corresponding critical values tend to stabilize when the sample size becomes larger than 8-10 (see tables below, where degrees of freedom [df] = sample size - 1). This is one of the reasons ProUCL guidance documents suggest a minimum data set size of 10 when the number of observations determined from sample-size calculations based upon EPA DQO process exceed the logistical/financial/temporal/constraints of a project. For samples of sizes 2-11, 95% critical values used to compute upper limits (UCLs, UPLs, UTLs, and USLs) based upon a normal distribution are summarized in the subsequent tables. In general, a similar pattern is followed for critical values used in the computation of upper limits based upon other distributions. For the normal distribution, Student's t-critical values are used to compute UCLs and UPLs which are summarized as follows. 54 ------- Table of Critical Values oft-Statistic .HI Upper-la:] probability p .O5- .025 .ill .01 3.U7S ( l.XSb l.n*S 1.5^3 1.47o 1 .440 1.415 1.397 1.383 1.372 -..314 1.420 1.35* I 132 1.015 .443 .845 .800 .833 .812 12.71 4.30.5 3.1H2 2.77a 2.571 2.447 1365 2.31*0 2.262 2.228 1x84 4.K44 <.4o'2 2.444 2.757 2.612 2.517 2.449 2.3*8 2.359 31.« f.>*5 4.541 1.747 3J65 3.14.? 2 .9W 2.4% 2.421 2.?64 to One can see that once the sample size starts exceeding 9-10 (df= 8, 9), the difference between the critical values starts stabilizing. For example, for upper tail probability (= level of significance) of 0.05, the difference between critical values for df= 9 and df=\0 is only 0.021, where as the difference between critical values for df= 4 and 5 is 0.117; similar patterns are noted for other levels of significance. For the normal distribution, critical values used to compute UTL90-95, UTL95-95, USL90, and USL95 are described as follows. One can see that once the sample size starts exceeding 9-10, the difference between the critical values starts decreasing significantly. UTL90-95 UTL95-95 USL90 USL95 3 4 5 6 1 8 9 10 11 6.155 4.162 3.407 3.006 2.755 2.582 2.454 2.355 2.275 7.656 5.144 4.203 3.708 3.399 3.187 3.031 2.911 2.815 1.148 1.425 1.602 1.729 1.828 1.909 1.977 2.036 2.088 1.153 1.462 1.671 1.822 1.938 2.032 2.11 2.176 2.234 Note: Nonparametric upper limits (UPLs, UTLs, and USLs) are computed using higher order statistics of a data set. To achieve the desired confidence coefficient, samples of sizes much greater than 10 are required. For details, refer to Chapter 3. It should be noted that critical values of USLs are significantly lower than critical values for UTLs. Critical values associated with UTLs decrease as the sample size increases. Since, as the sample size increases the maximum of the data set also increases, and critical values associated with USLs increase with the sample size. 1.7.2 Sample Sizes for Bootstrap Methods Several nonparametric methods including bootstrap methods for computing UCL, UTL, and other limits for both full-uncensored data sets and left-censored data sets with NDs are available in ProUCL 5.1. Bootstrap resampling methods are useful when not too few (e.g., < 15-20) and not too many (e.g., > 500- 1000) observations are available. For bootstrap methods (e.g., percentile method, BCA bootstrap method, bootstrap-t method), a large number (e.g., 1000, 2000) of bootstrap resamples are drawn with replacement from the same data set. Therefore, to obtain bootstrap resamples with at least some distinct values (so that statistics can be computed from each resample), it is suggested that a bootstrap method should not be used when dealing with small data sets of sizes less than 15-20. Also, it is not necessary to bootstrap a large 55 ------- data set of size greater than 500 or 1000; that is when a data set of a large size (e.g., > 500) is available, there is no need to obtain bootstrap resamples to compute statistics of interest (e.g., UCLs). One can simply use a statistical method on the original large data set. Note: Rules-of-thumb about minimum sample size requirements described in this section are based upon professional experience of the developers. ProUCL software is not a policy software. It is recommended that the users/project teams/agencies make determinations about the minimum number of observations and minimum number of detects that should be present in a data set before using a statistical method. 1.8 Statistical Analyses by a Group ID The analyses of data categorized by a group ID variable such as: 1) Surface vs. Subsurface; 2) AOC1 vs. AOC2; 3) Site vs. Background; and 4) Upgradient vs. Downgradient monitoring wells are common in environmental applications. ProUCL 5.1 offers this option for data sets with and without NDs. The Group Option provides a tool for performing separate statistical tests and for generating separate graphical displays for each member/category of the group (samples from different populations) that may be present in a data set. The graphical displays (e.g., box plots, quantile-quantile plots) and statistics (e.g., background statistics, UCLs, hypotheses tests) of interest can be computed separately for each group by using this option. Moreover, using the Group Option, graphical methods can display multiple graphs (e.g., Q-Q plots) on the same graph providing graphical comparison of multiple groups. It should be pointed out that it is the user's responsibility to provide an adequate amount of data to perform the group operations. For example, if the user desires to produce a graphical Q-Q plot (e.g., using only detected data) with regression lines displayed, then there should be at least two detected data values (to compute slope, intercept, sd) in the data set. Similarly, if the graphs are desired for each group specified by the group ID variable, there should be at least two observations in each group specified by the group variable. When ProUCL data requirements are not met, ProUCL does not perform any computations, and generates a warning message (colored orange) in the lower Log Panel of the output screen of ProUCL 5.1. 1.9 Statistical Analyses for Many Constituents/Variables ProUCL software can process multiple analytes/variables simultaneously in a user-friendly manner This option is useful when one has to process multiple variables and compute decision statistics (e.g., UCLs, UPLs, and UTLs) and test statistics (e.g., ANOVA test, trend test) for multiple variables. It is the user's responsibility to make sure that each selected variable has an adequate amount of data so that ProUCL can perform the selected statistical method correctly. ProUCL displays warning messages when a selected variable does not have enough data needed to perform the selected statistical method. 1.10 Use of Maximum Detected Value as Estimates of Upper Limits Some practitioners use the maximum detected value as an estimate of the EPC term. This is especially true when the sample size is small such as < 5, or when a UCL95 exceeds the maximum detected values (EPA 1992a). Also, many times in practice, the BTVs and not-to-exceed values are estimated by the maximum detected value (e.g., nonparametric UTLs, USLs). 56 ------- 1.10.1 Use of Maximum Detected Value to Estimate BTVs and Not-to-Exceed Values BTVs and not-to-exceed values represent upper threshold values from the upper tail of a data distribution; therefore, depending upon the data distribution and sample size, the BTVs and other not-to-exceed values may be estimated by the largest or the second largest detected value. A nonparametric UPL, UTL, and USL are often estimated by higher order statistics such as the maximum value or the second largest value (EPA 1992b, 2009, Hahn and Meeker 1991). The use of higher order statistics to estimate the UTLs depends upon the sample size. For data sets of size: 1) 59 to 92 observations, a nonparametric UTL95-95 is given by the maximum detected value; 2) 93 to 123 observations, a nonparametric UTL95-95 is given by the second largest maximum detected value; and 3) 124 to 152 observations, a UTL95-95 is given by the third largest detected value in the sample, and so on. 1.10.2 Use of Maximum Detected Value to Estimate EPC Terms Some practitioners tend to use the maximum detected value as an estimate of the EPC term. This is especially true when the sample size is small such as < 5, or when a UCL95 exceeds the maximum detected value. Specifically, the EPA (1992a) document suggests the use of the maximum detected value as a default value to estimate the EPC term when a 95% UCL (e.g., the H-UCL) exceeds the maximum value in a data set. ProUCL computes 95% UCLs of the mean using several methods based upon normal, gamma, lognormal, and non-discernible distributions. In the past, a lognormal distribution was used as the default distribution to model positively skewed environmental data sets. Additionally, only two methods were used to estimate the EPC term based upon: 1) normal distribution and Student's t-statistic, and 2) lognormal distribution and Land's H-statistic (Land 1971, 1975). The use of the H-statistic often yields unstable and impractically large UCL95 of the mean (Singh, Singh, and Engelhardt 1997; Singh, Singh, and laci 2002). For highly skewed data sets of smaller sizes (< 30, < 50), H-UCL often exceeds the maximum detected value. Since the use of a lognormal distribution has been quite common (suggested as a default model in the risk assessment guidance for Superfund [RAGS] document [EPA 1992a]), the exceedance of the maximum value by an H-UCL95 is frequent for many skewed data sets of smaller sizes (e.g., < 30, < 50). These occurrences result in the possibility of using the maximum detected value as an estimate of the EPC term. It should be pointed out that in some cases, the maximum observed value actually might represent an impacted location. Obviously, it is not desirable to use an observation potentially representing an impacted location to estimate the EPC for an AOC. The EPC term represents the average exposure contracted by an individual over an EA during a long period of time; the EPC term should be estimated by using an average value (such as an appropriate 95% UCL of the mean) and not by the maximum observed concentration. One needs to compute an average exposure and not the maximum exposure. Singh and Singh (2003) studied the performance of the max test (using the maximum observed value to estimate the EPC) via Monte Carlo simulation experiments. They noted that for skewed data sets of small sizes (e.g., < 10-20), even the max test does not provide the specified 95% coverage to the population mean, and for larger data sets it overestimates the EPC term, which may lead to unnecessary further remediation. Several methods, some of which are described in EPA (2002a) and other EPA documents, are available in versions of ProUCL (i.e., ProUCL 3.00.02 [EPA 2004], ProUCL 4.0 [U.S. EPA 2007], ProUCL 4.00.05 [EPA 2009, 2010], ProUCL 4.1 [EPA 2011]) for estimating the EPC terms. For data sets with NDs, ProUCL 5.0 (and ProUCL 5.1) has some new UCL (and other limits) computation methods which were not available in earlier versions of ProUCL. It is unlikely that the UCLs based upon those methods will exceed the maximum detected value, unless some outliers are present in the data set. 57 ------- 1.10.2.1 Chebyshev Inequality Based UCL95 ProUCL 5.1 (and its earlier versions) displays a warning message when the suggested 95% UCL (e.g., Hall's or bootstrap-t UCL with outliers) of the mean exceeds the detected maximum concentration. When a 95% UCL does exceed the maximum observed value, ProUCL suggests the use of an alternative UCL computation method based upon the Chebyshev inequality. One may use a 97.5% or 99% Chebyshev UCL to estimate the mean of a highly skewed population. The use of the Chebyshev inequality to compute UCLs tends to yield more conservative (but stable) UCLs than other methods available in ProUCL software. In such cases, when the sample size is large (and other UCL methods such as the bootstrap-t method yield unrealistically high values due to presence of outliers), one may want to use a 95% Chebyshev UCL or a Chebyshev UCL with a lower confidence coefficient such as 90% as an estimate of the population mean, especially when the sample size is large (e.g., >100, 150). The details (as functions of sample size and skewness) for the use of those UCLs are summarized in various versions of ProUCL Technical Guides (EPA 2004, 2007, 2009, 2010d, 2011, 2013a). Notes: Using the maximum observed value to estimate the EPC term representing the average exposure contracted by an individual over an EA is not recommended. For the sake of interested users, ProUCL displays a warning message when the recommended 95% UCL (e.g., Hall's bootstrap UCL) of the mean exceeds the observed maximum concentration. For such scenarios (when a 95% UCL does exceed the maximum observed value), an alternative UCL computation method based upon Chebyshev inequality is suggested by the ProUCL software. 1.11 Samples with Nondetect Observations ND observations are inevitable in most environmental data sets. Singh, Maichle, and Lee (2006) studied the performances (in terms of coverages) of the various UCL95 computation methods including the simple substitution methods (such as the DL/2 and DL methods) for data sets with ND observations. They concluded that the UCLs obtained using the substitution methods, including the replacement of NDs by DL/2; do not perform well even when the percentage of ND observations is low, such as less than 5% to 10%. They recommended avoiding the use of substitution methods for computing UCL95 based upon data sets with ND observations. 1.11.1 Avoid the Use of the DL/2 Substitution Method to Compute UCL95 Based upon the results of the report by Singh, Maichle, and Lee (2006), it is recommended to avoid the use of the DL/2 substitution method when performing a GOF test, and when computing the summary statistics and various other limits (e.g., UCL, UPL, UTLs) often used to estimate the EPC terms and BTVs. Until recently, the substitution method has been the most commonly used method for computing various statistics of interest for data sets which include NDs. The main reason for this has been the lack of the availability of the other rigorous methods and associated software programs that can be used to estimate the various environmental parameters of interest. Today, several methods (e.g., using KM estimates) with better performance, including the Chebyshev inequality and bootstrap methods, are available for computing the upper limits of interest. Several of those parametric and nonparametric methods are available in ProUCL 4.0 and higher versions. The DL/2 method is included in ProUCL for historical reasons as it had been the most commonly used and recommended method until recently (EPA 2006b). EPA scientists and several reviewers of the ProUCL software had suggested and requested the inclusion of the DL/2 substitution method in ProUCL for comparison and research purposes. 58 ------- Notes: Even though the DL/2 substitution method has been incorporated in ProUCL, its use is not recommended due to its poor performance. The DL/2 substitution method has been retained in ProUCL 5.1 for historical and comparison purposes. NERL-EPA, Las Vegas strongly recommends avoiding the use of this method even when the percentage of NDs is as low as 5% to 10%. 1.11.2 ProUCL Does Not Distinguish between Detection Limits, Reporting limits, or Method Detection Limits ProUCL 5.1 (and all previous versions) does not make distinctions between method detection limits (MDLs), adjusted MDLs, sample quantitation limits (SQLs), reporting limits (RLs), or DLs. Multiple DLs (or RLs) in ProUCL mean different values of the detection limits. It is user's responsibility to understand the differences between these limits and use appropriate values (e.g., DLs) for nondetect values below which the laboratory cannot reliably detect/measure the presence of the analyte in collected samples (e.g., soil samples). A data set consisting of values less than the DLs (or MDLs, RLs) is considered a left-censored data set. ProUCL uses statistical methods available in the statistical literature for left-censored data sets for computing statistics of interest including mean, sd, UCL, and estimates of BTVs. The user determines which qualifiers (e.g., J, U, UJ) will be considered as nondetects. Typically, all values with U or UJ qualifiers are considered as nondetect values. It is the user's responsibility to enter a value which can be used to represent a ND value. For NDs, the user enters the associated DLs or RLs (and not zeros or half of the detection limits). An indicator column/variable, D_x taking a value, 0, for all nondetects and a value, 1, for all detects is assigned to each variable, x, with NDs. It is the user's responsibility to supply the numerical values for NDs (should be entered as reported DLs) not qualifiers (e.g., J, U, B, UJ). For example, for thallium with nondetect values, the user creates an associated column labeled as D_thallium to tell the software that the data set will have nondetect values. This column, D_thallium consists of only zeros (0) and ones (1); zeros are used for all values reported as NDs and ones are used for all values reported as detects. 1.12 Samples with Low Frequency of Detection When all of the sampled values are reported as NDs, the EPC term and other statistical limits should also be reported as a ND value, perhaps by the maximum RL or the maximum RL/2. The project team will need to make this determination. Statistics (e.g., UCL95) based upon only a few detected values (e.g., < 4) cannot be considered reliable enough to estimate EPCs which can have a potential impact on human health and the environment. When the number of detected values is small, it is preferable to use ad hoc methods rather than using statistical methods to compute EPCs and other upper limits. Specifically, for data sets consisting of < 4 detects and for small data sets (e.g., size < 10) with low detection frequency (e.g., < 10%), the project team and the decision makers should decide, on a site-specific basis, how to estimate the average exposure (EPC) for the constituent and area under consideration. For data sets with low detection frequencies, other measures such as the median or mode represent better estimates (with lesser uncertainty) of the population measure of central tendency. Additionally, when most (e.g., > 95%) of the observations for a constituent lie below the DLs, the sample median or the sample mode (rather than the sample average) may be used as an estimate of the EPC. Note that when the majority of the data are NDs, the median and the mode may also be represented by a ND value. The uncertainty associated with such estimates will be high. The statistical properties, such as the bias, accuracy, and precision of such estimates, would remain unknown. In order to be able to compute defensible estimates, it is always desirable to collect more samples. 59 ------- 1.13 Some Other Applications of Methods in ProUCL 5.1 In addition to performing background versus site comparisons for CERCLA and RCRA sites, performing trend evaluations based upon time-series data sets, and estimating EPCs in exposure and risk evaluation studies, the statistical methods in ProUCL can be used to address other issues dealing with environmental investigations that are conducted at Superfund or RCRA sites. 1.13.1 Identification of COPCs Risk assessors and remedial project managers (RPMs) often use screening levels or BTVs to identify COPCs during the screening phase of a cleanup project at a contaminated site. The screening for COPCs is performed prior to any characterization and remediation activities that are conducted at the site. This comparison is performed to screen out those constituents that may be present in the site medium of interest at low levels (e.g., at or below the background levels or some pre-established screening levels) and may not pose any threat and concern to human health and the environment. Those constituents may be eliminated from all future site investigations, and risk assessment and risk management studies. To identify the COPCs, point-by-point site observations are compared with some pre-established soil screening levels (SSL) or estimated BTVs. This is especially true when the comparisons of site concentrations with screening levels or BTVs are conducted in real time by the sampling or cleanup crew onsite. The project team should decide the type of site samples (discrete or composite) and the number of site observations that should be collected and compared with the screening levels or the BTVs. In case BTVs or screening levels are not known, the availability of a defensible site-specific background or reference data set of reasonable size (e.g., at least 10) is required for computing reliable and representative estimates of BTVs and screening levels. The constituents with concentrations exceeding the respective screening values or BTVs may be considered COPCs, whereas constituents with concentrations (e.g., in all collected samples) lower than the screening values or BTVs may be omitted from all future evaluations. 1.13.2 Identification of Non-Compliance Monitoring Wells In MW compliance assessment applications, individual (often discrete) constituent concentrations from a MW are compared with some pre-established limits such as an ACL or a maximum concentration limit (MCL). An exceedance of the MCL or the BTV (e.g., estimated by a UTL95-95 or a UPL95) by a MW concentration may be considered an indication of contamination in that MW. For individual concentration comparisons, the presence of contamination (determined by an exceedance) may have to be confirmed by re-sampling from that MW. If concentrations of constituents in the original sample and re-sample(s) exceed the MCL or BTV, then that MW may require further scrutiny, perhaps triggering remediation activities. If the concentration data from a MW for 4 to 5 continuous quarters (or some other designated time period determined by the project team) are below the MCL or BTV level, then that MW may be considered as complying with (achieving) the pre-established or estimated standards. 1.13.3 Verification of the Attainment of Cleanup Standards, Cs Hypothesis testing approaches are used to verify the attainment of the cleanup standard, Cs, at site AOCs after conducting remediation and cleanup at those site AOCs (EPA 1989a, 1994). In order to assess the attainment of cleanup levels, a representative data set of adequate size perhaps obtained using the DQO process (or a minimum of 10 observations should be collected) needs to be made available from the remediated/excavated areas of the site under investigation. The sample size should also account for the 60 ------- size of the remediated site areas: meaning that larger site areas should be sampled more (with more observations) to obtain a representative sample of the remediated areas under investigation. Typically, the null hypothesis of interest is Ho: Site Mean, /& > Cs versus the alternative hypothesis, Hi: Site Mean, /4 < Cs, where the cleanup standard, Cs, is known a priori. 1.13.4 Using BTVs (Upper Limits) to Identify Hot Spots The use of upper limits (e.g., UTLs) to identify hot spot(s) has also been mentioned in the Guidance for Comparing Background and Chemical Concentrations in Soil for CERCLA Sites (EPA 2002b). Point-by- point site observations are compared with a pre-established or estimated BTV. Exceedances of the BTV by site observations may represent impacted locations with elevated concentrations (hot spots). 1.14 Some General Issues, Suggestions and Recommendations made by ProUCL Some general issues regarding the handling of multiple DLs by ProUCL and recommendations made about various substitution and ROS methods for data sets with NDs are described in the following sections. 1.14.1 Handling of Field Duplicates ProUCL does not pre-process field duplicates. The project team determines how field duplicates will be handled and pre-processes the data accordingly. For an example, if the project team decides to use average values for field duplicates, then averages need to be computed and field duplicates need to be replaced by their respective average values. It is the user's responsibility to feed in appropriate values (e.g., averages, maximum) for field duplicates. The user is advised to refer to the appropriate EPA guidance documents related to collection and use of field duplicates for more information. 1.14.2 ProUCL Recommendation about ROS Method and Substitution (DL/2) Method For data sets with NDs, ProUCL can compute point estimates of population mean and standard deviation using the KM and ROS methods (and also using the DL/2 substitution method). The substitution method has been retained in ProUCL for historical and research purposes. ProUCL uses Chebyshev inequality, bootstrap methods, and normal, gamma, and lognormal distribution based equations on KM (or ROS) estimates to compute upper limits (e.g., UCLs, UTLs). The simulation study conducted by Singh, Maichle and Lee (2006) demonstrated that the KM method yields accurate estimates of the population mean. They also demonstrated that for moderately skewed to highly skewed data sets, UCLs based upon KM estimates and BCA bootstrap (mild skewness), KM estimates and Chebyshev inequality (moderate to high skewness), and KM estimates and bootstrap-t method (moderate to high skewness) yield better (in terms of coverage probability) estimates of EPCs than other UCL methods based upon the Student's t- statistic on KM estimates, percentile bootstrap method on KM or ROS estimates. 1.14.3 Unhandled Exceptions and Crashes in ProUCL A typical statistical software, especially developed under limited resources may not be able to accommodate data sets with all kinds of deficiencies such as all missing values for a variable, or all nondetect values for a variable. An inappropriate/insufficient data set can occur in various forms and not all of them can be addressed in a scientific program like ProUCL. Specifically, from a programming point of view, it can be quite burdensome on the programmer to address all potential deficiencies that can occur 61 ------- in a data set. ProUCL 5.1 addresses many data deficiencies and produces warming messages. All data deficiencies causing unhandled exceptions which were identified by users have been addressed in ProUCL 5.1. However, when ProUCL yields an unhandled exception or crashes, it is highly likely that there is something wrong with the data set; the user is advised to review the input data set to make sure that the data set follows ProUCL data and formatting requirements. 1.15 The Unofficial User Guide to ProUCL4 (Helsel and Gilroy 2012) Several ProUCL 4.1 users sent inquiries about the validity of the comments made about the ProUCL software in the Unofficial User Guide to ProUCL4 (Helsel and Gilroy, 2012) and in the Practical Stats webinar, "ProUCL v4: The Unofficial User Guide," presented by Dr. Helsel on October 15, 2012 (Helsel 2012a). Their inquiries led us to review comments made about the ProUCL4 software and its associated guidance documents (EPA 2007, 2009a, 2009b, 2010c, 2010d, and 2011) in the "The Unofficial Users Guide to ProUCL4" and in the webinar, "ProUCL v4: The Unofficial User Guide". These two documents collectively are referred to as the Unofficial ProUCLv4 User Guide in this ProUCL document. The pdf document describing the material presented in the Practical Stats Webinar (Helsel 2012a) was downloaded from the http://www.practicalstats.com website. In the "ProUCL v4: The Unofficial User Guide", comments have been made about the software and its guidance documents, therefore, it is appropriate to address those comments in the present ProUCL guidance document. It is necessary to provide the detailed response to assure that: 1) rigorous statistical methods are used to compute decision making statistics; and 2) the methods incorporated in ProUCL software are not misrepresented and misinterpreted. Some general responses and comments about the material presented in the webinar and in the Unofficial User Guide to ProUCLv4 are described as follows. Specific comments and responses are also considered in the respective chapters of ProUCL 5.1 (and ProUCL 5.0) guidance documents. Note: It is noted that the Kindle version of "ProUCL v4: Unofficial User Guide" is no longer available on Amazon. Several incorrect theoretical statements and statements misrepresenting ProUCL 4 were made in that Unofficial User Guide; therefore, a brief response to some of those statements has been retained in ProUCL 5.1 guidance documents. ProUCL is a freeware software package which has been developed under limited government funding to address statistical issues associated with various environmental site projects. Not all statistical methods (e.g., Levene test) described in the statistical literature have been incorporated in ProUCL. One should not compare ProUCL with commercial software packages which are expensive and not as user-friendly as the ProUCL software when addressing environmental statistical issues. The existing and some new statistical methods based upon the research conducted by ORD-NERL, EPA Las Vegas during the last couple of decades have been incorporated in ProUCL to address the statistical needs of various environmental site projects and research studies. Some of those new methods may not be available in text books, in the library of programs written in R-script, and in commercial software packages. However, those methods are described in detail in the cited published literature and also in the ProUCL Technical Guides (e.g., EPA 2007, 2009a, 2009b, 2010c, 2010d, and 2011). Even though for uncensored data sets, programs which compute gamma distribution based UCLs and UPLs are available in R Script, programs which compute a 95% UCL of mean based upon a gamma distribution on KM estimates are not as easily available. • In the Unofficial ProUCL v4 User Guide, several statements have been made about percentiles. There are several ways to compute percentiles. Percentiles computed by ProUCL may or may not be 62 ------- identical (don't have to be) to percentiles computed by NADA for R (Helsel 2013) or described in Helsel and Gilroy (2012). To address users' requests, ProUCL 4.1 (2011) and its higher versions compute percentiles that are comparable to the percentiles computed by Excel 2003 and higher versions. The literature search suggests that there are a total of nine (9) known types of percentiles, i.e., 9 different methods of calculating percentiles in statistics literature (Hyndman and Fan, 1996). The R programming language (R Core Team 2012) computes percentiles using those 9 methods using the following statement in R Quantile (x, p, type=k) where p = percentile, k = integer between 1-9 ProUCL computes percentiles using Type 7; Minitab 16 and SPSS compute percentiles using Type 6. It is simply a matter of choice, as there is no 'best' type to use. Many software packages use one type for calculating a percentile, and another for generating a box plot (Hyndman and Fan 1996). An incorrect statement "By definition, the sample mean has a 50% chance of being below the true population mean" has been made in Helsel and Gilroy (2012) and also in Helsel (2012a). The above statement is not correct for means of skewed distributions (e.g., lognormal or gamma) commonly occurring in environmental applications. Since Helsel (2012) prefers to use a lognormal distribution, the incorrectness of the above statement has been illustrated using a lognormal distribution. The mean and median of a lognormal distribution (details in Section 2.3.2 of Chapter 2 of ProUCL 5.1 Technical Guide) are given by: mean =jul = exp(ju + 0.5<72); and median =M = exp(//) From the above equations, it is clear that the mean of a lognormal distribution is always greater than the median for all positive values of a (sd of log-transformed variable). Actually the mean is greater than the pth percentile when a >2zp. For example, when p = 0.80, zp = 0.845, and mean of a lognormal distribution, y.\ exceeds XQ.SO, the 80th percentile when a > 1.69. In other words, when a > 1.69 the lognormal mean will exceed the 80th percentile of a lognormal distribution. Here zp represents the/?* percentile of the standard normal distribution (SND) with mean 0 and variance 1. To demonstrate the incorrectness of the above statement, a small simulation study was conducted. The distribution of sample means based upon samples of size 100 were generated from lognormal distributions with n =4, and varying skewness. The experiment was performed 10,000 times to generate the distributions of sample means. The probabilities of sample means less than the population means were computed. The following results are noted. Table 1-2. Probabilities p(x ------- The probabilities summarized in the above table demonstrate that the statement about the mean made in Helsel and Gilroy (2012) is incorrect. Graphical Methods: Graphical methods are available in ProUCL as exploratory tools which can be generated for both uncensored and left-censored data sets. Exploratory graphical methods are used to understand possible patterns present in data sets and not to compute statistics used in the decision making process. The Unofficial ProUCL Guide makes several comments about box plots and Q-Q plots incorporated in ProUCL. The Unofficial ProUCL Guide states that all graphs with NDs are incorrect. These statements are misleading and incorrect. The intent of the graphical methods in ProUCL is exploratory for the purpose of gaining information (e.g., outliers, multiple populations, data distribution, patterns, and skewness) about a data set. Based upon the data displayed (ProUCL displays a message [e.g., as a sub-title] in this regard) on those graphs, all statistics shown on those graphs generated by ProUCL are correct. Box Plots: In statistical literature, one can find several ways to generate box plots. The practitioners may have their own preferences to use one method over the other. All box plot methods including the one in ProUCL convey the same information about the data set (outliers, mean, median, symmetry, skewness). ProUCL uses a couple of development tools such as FarPoint spread (for Excel type input and output operations) and ChartFx (for graphical displays); and ProUCL generates box plots using the built-in box plot feature in ChartFx. For all practical and exploratory purposes, box plots in ProUCL are equally good (if not better) as those available in the various commercial software packages, for examining data distribution (skewed or symmetric), identifying outliers, and comparing multiple groups (main objectives of box plots in ProUCL). o As mentioned earlier, it is a matter of choice of using percentiles/quartiles to construct a box plot. There is no 'best' method for constructing a box plot. Many software packages use one method (out of 9 as specified above) for calculating a percentile, and another for constructing a box plot (Hyndman and Fan 1996). Q-Q plots: All Q-Q plots incorporated in ProUCL are correct and of high quality. In addition to identifying outliers, Q-Q plots are also used to assess data distributions. Multiple Q-Q plots are useful for performing point-by-point comparisons of grouped data sets, unlike box plots based upon the five- point summary statistics. ProUCL has Q-Q plots for normal, lognormal, and gamma distributions - not all of these graphical capabilities are directly available in other software packages such as NADA for R (Helsel 2013). ProUCL offers several exploratory options for generating Q-Q plots for data sets with NDs. Only detected outlying observations may require additional investigation; therefore, from an exploratory point of view, ProUCL can generate Q-Q plots excluding all NDs (and other options). Under this scenario there is no need to retain place holders (computing plotting positions used to impute NDs) as the objective is not to impute NDs. To impute NDs, ProUCL uses ROS methods (Gamma ROS and log ROS) requiring place holders; and ProUCL computes plotting positions for all detects and NDs to generate a proper regression model which is used to impute NDs. Also for comparison purposes, ProUCL can be used to generate Q-Q plots on data sets obtained by replacing NDs by their respective DLs or DL/2s. In these cases, no NDs are imputed, and there is no need to retain placeholders for NDs. On these Q-Q plots, ProUCL displays some relevant statistics which are computed based upon the data displayed on those graphs. Helsel (2012a) states that the Summary Statistics module does not display KM estimates and that statistics based upon logged data are useless. Typically, estimates computed after processing the data 64 ------- do not represent summary statistics. Therefore, KM and ROS estimates are not displayed in the Summary Statistics module. These statistics are available in several other modules including the UCL and BTV modules. At the request of several users, summary statistics are computed based upon logged data. It is believed that the mean, median, or standard deviation of logged data do provide useful information about data skewness and data variability. To test for the equality of variances, the F-test, as incorporated in ProUCL, performs fairly well and the inclusion of the Levene's (1960) test will not add any new capability to the ProUCL software. Therefore, taking budget constraints into consideration, Levene's test has not been incorporated in the ProUCL software. o Although it makes sense to first determine if the two variances are equal or unequal, this is not a requirement to perform a t-test. The t-distribution based confidence interval or test for /Hi - /Li2 based on the pooled sample variance does not perform better than the approximate confidence intervals based upon Satterthwaite's test. Hence testing for the equality of variances is not required to perform a two-sample t-test. The use of Welch-Satterthwaite's or Cochran's method is recommended in all situations (see Hayes 2005). Helsel (2012a) suggests that imputed NDs should not be made available to the users. The developers of ProUCL and other researchers like to have access to imputed NDs. As a researcher, for exploratory purposes only, one may want to have access to imputed NDs to be used in exploratory advanced methods such as multivariate methods including data mining, cluster and principal component analyses. It is noted that one cannot easily perform exploratory methods on multivariate data sets with NDs. The availability of imputed NDs makes it possible for researchers and scientists to identify potential patterns present in complex multivariate data by using data mining exploratory methods on those multivariate data sets with NDs. Additional discussion on this topic is considered in Chapter 4 of the ProUCL 5.1 Technical Guide. o The statements summarized above should not be misinterpreted. One may not use parametric hypothesis tests such as a t-test or a classical ANOVA on data sets consisting of imputed NDs. These methods require further investigation as the decision errors associated with such methods remain unqualified. There are other methods such as the Gehan and T-W tests in ProUCL 5.0/ProUCL 5.1 which are better suited to perform two-sample hypothesis tests using data sets with multiple detection limits. Outliers: Helsel (2012a) and Helsel and Gilroy (2012) make several comments about outliers. The philosophy (with input from EPA scientists) of the developers of ProUCL about the outliers in environmental applications is that those outliers (unless they represent typographical errors) may potentially represent impacted (site related or otherwise) locations or monitoring wells; and therefore may require further investigation. Moreover, decision statistics such as a UCL95 based upon a data set with outliers gets inflated and tends to represent those outliers instead of representing the population average. Therefore, a few low probability outliers coming from the tails of the data distribution may not be included in the computation of the decision making upper limits (UCLs, UTLs), as those upper limits get distorted by outliers and tend not to represent the parameters they are supposed to estimate. o The presence of outliers in a data set tends to destroy the normality of the data set. In other words, a data set with outliers can seldom (may be when outliers are mild, lying around the border of the central and tail parts of a normal distribution) follow a normal distribution. 65 ------- There are modern robust and resistant outlier identification methods (e.g., Rousseeuw and Leroy 1987; Singh and Nocerino 1995) which are better suited to identify outliers present in a data set; several of those robust outlier identification methods are available in the Scout 2008 version 1.0 (EPA 2009) software package. o For both Rosner and Dixon tests, it is the data set (also called the main body of the data set) obtained after removing the outliers (and not the data set with outliers) that needs to follow a normal distribution (Barnett and Lewis 1994). Outliers are not known in advance. ProUCL has normal Q-Q plots which can be used to get an idea about potential outliers (or mixture populations) present in a data set. However, since a lognormal model tends to accommodate outliers, a data set with outliers can follow a lognormal distribution; this does not imply that the outlier which may actually represent an impacted/unusual location does not exist! In environmental applications, outlier tests should be performed on raw data sets, as the cleanup decisions need to be made based upon values in the raw scale and not in log-scale or some other transformed space. More discussion about outliers can be found in Chapter 7 of the ProUCL 5.1 Technical Guide. In Helsel (2012a), it is stated, "Mathematically, the lognormal is simpler and easier to interpret than the gamma (opinion)." We do agree with the opinion that the lognormal is simpler and easier to use but the log-transformation is often misunderstood and hence incorrectly used and interpreted. Numerous examples (e.g., Example 2-1 and 2-2, Chapter 2 of ProUCL Technical Guide) are provided in the ProUCL guidance documents illustrating the advantages of the using a gamma distribution. It is further stated in Helsel (2012a) that ProUCL prefers the gamma distribution because it downplays outliers as compared to the lognormal. This argument can be turned around - in other words, one can say that the lognormal is preferred by practitioners who want to inflate the effect of the outlier. Setting this argument aside, we prefer the gamma distribution as it does not transform the variable so the results are in the same scale as the collected data set. As mentioned earlier, log- transformation does appear to be simpler but problems arise when practitioners are not aware of the pitfalls (e.g., Singh and Ananda 2002; Singh, Singh, and laci 2002) associated with the use of lognormal distribution. Helsel (2012a) and Helsel and Gilroy (2012) state that "lognormal and gamma are similar, so usually if one is considered possible, so is the other." This is another incorrect and misleading statement; there are significant differences in the two distributions and in their mathematical properties. Based upon the extensive experience in environmental statistics and published literature, for skewed data sets that follow both lognormal and gamma distributions, the developers favor the use of the gamma distribution over the lognormal distribution. The use of the gamma distribution based decision statistics is preferred to estimate the environmental parameters (mean, upper percentile). A lognormal model tends to hide contamination by accommodating outliers and multiple populations whereas a gamma distribution tends not to accommodate contamination (elevated values) as can be seen in Examples 2-1 and 2-2 of Chapter 2 of the ProUCL 5.1 Technical Guide. The use of the lognormal distribution on a data set with outliers tends to yield inflated and distorted estimates which may not be protective of human health and the environment; this is especially true for skewed data sets of small of sizes <20-30; the sample size requirement increases with skewness. o In the context of computing a UCL95 of mean, Helsel and Gilroy (2012) and Helsel (2012a) state that GROS and LROS methods are probably never better than the KM method. It should be noted that these three estimation methods compute estimates of mean and standard deviation and 66 ------- not the upper limits used to estimate EPCs and BTVs. The use of the KM method does yield good estimates of the mean and standard deviation as noted by Singh, Maichle, and Lee (2006). The problem of estimating mean and standard deviation for data sets with nondetects has been studied by many researchers as described in Chapter 4 of the ProUCL 5.1 Technical Guide. Computing good estimates of mean and sd based upon left-censored data sets addresses only half of the problem. The main issue is to compute decision statistics (UCL, UPL, UTL) which account for uncertainty and data skewness inherently present in environmental data sets. o Realizing that for skewed data sets, Student's t-UCL, CLT-UCL, and standard and percentile bootstrap UCLs do not provide the specified coverage to the population mean for uncensored data sets, many researchers (e.g., Johnson 1978; Chen 1995; Efron and Tibshirani 1993; Hall [1988, 1992]; Grice and Bain 1980; Singh, Singh, and Engelhardt 1997; Singh, Singh, and laci 2002) developed parametric (e.g., gamma) and nonparametric (e.g., bootstrap-t and Hall's bootstrap method, modified-t, and Chebyshev inequality) methods for computing confidence intervals and upper limits which adjust for data skewness. One cannot ignore the work and findings of the researchers cited above, and assume that Student's t-statistic based upper limits or percentile bootstrap method based upper limits can be used for all data sets with varying skewness and sample sizes. o Analytically, it is not feasible to compare the various estimation and UCL computation methods for skewed data sets containing ND observations. Instead, researchers use simulation experiments to learn about the distributions and performances of the various statistics (e.g., KM-t- UCL, KM-percentile bootstrap UCL, KM-bootstrap-t UCL, KM-Gamma UCL). Based upon the suggestions made in published literature and findings summarized in Singh, Maichle, and Lee (2006), it is reasonable to state and assume that the findings of the simulation studies performed on uncensored skewed data sets comparing the performances of the various UCL computation methods can be extended to skewed left-censored data sets. o Like uncensored skewed data sets, for left-censored data sets, ProUCL 5.0/ProUCL 5.1 has several parametric and nonparametric methods to compute UCLs and other limits which adjust for data skewness. Specifically, ProUCL uses KM estimates in gamma equations; in the bootstrap-t method, and in the Chebyshev inequality to compute upper limits for left-censored skewed data sets. • Helsel (2012a) states that ProUCL 4 is based upon presuppositions. It is emphasized that ProUCL does not make any suppositions in advance. Due to the poor performance of a lognormal model, as demonstrated in the literature and illustrated via examples throughout ProUCL guidance documents, the use of a gamma distribution is preferred when a data set can be modeled by a lognormal model and a gamma model. To provide the desired coverage (as close as possible) for the population mean, in earlier versions of ProUCL (version 3.0), in lieu of H-UCL, the use of Chebyshev UCL was suggested for moderately and highly skewed data sets. In later (3.00.02 and higher) versions of ProUCL, depending upon skewness and sample size, for gamma distributed data sets, the use of the gamma distribution was suggested for computing the UCL of the mean. Upper limits (e.g., UCLs, UPLs, UTLs) computed using the Student's t statistic and percentile bootstrap method (Helsel 2012, NADAfor R, 2013) often fail to provide the desired coverage (e.g., 95% confidence coefficient) to the parameters (mean, percentile) of most of the skewed environmental populations. It is suggested that the practitioners compute the decision making statistics (e.g., UCLs, UTLs) by taking: data 67 ------- distribution; data set size; and data skewness into consideration. For uncensored and left-censored data sets, several such upper limits computation methods are available in ProUCL 5.1 and its earlier versions. Contrary to the statements made in Helsel and Gilroy (2012), ProUCL software does not favor statistics which yield higher (e.g., nonparametric Chebyshev UCL) or lower (e.g., preferring the use of a gamma distribution to using a lognormal distribution) estimates of the environmental parameters (e.g., EPC and BTVs). The main objectives of the ProUCL software funded by the U.S. EPA is to compute rigorous decision statistics to help the decision makers and project teams in making sound decisions which are cost-effective and protective of human health and the environment. Cautionary Note: Practitioners and scientists are cautioned about: 1) the suggestions made about the computations of upper limits described in some recent environmental literature such as the NADA books (Helsel [2005, 2012]); and 2) the misleading comments made about the ProUCL software in the training courses offered by Practical Stats during 2012 and 2013. Unfortunately, comments about ProUCL made by Practical Stats during their training courses lack professionalism and theoretical accuracy. It is noted that NADA packages in R and Minitab (2013) developed by Practical Stats do not offer methods which can be used to compute reliable or accurate decision statistics for skewed data sets. Decision statistics (e.g., UCLs, UTLs, UPLs) computed using the methods (e.g., UCLs computed using percentile bootstrap, and KM and LROS estimates and t-critical values) described in the NADA books and incorporated in NADA packages do not take data distribution and data skewness into consideration. The use of statistics suggested in NADA books and in Practical Stats training sessions often fail to provide the desired specified coverage to environmental parameters of interest for moderately skewed to highly skewed populations. Conclusions derived based upon those statistics may lead to incorrect conclusions which may not be cost-effective or protective of human health and the environment. Page 75 (Helsel [20121): One of the reviewers of the ProUCL 5.0 software drew our attention to the following incorrect statement made on page 75 of Helsel (2012): "If there is only 1 reporting limit, the result is that the mean is identical to a substitution of the reporting limit for censored observations." An example of a left-censored data set containing ND observations with one reporting limit of 20 which illustrates this issue is described as follows. Y D_y 20 0 20 0 20 0 7 1 58 1 92 1 100 1 72 1 11 1 27 1 The mean and standard deviation based upon the KM and two substitution methods: DL/2 and DL are summarized as follows: 68 ------- Kaplan-Meier (KM) Statistics Mean 39.4 SD 35.56 DL Substitution method (replacing censored values by the reporting limit) Mean 42.7 SD 34.77 DL/2 Substitution method (replacing NDs by the reporting limit) Mean 39.7 SD 37.19 The above example illustrates that the KM mean (when only 1 detection limit is present) is not actually identical to the mean estimate obtained using the substitution, DL (RL) method. The statement made in Helsel's text (and also incorrectly made in his presentations such as the one made at the U.S. EPA 2007 National Association of Regional Project Managers (NARPM) Annual Conference: http://www.ttemidev.com/narpm2007Admin/conference/) holds only when all observations reported as detects are greater than the single reporting limit, which is not always true for environmental data sets consisting of analytical concentrations. 1.16 Box and Whisker Plots At the request of ProUCL users, a brief description of box plots (also known as box and whisker plots) as developed by Tukey (Hoaglin, Mosteller and Tukey 1991) is provided in this section. A box and whiskers plot represents a useful and convenient exploratory tool and provides a quick five-point summary of a data set. In statistical literature, one can find several ways to generate box plots. The practitioners may have their own preferences to use one method over the other. Box plots are well documented in the statistical literature and description of box plots can be easily obtained by surfing the net. Therefore, the detailed description about the generation of box plots is not provided in ProUCL guidance documents. ProUCL also generates box plots for data set with NDs. Since box plots are used for exploratory purposes to identify outliers and also to compare concentrations of two or more groups, it does not really matter how NDs are displayed on those box plots. ProUCL generates box plots using detection limits and draws a horizontal line at the highest detection limit. Users can draw up to four horizontal lines at other levels (e.g., a screening level, a BTV, or an average) of their choice. All box plot methods, including the one in ProUCL, represent five-point summary graphs including: the lowest and the highest data values, median (50th percentile=second quartile, Q2), 25th percentile (lower quartile, Ql), and 75th percentile (upper quartile, Q3). A box and whisker plot also provides information about the degree of dispersion (interquartile range (IQR) = Q3-Ql=length/height of the box in a box plot), the degree of skewness (suggested by the length of the whiskers) and unusual data values known as outliers. Specifically, ProUCL (and other software packages) use the following to generate a box and whisker plot. • Q1= 25th percentile, Q2= 50th (median), and Q3 = 75th percentile • Interquartile range= IQR = Q3-Q1 (the length/height of the box in a box plot) • Lower whisker starts at Q1 and the upper whisker starts at Q3. • Lower whisker extends up to the lowest observation or (Ql - 1.5 * IQR) whichever is higher • Upper whisker extends up to the highest observation or (Q3 + 1.5* IQR) whichever is lower 69 ------- • Horizontal bars (also known as fences) are drawn at the end of whiskers • Guidance in statistical literature suggests that observations lying outside the fences (above the upper bar and below the lower bar) are considered potential outliers An example box plot generated by ProUCL is shown in the following graph. Box Plot for Lead Box Plot with Fences and Outlier It should be pointed out that the use of box plots in different scales (e.g., raw-scale and log-scale) may lead to different conclusions about outliers. Below is an example illustrating this issue. Example 1-2. Consider an actual data set consisting of copper concentrations collected a Superfund Site. The data set is: 0.83, 0.87, 0.9, 1, 1, 2, 2, 2.18, 2.73, 5, 7, 15, 22, 46, 87.6, 92.2, 740, and 2960. Box plots using data in the raw-scale and log-scale are shown in Figures 1-1 and 1-2. 70 ------- Box Plot for Copper Figure 1-1. Box Plot of Raw Data in Original Scale Based upon the last bullet point of the description of box plots described above, from Figure 1-1, it is concluded that two observations 740 and 2960 in the raw scale represent outliers. Box Plot for In(copper) Figure 1-2. Box Plot of Data in Log-Scale However, based upon the last bullet point about box plots, from Figure 1-2, it is concluded that two observations 740 and 2960 in the log-scale do not represent outliers. The log-transformation has accommodated the two outliers. This is one of the reasons ProUCL guidance suggests avoiding the use of O OO O log-transformed data. The use of a log-transformation tends to hide/accommodate outliers/contamination. 71 ------- Note: ProUCL uses a couple of development tools such as FarPoint spread (for Excel type input and output operations) and ChartFx (for graphical displays). ProUCL generates box plots using the built-in box plot feature in ChartFx. The programmer has no control over computing various statistics (e.g., Ql, Q2, Q3, IQR) using ChartFx. So box plots generated by ProUCL can differ slightly from box plots generated by other programs (e.g., Excel). However, for all practical and exploratory purposes, box plots in ProUCL are equally good (if not better) as available in the various commercial software packages for investigating data distribution (skewed or symmetric), identifying outliers, and comparing multiple groups (main objectives of box plots). Precision in Box Plots: Box plots generated using ChartFx round values to the nearest integer. For increased precision of graphical displays (all graphical displays generated by ProUCL), the user can use the process described as follows. Position your cursor on the graph and right-click, a popup menu will appear. Position the cursor on Properties and right-click; a windows form labeled Properties will appear. There are three choice at the top: General, Series and Y-Axis. Position the e cursor over the Y-Axis choice and left-click. You can change the number of decimals to increase the precision, change the step to increase or decrease the number Y-Axis values displayed and/or change the direction of the label. To show values on the plot itself, position your cursor on the graph and right-click; a pop-up menu will appear. Position the cursor on Point Labels and right-click. There are other options available in this pop-up menu including changing font sizes. 72 ------- 73 ------- Chapter 2 Entering and Manipulating Data 2.1 Creating a New Data Set By executing ProUCL 5.1, the following file options will appear (the title will show ProUCL 5.1 instead ofProUCL5.0): File Edit Stats/Sample Sizes BIS Simulator Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation Panel Name By choosing the File ^- New option, a new worksheet shown below will appear. The user enters variable names and data following the ProUCL input file format requirements described in Section 2.3. VorkSheet File Edit Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation Panel Name Worksheet jds o 1 2 3 4 5 6 7 8 9 10 11 T 2 3 2.2 Opening an Existing Data Set The user can open an existing worksheet (*.xls, *.xlsx, *.wst, and *.ost) by choosing the File ^- Open Single File Sheet option. The following drop down menu will appear: File Edit Stats/Sample Sizes ISM Simulator Graphs Statistical Tests Upper Limits/BTVs New Open Single File Sheet Exit Open Excel File with Multiple Sheets Opens First Sheet in an Excel File or an Output or Older ProUCL (,WST) File 74 ------- t i, > Anita Singh > Desktop > ProUCL 5,0 K Organize » New folder .Favorites * Na,me Date modified H E Desktop H £ Downloads | test-data7-G ] test-data6-G 7/31/2013 7:58 PM or, D l§] test-data5-G 7/31/2013 7:55 PM ' Recent places a i C Desktop H j-a Libraries Q Documents s ,^ Music n B Pictures 8 Videos *^ Homegroup E Anita Singh '^ Comouter Filename: ] test-data4-G ] test-data3-G 7/31/2013 7:53 PM ] Tert-data1-G 7/31/2013 7:51 PM ] Test-data2-G 7/31/2013 7:50 PM ] Basic Calc for 20 example 08Feb13 5/25/2013 6:38 PM ] Adv Calc for 50 example 08Feb13 5/25/2013 6:37 PM ] DU4 data-bkgd metals-drom deana 5/16/2013 7:07 AM proucl-review-comments-7-2013 8/9/20139:41 PM Data-ProUCL 5.0 8/9/201 3 8:35 AM V Search ProUCL 5.0 fl E» m ® Type Size Microsoft Office E... 20KB Microsoft Office E... 18KB Microsoft Office E.,, 19KB Microsoft Office E.,. 18KB Microsoft Office E.,. 18KB Microsoft Office E.,. 19KB Microsoft Office E... 20KB Microsoft Office E... 888KB Microsoft Office E,,, 1,000KB Microsoft Office E... 41KB Filefolder File folder v [ Excel Files (,xls) BHJBIHBHBM • |l| Ll I Excel Files (.xlsx) Worksheet files (*.wst) 1 output files (*-ost) Choose a file by high lighting the type of file such as .xls as shown above. This option can also be used to read in a *.wst worksheet and *.ost output sheet generated by earlier versions (e.g., ProUCL 4.1 and older) of ProUCL. By choosing the File ^- Excel Multiple Sheets option, the user can open an Excel file consisting of multiple sheets. Each sheet will be opened as a separate file to be processed individually by ProUCL 5.1 Caution: If you are editing a file (e.g., an excel file using Excel), make sure to close the file before importing the file into ProUCL using the file open option. 2.3 Input File Format • The program can read Excel files. The user can perform typical Cut, Paste, and Copy operations available under the Edit Menu Option as shown below. a^ File Nav Name PCA-NDs SuperFun Edit Stats/Sample Sizes ISM Graphs Statistical Test: Configure Display > Cut Ctrl+X Copy Ctrl+C Paste Ctrl+V Header Name Full Precision w1 Log Panel *' Navigation Panel Excel 2003 75 ------- • The first row in all input data files consist of alphanumeric (strings of numbers and characters) names representing the header row. Those header names may represent meaningful variable names such as Arsenic, Chromium, Lead, Group-ID, and so on. o The Group-ID column holds the labels for the groups (e.g., Background, AOC1, AOC2, 1, 2, 3, a, b, c, Site 1, Site 2) that might be present in the data set. Alphanumeric strings (e.g., Surface, Sub-surface) can be used to label the various groups. Most of the modules of ProUCL can process data by a group variable. o The data file can have multiple variables (columns) with unequal numbers of observations. Most of the modules of ProUCL can process data by a group variable. o Except for the header row and columns representing the group labels, only numerical values should appear in all other rows. o All alphanumeric strings and characters (e.g., blank, other characters, and strings), and all other values (that do not meet the requirements above) in the data file are treated as missing values and are omitted from statistical evaluations. o Also, a large value denoted by 1E31 (= IxlO31) can be used to represent missing data values. All entries with this value are ignored from the computations. These values are counted under the number of missing values. 2.4 Number Precision • The user may turn "Full Precision" on or off by choosing Configure ^- Full Precision On/OFF • By leaving "Full Precision" turned off, ProUCL will display numerical values using an appropriate (default) decimal digit option; and by turning "Full Precision" off, all decimal values will be rounded to the nearest thousandths place. • The "Full Precision" on option is specifically useful when dealing with data sets consisting of small numerical values (e.g., < 1) resulting in small values of the various estimates and test statistics. These values may become so small with several leading zeros (e.g., 0.00007332) after the decimal. In such situations, one may want to use the "Full Precision" on option to see nonzero values after the decimal. Note: For the purpose of this User Guide, unless noted otherwise, all examples have used the "Full Precision " OFF option. This option prints out results up to 3 significant digits after the decimal. 76 ------- 2.5 Entering and Changing a Header Name Stats/Sample Size Configure Display Cut Ctrl+X Copy Ctrl + C Paste Ctrl+V Header Name 1. The user can change variable names (Header Name) using the following process. Highlight the column whose header name (variable name) you want to change by clicking either the column number or the header as shown below. Arsenic 1 4.5! "5.6 4.3 5.4 Right-click and then click Header Name. 0 1 2 I Header Name I 2 5.6 3 4 4.3 5.4 Change the Header Name. Header Name Header Name: [Arsenic Site 1 OK Cancel 77 ------- 4. Click the OK button to get the following output with the changed variable name. 2.6 Saving Files File Edit Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs New Open Single File Sheet Open Excel File with Multiple Sheets Close Save Save As... Print Print Preview Exit The Save option allows the user to save the active window in Excel 2003 or Excel 2007. The Save As option also allows the user to save the active window. This option follows typical Windows standards, and saves the active window to a file in .xls or .xlsx format. All modified/edited data files, and output screens (excluding graphical displays) generated by the software can be saved as .xls or .xlsx files. 78 ------- 2.7 Editing Click on the Edit menu item to reveal the following drop-down options. Edit Stats/Sample Sizes Configure Display Cut Ctrl+X Copy Ctrl+C Paste Ctrl+V Header Name • Cut option: similar to a standard Windows Edit option, such as in Excel. It performs standard edit functions on selected highlighted data (similar to a buffer). • Copy option: similar to a standard Windows Edit option, such as in Excel. It performs typical edit functions on selected highlighted data (similar to a buffer). Paste option: similar to a standard Windows Edit option, such as in Excel. It performs typical edit functions of pasting the selected (highlighted) data to the designated spreadsheet cells or area. 2.8 Handling Nondetect Observations and Generating Files with Nondetects • Several modules of ProUCL (e.g., Statistical Tests, Upper limits/BTVs, UCLs/EPCs) handle data sets containing ND observations with single and multiple DLs. • The user informs the program about the status of a variable consisting of NDs. For a variable with ND observations (e.g., arsenic), the detected values, and the numerical values of the associated detection limits (for less than values) are entered in the appropriate column associated with that variable. No qualifiers or flags (e.g., J, B, U, UJ, X) should be entered in data files with ND observations. • Data for variables with ND values are provided in two columns. One column consists of numerical values of detected observations and numerical values of detection limits (or reporting limits) associated with observations reported as NDs; and the second column represents their detection status consisting of only 0 (for ND values) and 1 (for detected values) values. The name of the corresponding variable representing the detection status should start with d_, or D_ (not case sensitive) and the variable name. The detection status column with variable name starting with a D_ (or a d_) should have only two values: 0 for ND values, and 1 for detected observations. • For example, the header name, D_Arsenic is used for the variable, Arsenic having ND observations. The variable D_Arsenic contains a 1 if the corresponding Arsenic value represents a detected entry, and contains a 0 if the corresponding entry represents a ND entry. If this format is not followed, the program will not recognize that the data set has NDs. An 79 ------- example data set illustrating these points is given as follows. ProUCL does not distinguish between lowercase and uppercase letters. B! D:\example.wst QOIXl 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 nil 0 1 2 Arsenic D_Arsenic Mercury 4.5 0 0.07 5.6 1 0.07 4.3] o| 0.11 5.4 TT~ 0.2 9.2 1 0.61 6.2 1 0.12 6.7 1 0.04 5.8 1 0.06 8.5 0.99 5.65 0.125 5.4 0.18 5.5 | 0.21 5.9 I 0.29 5.1 TT~ 0.44 5.2 1 0.12 4.5 0| 0.055 6.1 T| 0.055 6.1 1 0.21 6.8 Tl 0.67 si i 0.1 0.8 0.26 0.97 0.05 0.26 3 D_Mercury 1 1 0 0 1 1 1 1 f T~ 1 1 1 1 T~ 1 1 T~ T" 1 1 1 4 Vanadium 16.4 1E.8 17.2 19.4 15.3 30.8 29.4 13.8 18.9 17.25 17.2 16.3 16.8 17.1 10.3 15.1 24.3 18 16.9 12 1 Tl 5 Zinc 89.3 90.7 95.5 113 266 80.9 80.4 89.2 182 80.4 91.9 G nrj Group Surface Surface Surface Surface Surface Surface Surface Surface Surface Surface Subsurfac 112 Subsurfac 172 Subsurfac 99 Subsurfac 90.7 Subsurfac 66.3 Subsurfac 175 Subsurface 185 184 68.4 Subsurface Subsurface Subsurface - ^ 2.9 Caution • Care should be taken to avoid any misrepresentation of detected and nondetected values. Specifically, do not include any missing values (blanks, characters) in the D_column (detection status column). If a missing value is located in the D_column (and not in the associated variable column), the corresponding value in the variable column is treated as a ND, even if this might not have been the intention of the user. • It is mandatory that the user makes sure that only a 1 or a 0 are entered in the detection status D_column. If a value other than a 0 or a 1 (such as qualifiers) is entered in the D_ column (the detection column), results may become unreliable, as the software defaults to any number other than 0 or 1 as a ND value. • When computing statistics for full uncensored data sets without any ND values, the user should select only those variables (from the list of available variables) that contain no ND observations. Specifically, ND values found in a column chosen for the summary statistics (full-uncensored data set) will be treated as a detected value; whatever value (e.g., detection limit) is entered in that column will be used to compute summary statistics for a full- uncensored data set without any ND values. • It is mandatory that the header name of a nondetect column associated with a variable such as XYZ should be D_XYZ (or d_Xyz). No other characters or blanks are allowed. However, the header (column) names are not case sensitive. If the nondetect column is not labeled properly, methods to handle nondetect data will not be activated and shown. 80 ------- • Two-Sample Hypotheses: When using two-sample hypotheses tests (WMW test, Gehan test, and T-W test) on data sets with NDs, both samples or variables (e.g., site-As, Back-As) should be specified as having NDs, even though one of the variables may not have any ND observations. This means that a ND column (with 0 = ND, and 1 = detect) should be provided for each variable (here D_site-As, and D_Back-As) to be used in this comparison. If a variable (e.g., site-As) does not have any NDs, still a column with label D_site-As should be included in the data set with all entries = 1 (detected values). • The sample data set given on the previous page illustrates points related to this option and issues listed above. The data set contains some ND measurements for arsenic and mercury. It should be noted that mercury concentrations are used to illustrate the points related to ND observations; arsenic and zinc concentrations are used to illustrate the use of the group variable, Group (Surface, Subsurface). • If for mercury, one computes summary statistics (assuming no ND values) using "Full" data set option, then all ND values (with "0" entries in D_Mercury column) will be treated as detected values, and summary statistics will be computed accordingly. 2.10 Summary Statistics for Data Sets with Nondetect Observations • To compute statistics of interest (e.g., background statistics, GOF test, UCLs, WMW test) for variables with ND values, one should choose the ND option, With NDs, from the available menu options such as Stats/Sample Sizes, Graphs, Statistical Tests, Upper Limits/BTVs, and UCLs/EPCs. • The NDs option of these modules gets activated only when your data set contains NDs. • For data sets with NDs, the Stats/Sample Sizes module of ProUCL 5.0 computes summary statistics and other general statistics such as the KM mean and KM standard deviation based upon raw as well as log-transformed data. •a File Edit Navigation F Name Worksheet jds Well 10jds WMW-with NDs:xl Stats/Sample Sizes Graphs Statistical Tests General Statistics K Imputed NDs using ROS Methods > DQOs Based Sample Sizes > 3 2 3 4 — 5 7 — 8 17 Upper Limits/BTVs UCLs/EPCs Windows Help Full (w/o NDs) > With NDs > 0 1 D 0 0 1 5 6 7 The General Statistics/With NDs option also provides simple statistics (e.g., % NDs, Max detect, Min detect, Mean) based upon detected values. The statistics computed in log-scale (e.g., sd of log-transformed detected values) may help a user to determine the degree of skewness (e.g., mild, moderate, high) of a data set based upon detected values. These statistics may also help the user to choose the most appropriate method (e.g., KM bootstrap-t UCL or KM percentile bootstrap UCL) to compute UCLs, UPLs, and other limits used to compute decision statistics. 81 ------- • All other parametric and nonparametric statistics and estimates of population mean, variance, percentiles (e.g., KM, and ROS estimates) for variables with ND observations are provided in other menu options such as Upper Limits/BTVs and UCLs/EPCs. 2.11 Warning Messages and Recommendations for Data Sets with an Insufficient Amount of Data • ProUCL provides warning messages and recommendations for data sets with an insufficient amount of data for calculating meaningful estimates and statistics of interest. For example, it is not desirable to compute an estimate of the EPC term based upon a discrete (as opposed to composite or ISM) data set of size less than 5, especially when NDs are also present in the data set. • However, to accommodate the computation of UCLs and other limits based upon ISM data sets, ProUCL 5.0 allows users to compute UCLs, UPLs, and UTLs based upon data sets of sizes as small as 3. The user is advised to follow the guidance provided in the ITRC ISM Technical Regulatory Guidance Document (2012) to select an appropriate UCL95 to estimate the EPC term. Due to lower variability in ISM data, the minimum sample size requirements for statistical methods used on ISM data are lower than the minimum sample size requirements for statistical methods used on discrete data sets. • It is suggested that for data sets composed of observations resulting from discrete sampling, at least 10 observations should be collected to compute UCLs and various other limits. • Some examples of data sets with insufficient amount of data include data sets with less than 3 distinct observations, data sets with only one detected observation, and data sets consisting of all nondetects. • Some of the warning messages generated by ProUCL 5.0 are shown as follows. UCL Statistics for Uncensored Full Data Sets User Selected Options Date/Time of Computation 3/13/2013 9:28:43 PM From Rle Not-enough-data-setjds Full Precision OFF Confidence Coefficient 95% Number of Bootstrap Operations 2000 General Statistics Total Number of Observations 2 Number of Distinct Observations 2 Number of Hissing Observations D Minimum 2 Mean 4.5 Maximum 7 Median 4.5 Warning: This data set only has 2 observations! Data set is too small to compute reliable and meaningful statistics and estimates! The data set for variable x was not processed! It is suggested to collect at least 8 to 10 observations before using these statistical methods! If possible, compute and collect Data Quality Objectives (DQO) based sample size and analytical results. 82 ------- UCL Statistics (or Data Sets with Nor, Detects User Selected Options Date/Time of Computation 3/13/2013 9:27:39 PM From Rle Not-enough-data-setjds OFF 95% 2000 Full Precision Confidence Coefficient Number of Bootstrap Operations General Statistics Total Number of Observations Number of Detects Number of Distinct Detects Minimum Detect Maximum Detect Variance Detects Mean Detects Median Detects Skewness Detects Mean of Logged Detects 2 2 10 13 4.5 N 11.5 11.5 N/A 2/34 Number of Distinct Observations Number of Non-Detects Number of Distinct Non-Detects Minimum Non-Detect Maximum Non-Detect Percent Non-Detects SD Detects CV Detects Kurtosis Detects SD of Logged Detects Warning: Data set has only 2 Detected Values. This is not enough to compute meaningful or reliable statistics and estimates. 5 71.43% 2.121 0.184 N/A 0.186 Normal GOF Test on Detects Only Not Enough Data to Perform GOF Tes User Selected Options From Rle Full Precision OFF Confidence Coefficient 9K Coverage Different or Future K Observations Number of Bootstrap Operations Background Statistics for Data Sets with Nor Detects Not-enough-data-set_ajtls 95% 2000 Total Number of Observations Number of Distinct Observations Number of Detects Number of Distinct Detects Minimum Detect Maximum Detect Variance Detected Mean Detected Mean of Detected Logged Data General Statistics 7 6 0 0 N/A N/A N/A N/A N/A Number of Missing Observations Number of Non-Detects Number of Distinct Non-Detects Minimum Non-Detect Maximum Non-Detect Percent Non-Detects SD Detected SD of Detected Logged Data 13 100% N/A N/A Warning: All observations are Non-Detects (NDs). therefore all statistics and estimates should also be NDs! Specifically, sample mean. UCLs. UPU. and other statistics are also NDs lying below the largest detection limit! The Project Team may decide to use alternative site specific values to estimate environmental parameters (e.g.. EPC. BTV). The data set for variable yy was not processed! 83 ------- 2.12 Handling Missing Values • The modules (e.g., Stats, GOF, UCLs, BTVs, Regression, Trend tests) of ProUCL 5.0 can handle missing values within a data set. Appropriate messages are displayed when deemed necessary. • All blanks, alphanumeric strings (except for group variables), or the specific large value Ie31 are considered as missing values. • A group variable (representing two or more groups, populations, MWs) can have alphanumeric values (e.g., MW01, MW02, AOC1, AOC2). • ProUCL ignores all missing values in all statistical evaluations it performs. Missing values are therefore not treated as being part of a data set. • Number of Valid Samples or Number of Valid Observations represents the Total Number of Observations minus the Number of Missing Values. If there are no missing values, then number of valid samples = total number of observations. Valid Samples = Total Number of Observations - Missing Values. • It is important to note, however, that if a missing value not meant (e.g., a blank, or Ie31) to represent a group category is present in a "Group" variable, ProUCL 5.1/ProUCL 5.0 will treat that blank value (or Ie31 value) as a new group. All variables and values that correspond to this missing value will be treated as part of a new group and not with any existing groups. It is therefore important to check the consistency and validity of all data sets before performing statistical evaluations. • ProUCL prints out the number of missing values (if any) and the number of reported values (excluding the missing values) associated with each variable in the data sheet. This information is provided in several output sheets (e.g., General statistics, BTVs, UCLs, Outliers, OLS, Trend Tests) generated by ProUCL 5.1. • Number of missing values in Regression: The OLS module also handles the number of missing values in the two columns (X and Y) representing independent (X) and dependent (Y) variables. ProUCL provides warning messages for bad data sets (e.g., all identical values) when statistics of interest cannot be computed. However, a bad/extreme data set can occur in numerous different ways, and ProUCL may not cover all of those extreme/bad data sets. In such cases, ProUCL may still yield an error message. The user needs to review and fix his data set before performing regression or trend analysis again. For further clarification of labeling missing values, the following example illustrates the terminology used for the number of valid samples and of unique and distinct samples on output sheets generated by the ProUCL software. Example: The following example illustrates the notion of Valid Samples, Unique or Distinct Samples, and Missing Values. The data set also has ND values. ProUCL 5.0 computes these numbers and prints them on the UCLs and background statistics output. 84 ------- X 2 4 2.3 1.2 w34 l.OE+031 anm 34 23 0.5 0.5 2.3 2.3 2.3 34 73 D x 1 1 1 0 0 0 0 0 1 1 0 0 1 1 1 1 1 Valid Samples: Represents the total number of observations (censored and uncensored inclusive) excluding the missing values. In this case the number of valid samples = 9. If a data set has no missing value, then the total number of data points equals number of valid samples. Missing Values: All values not representing a real numerical number are treated as missing values. Specifically, all alphanumeric values including blanks are considered to be missing values. Big numbers such as 1.0e31 are also treated as missing values and are considered as not valid observations. In the example above the number of missing values = 4. Unique or Distinct Samples: The number of unique samples or number of distinct samples represents all unique (or distinct) detected and nondetected values. This is computed separately for detects and NDs. This number is especially useful when using bootstrap methods. As well known, it is not desirable and advisable to use bootstrap methods, when the number of unique samples is small. In the example above total number of unique or distinct samples = 8, number of distinct detects = 6, and number of distinct NDs (with different detection limits) = 2. G eneral S tatistks Total Number of Observations 13 Number of Detects 10 Number of Distinct Detects 6 Minimum Detect 2 Maximum Detect 73 Variance Detects 555.5 Number of Distinct Observations 8 Number of Missing Observations 4 Number of Non-Detects 3 Number of Distinct Non-Detects 2 M inimum N on-D elect 0.5 M aximum N on-D etect 1.2 Percent N on-D elects 23.08% 85 ------- 2.13 User Graphic Display Modification Advanced users are provided two sets of tools to modify graphics displays. A graphics tool bar is available above the graphics display; the user can right-click on the desired object within the graphics display, and a drop-down menu will appear. The user can select an item from the drop-down menu list by clicking on that item. This will allow the user to make modifications as available for the selected menu item. An illustration is given as follows. 2.13.1 Graphics Tool Bar for : Arsenic NROS Arsenic The user can change fonts, font sizes, vertical and horizontal axis's, select new colors for the various features and text. All these actions are generally used to modify the appearance of the graphic display. The user is cautioned that these tools can be unforgiving and may put the user in a situation where the user cannot go back to the original display. Users are on their own in exploring the robustness of these tools. Therefore, less experienced users may not want to use these drop-down menu graphic tools. 2.13.2 Drop-Down Menu Graphics Tools Graphs can be modified by using the options shown on the two graphs displayed below. These tools allow the user to move the mouse to a specific graphic item like an axis label or a display feature. The user then right-clicks their mouse and a drop-down menu will appear. This menu presents the user with available options for that particular control or graphic object. For example, the user can change colors, title name, axes labels, font size, and re-size the graphs. There is less chance of making an unrecoverable error but that risk is always present. As a cautionary note, the user can always delete the graphics window and redraw the graphical displays by repeating their operations from the datasheet and menu options available in ProUCL. A couple of examples of a drop-down menu obtained by right-clicking the mouse on the background area of the graphics display are given as follows. 86 ------- ! Histo_Group.gst at m Histograms for Arsenic, NROS_Arsenic O 2 0 Properties... 51 Statistical Studies - n x mj File Edit Slau>'Sample Suet BIS Simulator Graphs Statistical Tens Upper Ltmits/BTVs UCLtlPCs Windows Help Box Plot for Na Ben PW Ful.g* Bo«Pto!FUl.a.9« Bo.FWUJ.9* G:7DB50FH T1C-35PM G:7inBPM >fr*oiTOton] Box Ptt Ful&aph G7113BPH 87 ------- Chapter 3 Select Variables Screen 3.1 Select Variables Screen • The Select Variable screen is associated with all modules of ProUCL. • Variables need to be selected to perform statistical analyses. • When the user clicks on a drop-down menu for a statistical procedure (e.g., UCLs/EPCs), the following window will appear. Available Variables Selected Variables Name ID Aluminum Chromium Iron Manganese Thallium Vanadium Benzo lajpynene S Naphthalene 5 Benzo[a)pyrene ... 10 Name Arsenic Lead ID 1 4 Select Group Column (Optional) The Options button is available in certain menus. The use of this option leads to another pop- up window such as shown below. This window provides the options associated with the selected statistical method (e.g., BTVs, OLS Regression). Enter BTV level Confidence Level Coverage 0.95 Different or Future K Observations 1 Number of Bootstrap Operations 2000 88 ------- Display Intervals Confidence Level 0.95 0 Display Regression Table EH Display Diagnostics Graphics Options 0 Display XY Plot XT Plot Title Classical Regression | 0 Display Confidence Interval 0 Display Prediction Interval ProUCL can process multiple variables simultaneously. ProUCL software can generate graphs, and compute UCLs, and background statistics simultaneously for all selected variables shown in the right panel of the screen shot displayed on the previous page. If the user wants to perform statistical analysis on a variable (e.g., manganese) by a Group variable, click the arrow below the Select Group Column (Optional) to get a drop-down list of available variables from which to select an appropriate group variable. For example, a group variable (e.g., Well ID) can have alphanumeric values such as MW8, MW9, and MW1. Thus in this example, the group variable name, Well ID, takes 3 values: MW1, MW8, and MW9. The selected statistical method (e.g., GOF test) performs computations on data sets for all the groups associated with the selected group variable (e.g., Well ID) Available Variables Selected Variables Name ID Well ID 0 MW-ID 2 Manganese 3 MW-89 5 GW-Mn-89 6 MW9 B MN9 9 MN-99 11 index 14 < > » « Name ID Mn-GW 1 < > Select Group Column (Optional) I Options | OK Cmcd | 89 ------- The Group variable is useful when data from two or more samples need to be compared. Any variable can be a group variable. However, for meaningful results, only a variable, that really represents a group variable (categories) should be selected as a group variable. The number of observations in the group variable and the number observations in the selected variables (to be used in a statistical procedure) should be the same. In the example below, the variable "Mercury" is not selected because the number of observations for Mercury is 30; in other words mercury values have not been grouped. The group variable and each of the selected variables have 20 data values. Select Variables Available Variables Selected Variables Name Areenic Iron Lead Thallium Vanadium Benzoiajpyrene Naphthalene Benzo[a)pyTene .. ID 1 3 4 6 7 8 9 10 Name Aluminum Chromium Manganese ID 0 2 5 Select Group Column (Optional) Options OK Cancel • As mentioned earlier, one should not assign any missing value such as a "Blank" for the group variable. If there is a missing value (represented by blanks, strings or 1E31) for a group variable, ProUCL will treat those missing values as a new group. As such, data values corresponding to the missing Group will be assigned to a new group. • The Group Option is a useful tool for performing statistical tests and methods (including graphical displays) separately for each of the group (samples from different populations) that may be present in a data set. For example, the same data set may consist of samples from multiple populations. The graphical displays (e.g., box plots, Q-Q plots) and statistics of interest can be computed separately for each group by using this option. Notes: Once again, care should be taken to avoid misrepresentation and improper use of group variables. Do not assign any form of a missing value for the group variable. 3.1.1 Graphs by Groups The following options are available to generate graphs by groups. 90 ------- Graphs Statistical Tests U| Box Plot Multiple Box Plots Histogram Q-Q Plots Multiple Q-Q Plots Individual or multiple graphs (Q-Q plots, box plots, and histograms) can be displayed on a graph by selecting the Group Column (Optional) option shown as follows. Available Variables Name ID CuLT=1 1 Zn 2 ZnLT=1 3 Zone 4 Zone-Basin 5 Zn-Basin 10 Zone-Allu 13 Zn-Allu 14 nip-Zn-allu-7M) 17 <• > » « i | Options | Selected Variables Name ID Cu 0 Select Group Column (Option, Zone (Count = 118) v OK | CaiceJ | J) An individual graph for each group (specified by the selected group variable) is produced by selecting the Individual Graph option; and multiple graphs (e.g., side-by-side box plots, multiple Q-Q plots on the same graph) are produced by selecting the Group Graph option as shown below. Using the Group Graph option, multiple graphs are displayed for all sub-groups included in the Group variable. This option is used when data are given in the same column and are classified by a group variable. 91 ------- Graphs by Groups O Individual Graphs Label 1 a \2 2 a \2 3 a \2 4 a • Group Graphs Value • Multiple graphs for selected variables are produced by selecting options: Multiple Box Plots or Multiple Q-Q Plots. Using the Group Graph option, multiple graphs for all selected variables are shown on the same graphical display. This option is useful when data (e.g., site lead and background lead) to be compared are given in different columns. Notes: It should be noted that it is the users' responsibility to provide an adequate amount of detected data to perform the group operations. For example, if the user desires to produce a graphical Q-Q plot (using only detected data) with regression lines displayed, then there should be at least two detected points (to compute slope, intercept, and sd) in the data set. Similarly, if graphs are desired for each group specified by a Group ID variable, there should be at least two detected observations in each group specified by the Group ID variable. ProUCL displays a warning message (in orange) in the lower Log Panel of the ProUCL screen when not enough data are available to perform a statistical or graphical operation. 92 ------- Chapter 4 General Statistics The General Statistics option is available under the Stats/Sample Sizes module of ProUCL 5.0. This option is used to compute general statistics including simple summary statistics (e.g., mean, standard deviation) for all selected variables. In addition to simple summary statistics, several other statistics are computed for full uncensored data sets (Full w/o NDs), and for data sets with nondetect (with NDs) observations (e.g., estimates based upon the KM method). Two Menu options: Full w/o NDs and With NDs are available. • Full (w/o NDs): This option computes general statistics for all selected variables. • With NDs: This option computes general statistics including the KM method based mean and standard deviations for all selected variables with ND observations. Each menu option (Full (w/o NDs) and With NDs) has two sub-menu options: • Raw Statistics • Log-Transformed When computing general statistics for raw data, a message will be displayed for each variable that contains non-numeric values. The General Statistics option computes log-transformed (natural log) statistics only if all of the data values for the selected variable(s) are positive real numbers. A message will be displayed if non-numeric characters, zero, or negative values are found in the column corresponding to a selected variable. 4.1 General Statistics for Full Data Sets without NDs 1. Click General Statistics ^ Full (w/o NDs) i;J File Edit Stats/Sample Sizes Graphs Statistical Tests Upper Limrts/BTVs UCLs/EPCs Windows Help Navigation F General Statistics Full (w/o NDs) Raw Statistics 10 Name Work Sheet jds Well 1(bds Imputed NDs using ROS Methods DQOs Based Sample Sizes With NDs Log-Transformed 2. Select either Log-Transformed or Raw Statistics option. 3. The Select Variables screen (see Chapter 3) will appear. • Select one or more variables from the Select Variables screen. • If statistics are to be computed by a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in drop-down list of available variables, and select a proper group variable. 93 ------- Select Variables Available Variables Selected Variables Select Group Column (Optional) sp-Jength (Count = 150} sp-width (Count = 150} pit -length (Count = 150} pt-width (Count = 150} • Click on the OK button to continue or on the Cancel button to cancel the General Statistics option. User Selected Options From File FULLIRIS-ndsjds Full Precision OFF Raw Statistics From Rle: FULLIRIS-ndsjds i Statistics for Uncensored Data Sets NumObs # Missing sp-length (1) 50 sp-length (2) 50 sp-length (3) 50 4.3 4.9 4.9 5.8 7 7.9 Mean 5.006 5.936 6.588 SD 0.352 0.516 0.636 SEM 0.0498 0.073 O.OB99 MAD/0.675 Skewness Kurtosis CV Percentiles for Uncensored Data Sets Variable NumObs # Missing 10%ile sp-length (1) 50 0 4.59 sp-length (2) 50 0 5.38 sp-length (3) ~50 0 5lT 4.7 5.5 6.1 25'jle(Q1) 50''jle(Q2) 75%ile(Q3) 4.8 5 5.2 5.6 5.9 6.3 6.225 6.5 6.9 0.297 0.519 0.593 SOMIe 5.32 6.4 7.2 0.12 0.105 0.118 -0.253 0.0704 -0.533 0.087 0.0329 0.0965 5.41 6.7 7.61 5.61 5.751 6.755 6.951 77 7.802 94 ------- Log-Transformed Statistics User Selected Options From File FULLIRIS-ndsjds Full Precision OFF From Rle: FULLIRIS-ndsjds Summary Statistics for Uncensored Log-Transformed Data Sets Variable NumObs ft Missing Minimum Maximum Mean Variance SD sp-tengthfl) 50 0 1.459 1.758 1.608 0.00437 0.0705 sp4ength{2! 50 0 1.583 1.346 1.777 0.00761 0.0872 sp-length{3) 50 0 1.589 2.067 1.881 0.00343 0.0371 MAD/0.675 Skewness Kiirtosis CV 0.0605 -0.0553 -0.231 0.0438 0.0873 -0.0852 -0.463 0.0431 0.0885 -0.136 0.432 0.0516 PercentiEes for Uncensored Log-Transformed Data Sets Variable NumObs »Missing KTSIe 20' ed NDs using ROS Methods > Based Sample Sizes > J • *' 4 3 5 8 4 7 17 Full (w/o NDs) t 1 With NDs t J „ 567 Raw Statistics Log-Transformed 1 0 0 1 8 3 10 11 2. Select either Log-Transformed or Raw Statistics option. 95 ------- 3. The Select Variables screen (Chapter 3) will appear. • Select variable(s) from the list of variables. • Only those variables that have ND values will be shown. The user should make sure that the variables with NDs are defined properly including the column showing the detection status of the various observations. • If statistics are to be computed by a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop- down list of available variables. Select a proper group variable. • Click on the OK button to continue or on the Cancel button to cancel the summary statistics operations. Raw Statistics - Data Set with NDs User Selected Options From File Zn-alluvial-fan-datajds Full Precision OFF From Rle: Zn-alluvial-fan-datajds Summary Statistics for Censored Data Set (with NDs) using Kaplan Meier Method Variable NumObs « Missing Num Ds NumNDs f. NDs Min ND Max ND KM Mean KM Var KM SD KM CV Cu {alluvial fan) 65 3 48 17 26.155, 1 20 3.608 13.08 3.616 1.002 Cu (basin trough) 49 1 35 14 28.57,1 1 15 4.362 21.64 4.651 1.066 Summary Statistics for Raw Data Sets using Detected Data Only Variable NumObs fl Missing Minimum Maximum Mean Median Var SD MAD/0 675 Skewness CV Cu {alluvial fan) 4S 3 1 20 4.146 2 16.04 4.005 1483 2.256 0.966 Cu (basintraugh) 35 1 1 23 5.229 3 27.18 5.214 2.965 1.878 0.997 Percentiles using all Detects (Ds) and Non Detects (NDs) Variable NumObs S Missing lOSIe 20%ile 25%ile(Qt) 50Sle{Q2) 75%ile(Q3) SOSIe 90Xile 95%ile 99Xile Cu {alluvial fan) 65 3 1 2 2 3 5 7 10 15.2 20 Cu (basin trough) 43 1 1 2 2 48 9.4 12.4 15 20.12 The Summary Statistics screen shown above can be saved as an Excel 2003 or 2007 file. Click Save from the file menu. 96 ------- Chapter 5 Imputing Nondetects Using ROS Methods The imputing of NDs using regression on order statistics (ROS) methods option is available under the Stats/Sample Sizes module of ProUCL 5.1. This option is provided for advanced users who want to use the detected and imputed NDs data for exploratory and data mining purposes on multivariate data sets. For exploratory methods, such as principal component analysis (PCA), cluster, and discriminant analysis to gain additional insight into potential structures and patterns present in a multivariate (more than one variable) data set, one may want to use imputed values in graphical displays (line graphs, scatter plots, boxplots etc.) and in the analyses. To derive conclusions based upon multivariate data sets with nondetects, the developers suggest the use of the KM method based covariance or correlation matrix to perform PCA and regression analysis. These methods are beyond the scope of the ProUCL software which deals only with univariate methods. The details of computing an Orthogonalized Kettenring and Gnanadesikan (OKG) positive definite KM matrix can be found in Maronna, Martin, and Yohai (2006) and in Scout 2008 Version 1.0 guidance documents (2009) which can be downloaded from the EPA Site (http://archive.epa.gov/esd/archive-scout/web/html/). One may not use ROS imputed data to perform parametric statistical tests such as t-test and ANOVA test without further investigation. These issues require further research to evaluate decision errors associated with conclusions derived using such methods. The ROS methods can be used to impute ND observations using a normal, lognormal, or gamma model. ProUCL has three ROS estimation methods that can be used to impute ND observations. The use of this option generates additional columns consisting of all imputed NDs and detected observations. These columns are appended to the existing open spreadsheet file. The user should save the updated file if they want to use the imputed data for their other application(s) such as PCA or discriminant analysis. It is not easy to perform multivariate statistical methods on data sets with NDs. The availability of imputed NDs in a data file helps the advanced users who want to use exploratory methods on data sets with ND observations. Like other statistical methods in ProUCL, NDs can also be imputed by a group variable. One can impute NDs using the following steps. 1. Click Imputed NDs using ROS Methods ^ Lognormal ROS rrOULL D.U - IWMW-WIm INUS.XI5J File Edit Stats/Sample Size; Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation F Name Worksheet ads Well "IQjds General Statistics > Imputed NDs using ROS Methods t DQOs Based Sample Sizes t 2 IWMWvth NDsjds I 3 4 Normal ROS Gamma ROS Loqnormal ROS 8 L : 5 6 7 S 9 10 11 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen; NDs can be imputed using a group variable as shown in the following screen shot. 97 ------- Select Variables Available Variables Selected Variables Name Cu ID 0 Name Zn ID 1 Select Group Column (Optional) Click on the OK button to continue or on the Cancel button to cancel the option. Output Screen for ROS Est. NDs (Lognormal ROS) Option 0 Cu 1 1 3 3 5 1 4~ 4 2 2 1 2 5 11 1 2 2 2 2 20 2 2 3 3 2D~ TO" 7 5 1 Zn 10 9 5 18 10 12 10 11 11 19 8 3 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 20 2 Zone Alluvial Fan Alluvial Fan Alluvial Fan .Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan .Alluvial Fan Alluvial Fan .Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan .Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan .Alluvial Fan Alluvial Fan Alluvial Fan Alluvial Fan 3 D_Qj 0 0 1 1 1 r 1 1 1 1 1 1 0 1 0 1 1 1 1 0 1 1 1 1 0 B~ 1 1 4 5 D_Zh Ln ROS_Zn (alluvial fan) 0 2.12437734466611 1 I 3 1.0DOOQOE-rf>31 1 5 1 18 0 2.7045642735474 f| 12 1 10 1 11 1 11 1 19 1 I 8 0 2.49463676836713 0 3.1603475071042 0 3.55392730586941 1 10 1 10 1 10 1 10 0 3.92469067412296 1 10 0 4.26969100939485 1 10 0 4.60094330444612 1 10 0 4.9229S559179133 f ~ 10" 1 10 1 20 6 LnROS_Zn [basin trough) 20 10 60 20 12 fT 3.48713118440742 14 4.98477186220711 17 1.87132713438924 11 5 12 4 3 6 3 15 13 4 20 8T w 60 40 30" 40 17 Notes: For grouped data, ProUCL generates a separate column for each group in the data set as shown in the above table. Columns with a similar naming convention are generated for each selected variable and distribution using the ROS option. 98 ------- Chapter 6 Graphical Methods (Graph) The graphical methods described here are used as exploratory tools to get some idea about data distributions (e.g., skewed, symmetric), potential outliers and/or multiple populations present in a data set. The following graphical methods are available under the Graphs option of ProUCL 5.1 Box Plot > Multiple Box Plots > Histogram » Q-Q Plots ^ Multiple Q-Q Plots > All graphical displays listed above can be generated using uncensored full data sets (Full w/o NDs) as well as left-censored data sets with nondetect (With NDs) observations. On box plot graphs for data sets with NDs, a horizontal line is also displayed at the highest RL/DL associated with ND observations. 0-0 Plots and Histograms: Q-Q plots and histograms can be generated individually as well as by using a Group variable. Graphs generated using the Group Graphs option shown below is useful when data for selected variable(s) are given in the same column (stacked data) categorized by a Group ID. Options_QCLPIot_NoN Ds Graphs by Groups Individual Graphs • Group Graphs OK Cancel For data sets with NDs, three options described below are available to draw Q-Q plots and histograms. Specifically, these graphs are displayed only for detected values, or with NDs replaced by 1/2DL values, or with NDs replaced by the respective DLs. The statistics displayed on a Q-Q plot (mean, sd, slope, intercept) are computed according to the selected method. On Q-Q plots, ND values are displayed using a different symbol. The exploratory Q- Q plots described here do not require any placeholders for NDs. These graphs are used only to determine the distribution of detected values and to identify potential outliers and/or multiple populations present in a data set. On histograms, the user can change the number of bins (more bins, less bins) used to generate histograms. 99 ------- Graphs by Groups O Individual Graphs • Group Graphs Select How to Handle Nondelect Values >:•:< Use Reported Detection Limit O Use Detection Unit Divided by 2.0 O Do not Display Nondetects Do not Display Nondetects: Selection of this option excludes all NDs from a graphical method (Q-Q plots and histograms) and plots only detected values. The statistics shown on Q-Q plots are computed only using the detected data. Use Reported Detection Limit: Selection of this option treats DLs/RLs as detected values associated with the ND values. The graphs are generated using the numerical values of detection limits and statistics displayed on Q-Q plots are computed accordingly. Use Detection Limit Divided by 2.0: Selection of this option replaces the DLs with their half values. All Q-Q plots and histograms are generated using the half detection limits and detected values. The statistics displayed on Q-Q plots are computed accordingly. For data sets in different columns, one can use the Multiple Q-Q Plots option. By default, this option will display multiple Q-Q plots for all selected variables on the same graph. One can also generate multiple Q-Q plots by using a group variable. Box Plot: Like Q-Q plots, box plots can also be generated by a Group variable. This option is useful when all data are listed in the same column (stacked data) categorized by a Group ID variable. On box plots with NDs, a horizontal line is displayed at the highest detection limit level. ProUCL 5.1 constructs a box plot using all detected and nondetected (using associated DL values) values. A horizontal line is displayed at the highest detection limit. Box Plots are generated using ChartFx, a software used in the development of ProUCL 5.1. Multiple Box Plots: For data in different columns, one can use the Multiple Box Plots option to display multiple box plots for all selected variables on the same graph. One can also generate multiple box plots by using a group variable. Box Plots have an optional feature, which can be used to draw up to four (4) horizontal lines at pre-established screening levels or at statistical limits (e.g., upper limits) computed using a background data set. This option can be used when box plots are generated using onsite data and one may be interested in comparing onsite data with background threshold values and/or pre-established screening levels. This type of box plot represents a visual comparison of site data with background threshold values and/or other action levels. Up to four (4) values can be displayed on a box plot as shown below. If the user inputs a value in the value column, the check box in that row will get activated. For example, the user may want to display horizontal lines at a background UTL95-95 or some pre-established action level(s) on box plots generated using AOCs data. 100 ------- 6.1 Box Plot A brief description of the method used to generate Tukey's box plot (also known as box and whisker plot) is described first. Box Plot (Box and Whiskers Plot): A box plot (box and whiskers plot) represents a convenient exploratory tool and provides a quick five-point summary of a data set. In statistical literature, one can find several ways to generate box plots. The practitioners may have their own preferences to use one method over the other. Box plots are well documented in the statistical literature and a description of box plots can be easily obtained by surfing the net. Therefore, a detailed description about the generation of box plots is not provided in ProUCL guidance documents. All box plot methods including the one in ProUCL represent five-point summary graphs including: the lowest and the highest data values, median (50th percentile=second quartile, Q2), 25th percentile (lower quartile, Ql), and 75th percentile (upper quartile, Q3). A box and whisker plot also provides information about the degree of dispersion (interquartile range (IQR) = Q3-Ql=length/height of the box in a box plot), the degree of skewness (suggested by the length of the whiskers) and unusual data values known as outliers. Specifically, ProUCL (and various other software packages) use the following to generate a box and whisker plot. • Q1= 25th percentile, Q2= 50th (median), and Q3 = 75th percentile • Interquartile range= IQR = Q3-Q1 (the height of the box in a box plot) • Lower whisker starts at Q1 and the upper whisker starts at Q3. • Lower whisker extends up to the lowest observation or (Ql - 1.5 * IQR) whichever is higher • Upper whisker extends up to the highest observation or (Q3 + 1.5* IQR) whichever is lower • Horizontal bars (also known as fences) are drawn at the end of whiskers • Observations lying outside the fences (above the upper bar and below the lower bar) represent potential outliers ProUCL uses a couple of development tools such as FarPoint spread (for Excel type input and output operations) and ChartFx (for graphical displays). ProUCL generates box plots using the built-in box plot feature in ChartFx. The programmer has no control over computing the statistics (e.g., Ql, Q2, Q3, IQR) using ChartFx. Boxplots generated by ProUCL can slightly differ from box plots generated by other programs (e.g., Excel). However, for all practical and exploratory purposes, box plots in ProUCL are equally good (if not better) than those available in the various commercial software packages for exploring data distribution (skewed or symmetric), identifying outliers, and comparing multiple groups 101 ------- (main objectives of box plots in ProUCL). More details about Box Plots can be found in Section 1.16 of Chapter 1 of this document. 1. Click Graphs >• Box Plot Graphs Statistical Tests Upper Limits/BTVs UCLs/E Box Plot > Multiple Box Plots > Full (w/o NDs) With NDs Histogram I Q-Q Plots Multiple Q-Q Plots cc 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs are to be produced by using a Group variable, select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select an appropriate variable representing a group variable as shown below. The default option for Graph by Groups is Group Graphs. This option produces side-by- side box plots for all groups included in the selected Group ID Column (e.g., Zone here). The Group Graphs option is used when multiple graphs categorized by a group variable need to be produced on the same graph. The Individual Graphs option generates individual graphs for each selected variable or one box plot for each group for the variable categorized by a Group ID column (variable). 102 ------- Graphs by Groups O Individual Graphs * Group Graphs Label Value 1 0 Screening Level= 2 D C 3 D 4 D While generating box plots, one can display horizontal lines at specified screening levels or a BTV estimate (e.g., UTL95-95) computed using a background data set. For data sets with NDs, a horizontal line is also displayed at the largest reported DL associated with a ND value. The use of this option may provide information about the analytical methods used to analyze field samples. Click on the OK button to continue or on the Cancel button to cancel the Box Plot (or other selected graphical) option. Box Plot Output Screen (Group Graph) Selected options: Label (Screening Level), Value (12) Box Plot for Cu Me« Non-Deled Value 20 Making Changes in Graphs using Toolbar One can use the toolbar to make changes in a graph generated by ProUCL. The toolbar can be activated by right clicking the mouse on your graph. The context menu on the box plot shown below appears. By using the context menu, one can change color, title, font size, legend box and label points. For example, one can edit the title by clicking title in the context menu. These are typical windows operations which 103 ------- can also be used in ProUCL. However, it should be pointed out that options which affect the computation of statistics displayed on a graph are not wired and can yield incorrect results. For example, changing scales along the x-axis or y-axis (e.g., to log scale) will not automatically displayed statistics in the changed log- scale. The resulting graph using options that affect computed statistics will be incorrect and should be avoided. These operations are illustrated by several screen captures displayed as follows. f R.e Edit £:Ete/£;noU SEES B!s SimUater S-iph: Statist tal Tefc Jpper Limils/B~Vj UOs/EPCs Wrdow; Help Box Plot for Aluminum Activating the toolbar shown above. Fie Edit S:it,'£«nol« Sizat EliSimUirttr G-iphs Statical lets JppaLrrvk'rV; UZU'EPCs Wrdow: Help Box Plot for Aluminum Changing color of the graph shown in the above graph. 104 ------- 5 ProUCL5.1-[BoxFb::JU3;tl •y Fit Ecft S-^te'EsrvoItSizs BISSiiriLlatcr G-ipht Statistcal TKtE JppsrlimiteTVs UCU'EPCs Wrdow: Help •#- ' * a x _ s > Changing title of the graph. r1 ProUCLS,!-[BQxFh;:j|.?a] -} fit Edit STst.'Scnal* 5cEt BtiSimi,|jtcr G-aphE SUtirt cal TK*E JpperUmitr:..'rV; UCU'EPC; Wrdow; Help ^ •' - >- " • ^ li?lKjr Box Plot for Aluminum Edit the title of the graph. 105 ------- 6.2 Histogram 1. Click Graphs ^- Histogram )UCL 5.0 - [Zn-Cu-ND-data.xls] Graphs | Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Box Plot > Multiple Box Plots t Histogram ^ Q-Q Plots ^ Multiple Q-Q Plots ^ 56789 Full (w/o NDs) WithNDs 1 10 11 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be produced by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select an appropriate variable representing a group variable as shown below. • When the option button is clicked for data sets with NDs, the following window will be shown. By default, histograms are generating using the RLs for NDs. Graphs by Groups O Individual Graphs * Group Graphs Select How to Handle Nondetect Values • Use Reported Detection Unit O Use Detection Unit Divided by 2.0 O Do not Display Nondetects The default selection for histograms (and for all other graphs) by a group variable is Group Graphs. This option produces multiple histograms on the same graph. If histograms are needed to be displayed individually, the user should check the radio button next to Individual Graphs. Click on the OK button to continue or on the Cancel button to cancel the histogram (or other selected graphical) option. 106 ------- Histogram Output Screen Selected options: Group Graphs Histogram for Cu Reported values used for nondetects alluvial fan Number of D N umbei of Number of Mean of D f Detects f Nondetects d Less Bins D More Bins Notes: ProUCL does not perform any GOF tests when generating histograms. Histograms are generated using the development software ChartFx and not many options are available to alter the histograms. The labeling along the x-axis is done by the development software and it is less than perfect. However, if one hovers the mouse on a bar, relevant statistics (e.g., begin point, midpint and end point) about the bar will appear on the screen. The Histogram option automatically generates a normal probability density function (pdf) curve irrespective of the data distribution. At this time, ProUCL 5.1 does not display a pdf curve for any other distribution (e.g., gamma) on a historgram. The user can increase or decrease the number of bins to be used in a histogram. 6.3 Q-Q Plots 1. Click Graphs be shown. Q-Q Plots. When that option button is clicked, the following window will Graphs Statistical Tests Upper Limits/BTVs UCLs/E Box Plot Multiple Box Plots Histogram Q-Q Plots Multiple Q-Q Plots Full (w/o NDsJ With NDs Q-Q Plots can be generated for data sets With NDs and without NDs [Full (w/o NDs)]. • Select either Full (w/o NDs) or With NDs option. • The Select Variables screen (Chapter 3) will appear. 107 ------- Select one or more variable(s) from the Select Variables screen. If graphs have to be produced by using a group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable as shown below. Click on the OK button to continue or on the Cancel button to cancel the selected Q-Q plots option. The following options screen appears providing choices on how to treat NDs. The default option is to use the reported values for all NDs. .QQ_Plot_w Graphs by Groups ; Individual Graphs • Group Graphs Select How to Handle Nondetect Values • Use Reported Detection Limit O Use Detection Unit Divided by 2.0 O Do not Display Nondetects Click on the OK button to continue or on the Cancel button to cancel the selected Q-Q plots option. The following Q-Q plot appears when used on the copper concentrations of two zones: Alluvial Fan and Basin Trough. Output Screen for Q-Q plots (With NDs) Selected options: Group Graph, No Best Fit Line Q-Q Plot for Cu Reported values used for nondetects Theoretical Quantiles (Standard Normal) NDs Displayed in smaller font alluvial far, • tajn ll.augh alluvial fen Total Numbs ol Dal Mumbet of Ncn-Oets Slope [displayed data] = 4.045 Intercept (displayed dataM 815 Co-relation, H = 1359 Detected Sd = 5.214 Slopaldisp^ed data] -4.534 Intelcapt Idiplayed dataN 5 49 Cwtdation. Fl =0.903 Note: The font size of ND values is smaller than that of the detected values. 108 ------- 6.4 Multiple Q-Q Plots 6.4.1 Multiple Q-Q plots (Uncensored data sets) 1. Click Graphs >• Multiple Q-Q Plots 2. Multiple Q-Q Plots can be generated for data sets With NDs and without NDs [Full (w/o NDs)]. • When that Option button is clicked, the following window will be shown. Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Box Plot t Multiple Box Plots t Histogram t Q-Q Plots > Multiple Q-Q Plots > 5 3.6 1.4 567 i d-spjengt d_sp*idth djjHe .2 1 1 .2 1 1 Full (w/o NDs) With NDs \ \ • Select either Full (w/o NDs) or With NDs. • The Select Variables Screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be produced by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable as shown below. • Click OK to continue or Cancel button to cancel the selected Multiple Q-Q Plots option. Example 6-1: The following graph is generated by using Fisher's (1936) data set for 3 Iris species. Output Screen for Multiple Q-Q Plots (Full w/o NDs) Selected Options: Group Graph, Best Fit Line Normal Q-Q Plot Theoretical Quantises [Standard Normal] 109 ------- If the user does not want the regression lines shown above, click on the Best Fit Line and all regression lines will disappear as shown below. Normal Q-Q Plot N-EO Mean * E 006 Sd-0952 Elope - 0 356 Intercept * 5.008 Carelation. R - 0 93! N.50 Mean - 5 336 Sd-0.518 Stow-0522 Intercept - 5.J36 Correlation, R - 0.33 .p-lengthfj) Correlation. R = 0.9E sp-widthti) N = EO Sd = 0 379 Slope - 0.379 Intercept = 3.428 . Theoretical Quantiles (Standard Normal) lhri) • sp-lenslh[2] • tp-lengt)i(3j 9 sp™dtli(1) O sp-wUlh[2] Q sp-width® Notes: For Q-Q plots and Multiple Q-Q plots option, for both "Full" as well as for data sets "With NDs," the values along the horizontal axis represent quantiles of a standardized normal distribution (Normal distribution with mean=0 and standard deviation=l). Quantiles for other distributions (e.g., Gamma distribution) are used when using the Goodness-of-Fit (GOF, G.O.F.) test option. 6.5 Multiple Box Plots 6.5.1 Multiple Box plots (Uncensored data sets) ^. Click Graphs > Multiple Box Plots 2. Multiple Q-Q Plots can be generated for data sets With NDs and without NDs [Full (w/o NDs)]. • When the option button is clicked, the following window will be shown. Graphs Box Statistical Tests Upper Limits/BTVs UCLs/EPCs Plot > Multiple Box Plots > Histogram t Q-Q Plots > Multiple Q-Q Plots > 5 6 7 Full (w/o NDs) krt-te With NDs .2 1 1 Select either Full (w/o NDs) or With NDs. The Select Variables screen (Chapter 3) will appear. Select one or more variable(s) from the Select Variables screen. 110 ------- • If graphs have to be produced by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable as shown below. • Click on the OK button to continue or on the Cancel button to cancel the selected Multiple Box Plots options. The following graph is generated by using the above options. Example 6-1 (continued): The following graph is generated by using the above options on Fisher's (1936) Iris data set collected from 3 species of Iris flower. Output Screen for Multiple Box Plots (Full w/o NDs) Selected options: Group Graph Multiple Box Plots 111 ------- Chapter 7 Classical Outlier Tests Outliers are inevitable in data sets originating from environmental applications. In addition to informal graphical displays (e.g., Q-Q plots and box plots) and classical outlier tests (Dixon test, Rosner test), there exist several robust outlier identification methods (e.g., Biweight, Huber, PROP, MCD) for identifying any number of multiple outliers present in data sets of various sizes (Scout 2008; EPA 2009). It is well known that the classical outlier tests: Dixon test and Rosner test suffer from masking (e.g., extreme outliers may mask intermediate outliers) effects. The use of robust outlier identification procedures is recommended for identifying multiple outliers, especially when dealing with multivariate (having multiple constituents) data sets. However, those preferred and more efficient robust outlier identification methods are beyond the scope of ProUCL 5.1. Several robust outlier identification methods (e.g., based upon Biweight, Huber, and PROP influence functions, Singh and Nocerino 1995) are available in the Scout 2008 vl.O software package (EPA, 2009). The two classical outlier tests: Dixon and Rosner tests (EPA 2006a; Gilbert 1987) are available in ProUCL 4.0 and higher versions of the ProUCL software. These tests can be used on data sets with and without ND observations. These tests require the assumption of normality of the data set without the outliers; as data sets consisting of outliers seldom follow a normal distribution. It should be noted that in environmental applications, one of the objectives is to identify high outlying observations that might be present in the right tail of a data distribution, as those observations often represent contaminated locations requiring further investigations. Therefore, for data sets with NDs, two options are available in ProUCL to deal with data sets with outliers. These options are: 1) exclude NDs and 2) replace NDs by DL/2 values. These options are used only to identify outliers and not to compute any estimates and limits used in decision-making processes. To compute the various statistics of interest, ProUCL uses rigorous statistical methods suited for left-censored data sets with multiple DLs. It is suggested that the outlier identification procedures be supplemented with graphical displays such as normal Q-Q plots and box plots. On a normal Q-Q plot, observations that are well-separated from the bulk of the data typically represent potential outliers needing further investigation. Also, significant and obvious jumps and breaks in a normal Q-Q plot can be indications of the presence of more than one population and/or data gaps due to lack of enough data points (data sets of smaller sizes). Data sets of large sizes (e.g., >100) exhibiting such behavior on Q-Q plots may need to be partitioned out into component sub-populations before estimating EPCs or BTVs. 112 ------- Outlier tests in ProUCL 5.1 are available under the Statistical Tests module. Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Outlier Tests > Goodness-of-Fit Tests > Single Sample Hypothesis ^ Two Sample Hypothesis > Oneway AN OVA ^ OLS Regression Trend Analysis ^ Full (w/o NDs) C With NDs > 1 ~T 1 ~T 0 ~~6~ i " ~T 1 i Dixon's Outlier Test (Extreme Value Test): Dixon's test is used to identify statistical outliers when the sample size is < 25. This test identifies outliers or extreme values in the left tail (Case 2) and also in the right tail (Case 1) of a data distribution. In environmental data sets, outliers found in the right tail, potentially representing impacted locations, are of interest. The Dixon test assumes that the data without the suspected outlier (s) are normally distributed. If the user wants to perform a normality test on the data set, he should first remove the outliers before performing the normality test. This test tends to suffer from masking in the presence of multiple outliers. This means that if more than one outlier (in either tail) is suspected, this test may fail to identify all of the outliers. Rosner Outlier Test: This test can be used to identify up to 10 outliers in data sets of sizes 25 and higher. This test also assumes that the data set without the suspected outliers is normally distributed. Like the Dixon test, if the user wants to perform a normality test on the data set, he should first remove the outliers (which are not known in advance) before performing the normality test. The detailed discussion of these two tests is given in the associated ProUCL Technical Guide. A couple of examples illustrating the identification of outliers in data sets with NDs are described in the following sections. 7.1 Outlier Test for Full Data Set 1. Click Outlier Tests ^ Full (w/o NDs) ^ Compute Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests Goodness-of-Fit Tests Single Sample Hypothesis Two Sample Hypothesis Oneway ANOVA OLS Regression Trend Analysis ^ Full (w/o NDs) > With NDs 1 ' 1 1 D 1 1 1 1 1 D 1 1 » Compute f I 1 1 0 1 1 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If outlier test needs to be performed by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. 113 ------- If at least one of the selected variables (or group) has 25 or more observations, then click the option button for the Rosner Test. ProUCL automatically performs the Dixon test for data sets of sizes <25. Number of Outliers for Rosner Test 2 Applicable to Rosner's Test (N >= 25) Only Dixon Test is for Ns25 and tests for 1 outlier • The default option for the number of suspected outliers is 1. To use the Rosner test, the user has to obtain an initial guess about the number of suspected outliers that may be present in the data set. This can be done by using graphical displays such as a Q-Q plot. On a Q-Q plot, higher observations that are well separated from the rest of the data may be considered as potential or suspected outliers. • Click on the OK button to continue or on the Cancel button to cancel the Outlier Test. 7.2 Outlier Test for Data Sets with NDs Two options: exclude NDs; or replace NDs by their respective DL/2 are available in ProUCL to perform outlier tests on data sets with NDs. 1. Click Outlier Tests ^ With NDs ^ Exclude NDs UCLs/EPCs Windows Help G o o d n ess- of - Fit Tests Single Sample Hypothesis Two Sample Hypothesis Oneway AN OVA OLS Regression Trend Analysis DL/2 Estimates Note: The above screen shot was generated using ProUCL 5.0; the exactly similar screen is generated using ProUCL 5.1 with the exception that in the title of the screen shot, ProUCL 5.0 will be replaced by ProUCL 5.1. Therefore, to save time, many intermediate screen shots used in the ProUCL 5.0 User Guide have been used in this ProUCL 5.1 User Guide. 114 ------- Output Screen for Dixon's Outlier Test Dixon's Outlier Test for TCE-1 Total N = 12 Number NDs= 4 Number Detects = 8 10% critical value: 0.479 5% critical value: 0.554 1'"-. critical value: 0.683 Note: NDs excluded from Outlier Test 2. Data Value 0.75 is a Potential Outlier (Lower Tail)? 1. Data Value 9.29 is a Potential Outlier (Upper Tail)? Test Statistic: 0.011 Test Statistic: 0.392 For 10% significance level. 9.29 is not an outlier. For 5% significance level. 9.29 is not an outlier. For 1% significance level, 9.29 is not an outlier. For 10%. significance level, D.75 is not an outlier. For 5% significance level, 0.75 is not an outlier. For 1% significance level, 0.75 is not an outlier. Q-Q plot without Four Nondetect Observations are Shown as Follows Q-Q Plot for TCE-1 TCE-I Nondetects not displayed ^^"^'^ 8 6 4 2 OK '• Number of Detects - 8 Detected Mean - 2 941 Detected Sd = 3 12 Slope (displayed daia] = 2 934 Intercept (displayed data(= 2.941 Cwwlatiotx R - 1875 QBeslFitLra 5.37 298 V9 •1.5 -1.0 -05 0.0 0.5 1 0 1.5 Theoretical Quantiles (Standard Normal) NDs Displayed in smaller font Example: Rosner's Outlier Test by a Group Variable, Zone • Selected Options: Number of Suspected Outliers = 4 • NDs excluded from the Rosner Test • Outlier test performed using the Select Group Column (Optional) 115 ------- Output Screen for Rosner's Outlier Test for Zinc in Zone: Alluvial Fan Rosnt IT'S Outlier Total N Number NDs Number Detects Mean of Detects SD of Detects Number of data Number of suspected outliers s not included in the following: U Mean sd 1 27.88 2 16.04 3 15.35 4 14.83 84.13 8.776 7.356 6.485 Test for 4 Outliers in Zn (alluvia 67 16 51 27.88 85.02 51 4 Potential Obs. Test outlier Number value 520 26 7.034 50 28 3.87 40 27 3.352 33 29 2.801 Ifan) I Critical value (5%) 3.137 3.127 3.11B 3.108 Critical value fTA) 3.4S8 3.478 3.469 3.468 For 5% significance level, there are 3 Potential Outliers 620, 50.40 For 1% Significance Level, there are 2 Potential Outliers Q-Q plot for Zinc Based upon Detected Data (Alluvial Fan) Q-Q Plot for Zn (alluvial fan) z- «N»™IW Nondetects not displayed O«NUI.I,«,>ID.I..B umbel ol N on-D eleclr -16 400 = 300 s 200 too HjCU ? n « 5'° Detected Sd- 85.02 Slope (displayed data) = 35.4 lntelCept(diSplayeddalaH27.aB n Best fit Line -1.8 -1.2 -0.6 0.0 0.6 1.2 1.8 Theoretical Quantiles (Standard Normal) NDs Displayed in smaller font 116 ------- Total N 50 Number NDs 4 Number Detects 4G Mean of Detects 23.13 SD of Detects 19.03 Number of data 46 Number of suspected outliers s not included in the following: Output Screen for Rosner's Outlier Test for Zinc in Zone: Basin Trough Rosner's Oilier Test for 4 Outliers in Zn (basin trough) Potential # I Mean sd outlier 1 23.13 18.82 90 2 21.64 16.32 70 3 20.55 14.73 60 4 19.63 13.57 60 Obs. Number 45 21 3 22 Test value 3.553 2.963 2.679 2.975 Critical value (5%) 3.09 3.09 3.08 2.07 Critical value (1%) 3.45 3.44 3.43 3.41 For 5% significance level, there are 4 Potential Outliers 90. 70. 60. 60 For Insignificance Level, there is 1 Potential Outlier 117 ------- 118 ------- Chapter 8 Goodness-of-Fit (GOF) Tests for Uncensored and Left-Censored Data Sets GOF tests are available under the Statistical Test module of ProUCL 5.0/ProUCL 5.1. Throughout this User Guide and in ProUCL software, "Full" represents uncensored data sets without ND observations. The details and usage of the various GOF tests are described in the associated ProUCL Technical Guide. In ProUCL 5.1, critical values associated with Lilliefors normality test are computed using a more efficient algorithm as described in the associated ProUCL 5.1 Technical Guide. Therefore, tables and graphs involving Lilliefors test have been re-generated using ProUCL 5.1. 8.1 Goodness-of-Fit Test in ProUCL Several GOF tests for uncensored full (Full (w/o NDs)) and left-censored (With NDs) data sets are available in the ProUCL software. • Full (w/o NDs) Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Outlier Tests ^ G o o d n ess- of - Fit Tests > Single Sample Hypothesis » Two Sample Hypothesis ^ Oneway AN OVA ^ OLS Regression Trend Analy:i: ^ 678 Normal Gamma Lognormal G.O.F, Statistics 0.071 22 0.043E 0.427 32 0.001 3f o This option is used on uncensored full data sets without any ND observations. This option can be used to determine GOF for normal, gamma, or lognormal distribution of the variable(s) selected using the Select Variables option. o Like all other methods in ProUCL, GOF tests can also be performed on variables categorized by a Group ID variable. o Based upon the hypothesized distribution (normal, gamma, lognormal), a Q-Q plot displaying all statistics of interest including the derived conclusion is also generated. o The G.O.F. Statistics option generates a detailed output log (Excel type spreadsheet) showing all GOF test statistics (with derived conclusions) available in ProUCL. This option helps a user to determine the distribution of a data set before generating a GOF Q-Q plot for the hypothesized distribution. This option was included at the request of some users in earlier versions of ProUCL. 119 ------- • With NDs o This option performs GOF tests on data sets consisting of both nondetected and detected data values. o Several sub-menu items shown below are available for this option. Statistical Tests Outlier Test Upper Limits/BTVs UCLs/EPCs Windows Help > > Goodness-of-Fit Tests > Single Sample Hypothesis I Two Sample Hypothesis ^ Oneway ANOVA > OLS Regression Trend Analysis > 6789 Full (w/o NDs) > With NDs > 10 11 Exclude NDs > Normal-ROS Estimates > Gamma-ROS Estimates * Log-ROS Estimates K 01^2 Estimates K G.O.F. Statistics 12 13 14 Normal Gamma Lognormal 1. Exclude NDs: tests for normal, gamma, or lognormal distribution of the selected variable(s) using only the detected values. 2. ROS Estimates: tests for normal, gamma, or lognormal distribution of the selected variable(s) using detected values and imputed nondetects. o Three ROS methods for normal, lognormal (Log), and gamma distributions are available. This option imputes the NDs based upon the specified distribution and performs the specified GOF test on the data set consisting of detects and imputed nondetects. 3. DL/2 Estimates: tests for normal, gamma, or lognormal distribution of the selected variable(s) using the detected values and the ND values replaced by their respective DL/2 values. This option is included for historical reasons and also for curious users. ProUCL does not make any recommendations based upon this option. 4. G.O.F. Statistics: Like full uncensored data sets, this option generates an output log of all GOF test statistics available in ProUCL for data sets with nondetects. The conclusions about the data distributions for all selected variables are also displayed on the generated output file (Excel-type spreadsheet). Multiple variables: When multiple variables are selected from the Select Variables screen, one can use one of the following two options: o Use the Group Graphs option to produce multiple GOF Q-Q plots for all selected variables in a single graph. This option may be used when a selected variable has data coming from two or more groups or populations. The relevant statistics (e.g., slope, intercept, correlation, test statistic and critical value) associated with the selected variables are shown on the right panel of the GOF Q-Q plot. To capture all the graphs and results shown on the window screen, it is preferable to print the graph using the 120 ------- Landscape option. The user may also want to turn off the Navigation Panel and Log Panel. o The Individual Graphs option is used to generate individual GOF Q-Q plots for each of the selected variables, one variable at a time (or for each group individually of the selected variable categorized by a Group ID). This is the most commonly used option to perform GOF tests for the selected variables. • GOF Q-Q plots for hypothesized distributions: ProUCL computes the relevant test statistic and the associated critical value, and prints them on the associated Q-Q plot (called GOF Q-Q plot). On a GOF Q-Q plot, the program informs the user if the data are gamma, normally, or lognormally distributed. o For all options described above, ProUCL generates GOF Q-Q plots based upon the hypothesized distribution (normal, gamma, lognormal). All GOF Q-Q plots display several statistics of interest including the derived conclusion. o The linear pattern displayed by a GOF Q-Q plot suggests an approximate GOF for the selected distribution. The program computes the intercept, slope, and the correlation coefficient for the linear pattern displayed by the Q-Q plot. A high value of the correlation coefficient (e.g., > 0.95) may be an indication of a good fit for that distribution; however, the high correlation should exhibit a definite linear pattern in the Q-Q plot without breaks and discontinuities. o On a GOF Q-Q plot, observations that are well separated from the majority of the data typically represent potential outliers needing further investigation. o Significant and obvious jumps and breaks and curves in a Q-Q plot are indications of the presence of more than one population. Data sets exhibiting such behavior of Q-Q plots may require partitioning of the data set into component subsets (representing sub-populations present in a mixture data set) before computing upper limits to estimate EPCs or BTVs. It is recommended that both graphical and formal goodness-of-fit tests be used on the same data set to determine the distribution of the data set under study. • Normality or Lognormality Tests: In addition to informal graphical normal and lognormal Q-Q plots, a formal GOF test is also available to test the normality or lognormality of the data set. o Lilliefors Test: a test typically used for samples of size larger than 50 (> 50). However, the Lilliefors test (generalized Kolmogorov Smirnov [KS] test) is available for samples of all sizes. There is no applicable upper limit for sample size for the Lilliefors test. o Shapiro and Wilk (SW. S-W) Test: a test used for samples of size smaller than or equal to 2000 (<= 2000). In ProUCL 5.0, the SW test uses the exact SW critical values for samples of size 50 or less. For samples of size, greater than 50, the SW test statistic is displayed along with the/"-value of the test (Royston 1982, 1982a). Notes: As with other statistical tests, sometimes these two GOF tests might lead to different conclusions. The user is advised to exercise caution when interpreting these test results. When one the GOF tests passes the hypothesized distribution, ProUCL 5.0/ProUCL 5.1 determines that the data set follows an 121 ------- approximate hypothesized distribution. It should be pointed out that for data sets of smaller sizes (e.g., <50), when Lilliefors tests determines that the data set follows a normal (lognormal) distribution and Shapiro-Wilk's test determines that the data set does not follow a normal (lognormal) distribution, the user may not use the approximate normality (lognormality) conclusion derived using the Lilliefors test. Under these situations, the user may determine the data distribution based upon the highest p-value associated with GOF test statistics for other distributions or may determine that the data set does not follow a discernible distribution. • GOF test for Gamma Distribution: In addition to the graphical gamma Q-Q plot, two formal empirical distribution function (EDF) procedures are also available to test the gamma distribution of a data set. These tests are the AD test and the KS test. o It is noted that these two tests might lead to different conclusions. Therefore, the user should exercise caution interpreting the results. o These two tests may be used for samples of sizes in the range of 4 - 2500. Also, for these two tests, the value (known or estimated) of the shape parameter, k (k hat) should lie in the interval [0.01, 100.0]. Consult the associated ProUCL Technical Guide for a detailed description of the gamma distribution and its parameters, including k. Extrapolation of critical values beyond these sample sizes and values of k is not recommended. Notes: Even though, the GOF Statistics option prints out all GOF test statistics for all selected variables, it is suggested that the user should look at the graphical Q-Q plot displays to gain extra insight (e.g., outliers, multiple population) into the data set. 8.2 Goodness-of-Fit Tests for Uncensored Full Data Sets 1. 2. Click Goodness-of-Fit Tests ^ Full (w/o NDs) roUCL 5.0 - [pyrene.xls] Statistical Tests I Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests > Single Sample Hypothesis t Two Sample Hypothesis t Oneway AN OVA > OLS Regression Trend Analysis >• 6 7 8 3 Full (w/o NDs) > With NDs t 1C 11 Normal Gamma Lognormal G.O.F. Statistics Select the distribution to be tested: Normal, Lognormal, or Gamma • To test for normality, click on Normal from the drop-down menu list. • To test for lognormality, click on Lognormal from the drop-down menu list. • To test for gamma distribution, click on Gamma from the drop-down menu list. 122 ------- 8.2.1 GOF Tests for Normal and Lognormal Distribution 1. Click Goodness-of-Fit Tests ^- Full (w/o NDs) ^- Normal or Lognormal •UCL 5.0 - [pyrene.: Statistical Tests I Upper Limits/BTVs UCLs/EPCs Windows Help 2. Outlier Tests Goodness-of-Fit Tests Single Sample Hypothesis > Two Sample Hypothesis > Oneway AN OVA > OLS Regression Trend Analysis > 8 10 11 Full (w/o NDs) With NDs Normal Gamma Lognormal G.O.F. Statistics The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be produced by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. • When the Option button is clicked, the following window will be shown. Select Goodness-of-Fit Opti Select Confidence Coefficient O 99* ® 95* O 90% Select GOF Method * Shapiro-Wilk O Lilliefors Graphs by Groups O Individual Graphs (§) Group Graphs Cancel The default option for the Confidence Level is 95%. 123 ------- The default GOF Method is Shapiro-Wilk. The default option for Graphs by Group is Group Graphs. If you want to see the plots for all selected variables individually, and then check the button next to Individual Graphs. Click OK button to continue or Cancel button to cancel the GOF tests. Notes: This option for Graphs by Group is specifically provided for when the user wants to display multiple graphs for a variable by a group variable (e.g., site AOC1, site AOC2, background). This kind of display represents a useful visual comparison of the values of a variable (e.g., concentrations of COPC- Arsenic) collected from two or more groups (e.g., upgradient wells, monitoring wells, residential wells). Example 8-la Consider the chromium concentrations data set used in Example 1-1 of Chapter 1. The lognormal and normal GOF test results on chromium concentrations are shown in the following figures. Output Screen for Lognormal Distribution (Full (w/o NDs)) Selected Options: Shapiro-Wilk Lognormal Q-Q Plot for Chromium Slope - 0.579 nlercepl - 2.334 Conelalion. R * 0 987 Shapilo-WilkTesI ExaclTestStalijnc-0.979 CrilicalValue[0.05]-0916 Dala^pear Lognormal Appro*. TestValue-0.378 p-Value - 0.849 Theoretical Quantiles (Standard Normal) 124 ------- Output Screen for Normal Distribution (Full (w/o NDs)) Selected Options: Shapiro-Wilk, Best Fit Line not Displayed Normal Q-Q Plot for Chromium Theoretical Quantises (Standard Normal} Mean-11.97 Sd = 6 892 Stops-S.58 Coitelafoi). R - D 324 Shspkc-Wik Test Appro*. Test Vd>.ie = 0 £70 D Best Fit Line 8.2.2 GOF Tests for Gamma Distribution 1. Click Goodness-of-Fit Tests ^ Full (w/o NDs) ^ Gamma Outlier Tests ^ Goodness-of-Fit Tests > Single Sample Hypothesis > Two Sample Oneway AN OLS Regress Trend Analj ; Hypothesis t OVA > ion sis > 6 7 S 9 1C 11 Full (w/o NDs) c With NDs >• Normal Gamma Lognormal G.O.F, Statistics 2. The Select Variables screen (described in Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be produced by using a group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. • When the option button is clicked, the following window will be shown. 125 ------- Select Goodness-of-Fit Options Select Confidence Coefficient O 99* ® 95X O Select GOF Method * Aiderson Dariing ' Kolmogorov Smiinov Graphs by Groups O Individual Graphs • Group Graphs o The default option for the Confidence Coefficient is 95%. o The default GOF method is Anderson Darling. o The default option for Graph by Groups is Group Graphs. If you want to see individual graphs, then check the radio button next to Individual Graphs. o Click the OK button to continue or the Cancel button to cancel the option. o Click OK button to continue or Cancel button to cancel the GOF tests. Example 8-lb. Consider arsenic concentrations data set used in Example 1-1 of Chapter 1. The Gamma GOF test results for arsenic concentrations, are shown in the following G.O.F. Q-Q plot. Output Screen for Gamma Distribution (Full (w/o NDs)) Selected Options: Anderson Darling with Best Line Fit Gamma Q-Q Plot for Arsenic Theoretical Quantiles of Gamma Distribution 126 ------- 8.3 Goodness-of-Fit Tests Excluding NDs This option is the most important option for a GOF test applied to data sets with ND observations. Based upon the skewness and distribution of detected data, ProUCL computes the appropriate decision statistics (UCLs, UPLs, UTLs, and USLs) which accommodate data skewness. Specifically, depending upon the distribution of detected data, ProUCL uses KM estimates in parametric or nonparametric upper limits computation formulae (UCLs, UTLs) to estimate EPC and BTV estimates. 1. Click Goodness-of-Fit Tests ^ With NDs ^ Exclude NDs 2. Select distribution to be tested: Normal, Gamma, or Lognormal. • To test for normality, click on Normal from the drop-down menu list. • To test for lognormality, click on Lognormal from the drop-down menu list. • To test for gamma distribution, click on Gamma from the drop-down menu list. 8.3.1 Normal and Lognormal Options 1. Click Goodness-of-Fit Tests ^ With NDs ^ Excluded NDs ^ Normal or Lognormal r Graphs senic 1 1 1 1 1 1 1 0 0 2 Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests > Single Sample Hypothesis t Two Sample Hypothesis ¥ Oneway ANOVA > OLS Regression Trend Analysis » 6 7 8 9 Full (w/o NDs) > With NDs > 10 11 Exclude NDs > Normal-ROS Estimates > Gamma-RQS Estimates > Log-ROS Estimates ¥ DL/2 Estimates > Using Censored Plot > G.O.F. Statistics 12 13 14 Normal Gamma Lognormal Note: In ProUCL 5.1, the censored probability plot option has been added as shown in the above drop- down menu as "Using Censored Plot." This option is very much the same as the Q-Q plot option generated by excluding NDs except that the hypothesized quantiles displayed along the x-axis adjust for quantiles associated with NDs. There is not much difference (except for the correlation, slope and intercept of the optional line displayed on the Q-Q plot) between these two graphs from the decision making point of view. Censored probability (Q-Q) plots do not provide additional information than tools and graphs already available in ProUCL 5.0. This was the reason that censored Q-Q plots were not included in ProUCL 5.0 and its earlier versions. 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. 127 ------- If graphs have to be produced by using a group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. When the option button: Normal or Lognormal is clicked, following window appears. ?ct Goodness-of-Fit Options Select Confidence Coefficient O MX ® 35X O 90% Select GOF Method * Shapiro-Wilk Lilliefors Graphs by Groups (_J Individual Graphs >:•:' Group Graphs OK Cancel o The default option for the Confidence Coefficient is 95%. o The default GOF Method is Shapiro-Wilk. o The default option for Graphs by Group is Group Graphs. If you want to see the plots for all selected variables individually, then check the button next to Individual Graphs. • Click the OK button to continue or the Cancel button to cancel the option. • Click the OK button to continue or the Cancel button to cancel the GOF tests. Example 8-2a. Consider the arsenic Oahu data set with NDs discussed in the literature (e.g., Helsel 2012; NADA in R [Helsel 2013]). The normal and lognormal GOF test results based upon the detected data are shown in the following two figures. Censored Q-Q plots are also displayed along with Q-Q plots based upon detected data. 128 ------- GOF Q-Q Plot for Normal Distribution (Exclude NDs) Selected Options: Shapiro-Wilk Normal Q-Q Plot (Statistics using Detected Data) for Arsenic -., Total Nui NumbeiofNDs = 13 MaxDL=2 Percent NDs = 54£ Mean = 1.236 Srf»tt9G5 Slope = 0.906 Intercept 1.23G Collection, R - Q 887 ShapiioWikTest Enact Test Value-0.777 Critical Val{0.05) = 0.850 p-Value = O.OOG73 Distribution Test Suspect D Best Fit Line 0.7 0.7 Theoretical Quantiles (Standard Normal) Censored Probability (Q-Q) Plot for Normal Distribution Selected Options: Shapiro-Wilk and Best Line Fit Normal Q-Q Plot for Arsenic Statistics Displayed using Censored Quantiles and Detected Data Numbeiol Detecli -11 M inimum Censored Quamtile - -1 347 Percent MDx = 54£ Mean of Detected fete = 1.236 Sd of Detected Data = 1965 Slope = 0.776 Intercept = 1.53 rniiflnhon.n-ll'ir* Stmpiio WifcTesl Exacl TeslValue - IJ /.'/ C.ilicalV3l|0.05) = a350 DataNol Normal Approx. Test Value = II ?i51 p-Value - 0 OOB73 Oistiibution Test Suspect Theoretical Quantiles (Standard Normal) Only Quantiles for Detects Displayed 129 ------- GOF Q-Q Plot for Lognormal Distribution (Exclude NDs) Selected Options: Lilliefors Test Lognormal Q-Q Plot (Statistics using Detected Data) for Arsenic 0.557 -0.557 NumbeiofNDs = 13 MaxDL=2 N =11 Percent NO s = 54£ Mean— 0.0255 LillieforsTest Test Statistic = 0.223 CriticalValuefG 05) - 0.251 D ata Appeal Lognoimal D isli ibulion T est Suspect Q Best Fit Line Theoretical Quantiles (Standard Normal) Censored Probability (Q-Q) Plot for Lognormal Distribution Selected options: Lilliefors Test with Best Fit Line Lognormal Q-Q Plot for Arsenic Statistics Displayed using Censored Quantiles and Detected Data Total Number nl Data = 24 Number of Detects = 11 M inimum Censoted U uam tile = 1 347 Percent NDs = 54£ MaKDL=2 H eanx of Delects = -0.0255 StdvxofDetects = aG94 Slope = 0.559 Coirel'-rfion, !"! - 0 97C, LilliefoisTest Teil Statistic = 0.229 CiiticalValuetO. 05) =0.251 I > isu ii H ji urn T est Suspect • Best Fit Line Theoretical Quantiles (Standard Normal) Only Quantiles for Detects Displayed 130 ------- 8.3.2 Gamma Distribution Option 1. Click Goodness-of-Fit Tests ^ With NDs *• Excluded NDs ^ Gamma 2. Statistical Tests ] Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests It Single Sample Hypothesis Two Sample Hypothesis Oneway AN OVA OLS Regression Trend Analysis 3 0.344 515657826 * 6789 Full (w/o NDs) t With NDs > 1 1 1 1 1 10 11 Exclude NDs > Normal-ROS Estimates ^ Gamma-ROS Estimates > Log-ROS Estimates > DL/2 Estimates > G.O.F. Statistics 12 13 14 Normal Gamma Lognormal The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be produced by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. • When the option button (Gamma) is clicked, the following window is shown. Select Goodness-of-Fit Options Select Confidence Coefficient O 99* ® 9SX O Select GOF Method O Anderaon Dating * Kblmogorov Smimov Graphs by Groups O Individual Graphs » Group Graphs o The default option for the Confidence Coefficient is 95%. o The default GOF test method is the Anderson Darling test. o The default option for Graph by Groups is Groups Graphs. If you want to display all selected variables on separate graphs, check the button next to Individual Graphs. 131 ------- • Click the OK button to continue or the Cancel button to cancel the option. • Click the OK button to continue or the Cancel button to cancel the GOF tests. Example 8-2b (continued). Consider the arsenic Oahu data set with NDs as discussed in Example 8-2a above. The gamma GOF test results based upon the detected data are shown in the following GOF Q-Q plot. Output Screen for Gamma Distribution (Exclude NDs) Selected Options: Anderson-Darling Test with Best Fit Line Gamma Q-Q Plot (Statistics using Detected Data) for Arsenic Mean = 1 2364 khal = 2256G Ihela hat = 0.5479 Slope -1.0345 Intercept = -0.0177 nders on-Darling Ted est Statistic = 1787 riticalValue|0.05) = 0,738 nt 6 amma Distributed Distribution T est Suspect • Best Fit Line Theoretical Quantiles of Gamma Distribution Censored Probability (Q-Q) Plot for Gamma Distribution Selected options: Anderson-Darling Test with Best Fit Line Gamma Q-Q Plot for Arsenic Statistics Displayed using Censored Quantiles and Detected Data Anraifc TotalNumbeiofOata = 24 Number of Delects -11 Number or NonDetects - 13 Minimum Censored Quamtile - 0.2 Percent NDi- 54% Mean of Detects = 1.2364 khalolDetects-22566 ! 11. t.i hat ui Deiecti - 0.5479 Slope = 0.9205 Intercept = OVf05B Correlation. R = 0.3882 Anderson Darling Test lestStahstb^ttTB? Critical Value[0.05) - 0738 Data not Gamma Distributed Distribution Test Suspect Theoretical Quantiles of Gamma Distribution Only Quantiles for Detects Displayed 132 ------- 8.4 Goodness-of-Fit Tests with ROS Methods 1. Click Goodness-of-Fit Tests ^ With NDs ^ Gamma-ROS Estimates or Log-ROS Estimates Statistical Tests | UpperLimits/BTVs UCLs/EPCs Windows Help Outlier Tests »• Goodness-of-Fit Tests > Single Sample Hypothesis k Two Sa Onewa OLSRe Trend L .9 0 344 mple Hypothesis k • ANOVA > jression analysis > 515657826 6 7 S 9 Full (w/o NDs) > With NDs > 1 1 1 1 1 1C 11 12 13 14 Exclude NDs > Normal-ROS Estimates > Gamma-ROS Estimates > Log-ROS Estimates > DL/2 Estimates > G.O.F. Statistics Normal Gamma Lognormal 2. Select the distribution to be tested: Normal, Lognormal, or Gamma • To test for normal distribution, click on Normal from the drop-down menu list. • To test for gamma distribution, click on Gamma from the drop-down menu list. • To test for lognormal distribution, click on Lognormal from the drop-down menu. 8.4.1 Normal or Lognormal Distribution (Log-ROS Estimates) 1. Click Goodness-of-Fit Tests ^ With NDs ^ Log-ROS Estimates ^ Normal, Lognormal ProUCL 5.0 - [Oahu-chapters 4 -5.xls] ^^^^^H Statistical Tests Outlier Test Upper Limits/BTVs UCLs/EPCs Windows Help : Goodness-of-Fit Tests K Single Sample Hypothesis Two Sample Hypothesis > Oneway AN OVA > OLS Regression Trend Analysis > .9 0.344 15 0.737 515657826 515657826 6 789 Full (w/o NDs) > With NDs > 0 10 11 Exclude NDs > Normal-ROS Estimates > Gamma-ROS Estimates k Log-ROS Estimates > DL/2 Estimates G.O.F, Statistics K 12 13 14 Normal Gamma Lognormal 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be produced by using a group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result 133 ------- in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. When the option button: Normal or Lognormal is clicked, the following window appears. Select Goodness-of-Fit Options Select Confidence Coefficient O 99% ® 95% O 90% Select GOF Method * Shapiro Wilk O LiHiefors Graphs by Groups O hdviduaJ Graphs (•) Group Graphs OK o The default option for the Confidence Coefficient is 95%. o The default GOF test Method is Shapiro-Wilk. o The default option for Graphs by Group is Group Graphs. If you want to display graphs for all selected variables individually, check the button next to Individual Graphs. • Click the OK button to continue or the Cancel button to cancel the option. • Click the OK button to continue or the Cancel button to cancel the GOF tests. Example 8-2c (continued). Consider the arsenic Oahu data set with NDs considered earlier in this chapter. The lognormal GOF test results on LROS data (detected and imputed LROS NDs) is shown in the following GOF Q-Q plot. 134 ------- Output Screen for Lognormal Distribution (Log-ROS Estimates) Selected Options: Shapiro Wilk test with Best Line Fit Lognormal Q-O Plot for Arsenic Statistics using ROS Lognormal Imputed Estimates Theoretical Quantiles (Standard Normal) Imputed NDs Displayed in smaller font Mean ' -0.209 Sd - 0 571 Slope = 0.568 Intercept * -0.20S Correlation. R = 0.363 Shapiro-Will Test AppFox.TestValue-OS24 p-Value - 0 0742 Note: The font size of ND values is smaller than that of the detected values. 8.4.2 Gamma Distribution (Gamma-ROS Estimates) 1. Click Goodness-of-Fit Tests ^ With NDs ^ Gamma-ROS Estimates ^ Gamma Statistical Tests | Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests > Single Sample Hypothesis t Two Sample Hypothesis > Oneway ANOVA > OLS Regression Trend Analysis > 6 7 8 9 Full (w/o NDs) > With NDs ^ 1 1 1 1 9^ 0.344 515657826 1 10 11 Exclude NDs > Normal-ROS Estimates k Gamma-ROS Estimates > Log-ROS Estimates > DL/2 Estimates ^ G.O.F. Statistics 12 13 14 Normal Gamma Lognormal 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be generated by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. • When the option button (Gamma) is clicked, the following window will be shown. 135 ------- Select Goodness-of-Fit Options Select Confidence Coefficient O 99* • 95% O 90* Select GOF Method Anderson Darling • KolmogorDV Smimov Graphs by Groups O Individual Graphs * Group Graphs o The default option for the Confidence Coefficient is 95%. o The default GOF test Method is Anderson Darling. o The default option for Graph by Groups is Group Graphs. If you want to generate separate graphs for all selected variables, the check the button next to Individual Graphs. • Click the OK button to continue or the Cancel button to cancel the GOF tests. Example 8-2d (continued). Consider the arsenic Oahu data set with NDs considered earlier. The gamma GOF test results on GROS data (detected and imputed GROS NDs) are shown in the following GOF Q-Q plot. Output Screen for Gamma Distribution (Gamma-ROS Estimates) Selected Options: Anderson Darling Gamma Q-Q Plot for Arsenic Statistics using ROS Gamma Imputed Estimates Theoretical Ouantiles of Garnma Distribution Imputed NDs Displayed in smaller font 136 ------- Note: The font size of ND values in the above graph (and in all GOF graphs) is smaller than that of detected values. 8.5 Goodness-of-Fit Tests with DL/2 Estimates 1. Click Goodness-of-Fit Tests ^ With NDs ^ DL/2 Estimates 2. Select the distribution to be tested: Normal, Gamma, or Lognormal • To test for normality, click on Normal from the drop-down menu list. • To test for lognormality, click on Lognormal from the drop-down menu list. • To test for a gamma distribution, click on Gamma from the drop-down menu list. 8.5.1 Normal or Lognormal Distribution (DL/2 Estimates) 1. Click Goodness-of-Fit Tests ^ With NDs ^ DL/2 Estimates ^ Normal or Lognormal ProUCL 5.0 - [Oahu-chapters 4 -5.xls] Statistical Tests I Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests t Single Sample Hypothesis Two Sample Hypothesis Oneway ANOVA OLS Regression Trend Analysis 3 £ TV .' PT~ C.737 515657826 515657826 > 6 7 2 Full (w/o NDs) > With NDs K 1 1 1 1 1 0 9 Exclud Norm; Gamrr Log-R 10 eNDs I-ROS Estin a-ROSEstir 3S Estimate 11 K nates > nates K S K DL/2 Estimates > G.O.F. Statistics 12 13 14 Normal Gamma Lognormal 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • If graphs have to be generated by using a group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. The rest of the process to determine the distribution (normal, lognormal, and gamma) of the data set thus obtained is the same as described in earlier sections. 8.6 Goodness-of-Fit Test Statistics The G.O.F. option displays all GOF test statistics available in ProUCL. This option is used when the user does not know which GOF test to use to determine the data distribution. Based upon the information 137 ------- provided by the GOF test results, the user can perform an appropriate GOF test to generate GOF Q-Q plot based upon the hypothesized distribution. This option is available for uncensored as well as left censored data sets. Input and output screens associated with the G.O.F statistics option for data sets with NDs are summarized as follows. 1. Click Goodness-of-Fit ^ With NDs ^ G.O.F. Statistics «3 File Edit Stats/Sample Sizes Graphs Navigation Panel Name Worksheet ;ds Well 10jds WMW-rath NDsxIs ASHALL7groupsxls Box Plot Full.gst Box Plot Full_a.gst Box Plot Full_b.gst 1 2 3 4 5 6 7 0 backgroun 1 4 5 7 12 15 18 Statistical Tests | Upper Limits/BTVs UCLs/EPCs Outlier Tests ^ Goodness-of-Fit Tests * Single Sample Hypothesis ^ Two Sample Hypothesis Oneway ANOVA OLS Regression Trend Analysis 34 1 1 " 5 Windows Help S 7 Full (w/o NDs) t With NDs t I B Exclude NDs N G L D ormal-RQS amma-RO? Dg-ROS Est b'2 Estimat 9 10 N Estimates ^ Estimates ^ mates ^ es ^ G.O.F. Statistics 11 2. The Select Variables screen (Chapter 3) will appear. • Select one or more variable(s) from the Select Variables screen. • When the option button is clicked, the following window will be shown. GOF_Conf Level F Select Confidence Coefficient O 397. © 95% O 90% OK Caned • The default confidence level is 95%. • Click the OK button to continue or the Cancel button to cancel the option. 138 ------- Example 8-2e (continued). Consider the arsenic Oahu data set with NDs discussed earlier. Partial GOF test results, obtained using the G.O.F. Statistics option, are summarized in the following table. Sample Output Screen for G.O.F. Test Statistics on Data Sets with Nondetect Observations Asenic Raw Statistics Statistics (Non-Detects Only) Statistics (Detects Only) Statistics (All: NDs treated as D Lvalue) Statistics {All: NDs treated as DL/2 value) Statistics (Normal ROS Imputed Data) Statistics (Gamma ROS Imputed Data) Statistics (LDgnormal ROS Imputed Data} Statistics (Detects Only) Statistics (NDs = DL) Statistics (NDs = DL/2) Statistics (Gamma ROS Estimates) Statistics (Lognormal ROS Estimates) Num Obs 24 Num Miss 0 Num Valid 24 Number Minimum Maximum 13 0.9 2 11 0.5 3.2 24 0.5 3.2 24 0.45 3.2 24 -0.09S5 3.2 24 0.119 3.2 24 0.349 3.2 K hat K Star Theta hat 2.257 1.702 0.54B 3.538 3.124 C.4C6 3.233 2.857 0.31 2.071 1.84 0.461 Detects 11 Mean 1.ECS 1.236 1.438 1.002 0.997 0.956 0.972 Log Mean -C.C2EE 0.215 -0.16 -0.209 NDs 13 Median 2 0.7 1.25 0.95 0.737 0.7 0.7 Log Stdv 0.694 0.574 0.542 0.571 *ANDs 54.17% SD 0.517 0.965 0.761 0.699 0.776 0.75B 0.718 LogCV -27.26 2.669 -3.381 -2.727 Normal GOF Test Results No NDs NDs=DL NDs = DL/2NormalROS Correlation Coefficient R Shapiro-Wilk (Detects Only) Shapiro-Wilk (NDs = DL) Shapiro-Wilk (NDs = DL/2) Shapiro-Wilk (Normal ROS Estimates) Lilliefors (Detects Only) Lilliefors (NDs = DL) Lilliefors (NDs = DL/2) Lilliefors (Normal ROS Estimates) 0.887 0.948 0.833 Test value 0.777 0.89 0.701 O.S6S 0.273 0.217 0.335 0.17 Crit. (0.05) O.S5 0.916 0.916 0.916 0.251 0.177 0.177 0.177 0.828 Conclusion with Alpha(0.05) Data Not Normal Data Not Normal Data Not Normal Data Not Normal Data Not Normal Data Not Normal Data Not Normal Data Appear Normal 139 ------- Sample Output Screen for G.O.F. Test Statistics on Data Sets with Nondetect Observations (continued) Gamma GOF Test Results NoNDs NDs = DL NDs = DL/2Gamma ROJ Correlation Coefficient R 0.964 0.956 0.924 0.975 Test value Crit. (0.05) Anderson-Darling (Detects Only) 0.787 0.738 Conclusion with Alpha(O.OS) Kolmogorov-Smirnov (Detects Only) 0.254 0.258 Anderson-Darling (NDs = DL) 0.98 0.75 Kolmogorov-Smirnov (NDs = DL) 0.214 0.179 Anderson-Darling (NDs = DL/2) 1.492 0.751 Kolmogorov-Smirnov (NDs = DU2) 0.261 0.179 Anderson-Darling (Gamma ROS Estimates) 0.48 0.755 Kolmogorov-Smirnov (Gamma ROS Est.) 0.126 0.18 Detected Data appear Approximate Gamma Disti Data Not Gamma Distributed Data Not Gamma Distributed Data Appear Gamma Distributed Lognormal GOF Test Results Correlation Coefficient R NoNDs 0.939 NDs = DL 0.959 s = DIJ2 Log ROS 0.933 0.963 Test value S hapiro-Wilk (D elects 0 nly) 0.86 Shapiro-Wilk (NDs = DL) 0.906 0.365 0.924 0.229 0.214 0.217 0.143 Shapiro-Wilk (NDs = DL/2] Shapiro-Wilk (Lognormal ROS Estimates) Lilliefors (Detects Only) Lilliefors(NDs = DL) Lilliefors (NDs = DL/2) Crit. (0.05) Conclusion with Alpha(0.05) 0.85 Data Appear Lognormal 0.916 Data Not Lognormal 0.916 Data Not Lognormal Lilliefors (Lognormal ROS Estimates) 0.916 D ata Appear Lognormal 0.251 D ata Appear Lognormal 0.177 Data Not Lognormal 0.177 Data Not Lognormal 0.177 Data Appear Lognormal Note: Substitution methods such as DL 01 DL/2 are not recommended. 140 ------- Chapter 9 Single-Sample and Two-Sample Hypotheses Testing Approaches This chapter illustrates single-sample and two-sample parametric and nonparametric hypotheses testing approaches as incorporated in the ProUCL software. All hypothesis tests are available under the Statistical Tests module of ProUCL 5.0/ProUCL 5.1. ProUCL software can perform these hypotheses tests on data sets with and without ND observations. It should be pointed out that, when one wants to use two-sample hypotheses tests on data sets with NDs, ProUCL assumes that samples from both of the samples/groups have ND observations. All this means is that, a ND column (with 0 or 1 entries only) needs to be provided for the variable in each of the two samples. This has to be done even if one of the samples (e.g., Site) has all detected entries; in this case the associated ND column will have all entries equal to '!.' This will allow the user to compare two groups (e.g., arsenic in background vs. site samples) with one of the groups having some NDs and the other group having all detected data. 9.1 Single-Sample Hypotheses Tests In many environmental applications, single-sample hypotheses tests are used to compare site data with pre-specified Cs or CLs. The single-sample hypotheses tests are useful when the environmental parameters such as the Cs, action level, or CLs are known, and the objective is to compare site concentrations with those known pre-established threshold values. Specifically, a t-test (or a sign test) may be used to verify the attainment of cleanup levels at an AOC after a remediation activity; and a test for proportion may be used to verify if the proportion of exceedances of an action level (or a compliance limit) by sample concentrations collected from an AOC (or a MW) exceeds a certain specified proportion (e.g., 1%, 5%, 10%). ProUCL 5.1 can perform these hypotheses tests on data sets with and without ND observations. However, it should be noted that for single-sample hypotheses tests (e.g., sign test, proportion test) used to compare site mean/median concentration level with a Csor a CL (e.g., proportion test), all NDs (if any) should lie below the cleanup standard, Cs. For proper use of these hypotheses testing approaches, the differences between these tests should be noted and understood. Specifically, a t-test or a Wilcoxon Signed Rank (WSR) test is used to compare the measures of location and central tendencies (e.g., mean, median) of a site area (e.g., AOC) to a cleanup standard, Cs, or action level also representing a measure of central tendency (e.g., mean, median); whereas, a proportion test compares if the proportion of site observations from an AOC exceeding a CL exceeds a specified proportion, P0 (e.g., 5%, 10%). ProUCL has graphical methods that may be used to visually compare the concentrations of a site AOC with an action level. This can be done using a box plot of site data with horizontal lines displayed at action levels on the same graph. The details of the various single-sample hypotheses testing approaches are provided in the associated ProUCL Technical Guide. 141 ------- 9.1.1 Single-Sample Hypothesis Testing for Full Data without Nondetects 1. Click Single Sample Hypothesis ^- Full (w/o NDs) ProUCL 5.0 - [Zn-Cu-ND-data.xls Statistical Tests I Upper Limits/BTVs UCLs/EPCs Window: Help 2. Outlier Tests ^ Goodness-of-Fit Tests ^ Single Sample Hypothesis ^ Two Sample Hypothesis > Oneway AN OVA > OLS Regression Trend Analysis » an 1 1 e 7 8 9 Full (w/o NDs) t With NDs ^ 10 11 12 tTest Proportion Sign Test Wilcoxon Signed Rank Select Full (w/o NDs) - This option is used for full data sets without nondetects. • To perform a t-test, click on t-Test from the drop-down menu as shown above. • To perform a Proportion test, click on Proportion from the drop-down menu. • To run a Sign test, click on Sign test from the drop-down menu. • To run a Wilcoxon Signed Rank (WSR) test, click on Wilcoxon Signed Rank from the drop-down menu. All single-sample hypothesis tests for uncensored and left-censored data sets can be performed by a group variable. The user selects a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. Select Variable Available Variables Selected Variable Name ID Y3 1 » Name ID X3 0 « < ^H > ^m Select Group Column (Optional) 1 -1 Options <— Select an action level OK Cancel 142 ------- 9.1.1.1 Single-Sample t-Test 1. Click Single Sample Hypothesis ^ Full (w/o NDs) > t-Test . EPA (2006)-cl Statistical Tests I Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests ^ Goodness- of -Fit Tests > Single Sample Hypothesis > Two Sample Hypothesis > Oneway AN OVA ^ Trend Analysis > 6 Fi W 7 jll (w/o ND. ith NDs 8 0 t tt 9 10 [ tTest Proportion Sign Test Wilcoxon > 11 gned Rank 12 2. The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. • When the Options button is clicked, the following window will be shown. Select Uncensored t Test Options Select Null Hypothesis Form » Sample Mean <= Action Level (Form 1) O Sample Mean >= Action Level (Form 2) O Sample Mean => Action Level - S (Form 2) O Sample Mean = Action Level (Two Sided) Confidence Level Substantial Difference. S (Form 2) Action Level OK Cancel Specify the Confidence Level; default is 0.95. o Specify meaningful values for Substantial Difference, S and the Action Level. The default choice for S is "0." o Select form of Null Hypothesis; default is Sample Mean <= Action Level (Form 1). o Click on OK button to continue or on Cancel button to cancel the test. 143 ------- Example 9-la. Consider the WSR data set described in EPA (2006a). One Sample t-test results are summarized as follows. Output for Single-Sample t-Test (Full Data w/o NDs) From File WSR EPA (2006H:hapter S-USerxIs Full Precision OFF Confidence Coefficient 35% Substantial Difference 0.000 Action Levd 800.000 Selected Null Hypothesis Mean <= Action Level (Form 1) Alternative hypothesis Mean > the Action Level WSR1 One Sample t-Test Raw Statistics Number of Valid Observations 10 Number of Distinct Observations 10 Minimum 750 Maximum 1161 Mean 925.7 Median 388 SD 136.7 SE of Mean 43.24 HO: Sample Mean <= 800 (Form 1) Test Value 2.907 Degrees of Freedom 9 Critical Value (0.05) 1.833 P-Value 0.00869 Conclusion with Alpha = 0.05 Reject HO. Conclude Mean > 800 P-Value < Alpha (0.05) 9.1.1.2 Single-Sample Proportion Test 1. Click Single Sample Hypothesis ^ Full (w/o NDs) ^ Proportion Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests > Single Sample Hypothesis > Two Sample Hypothesis > Oneway AN OVA > OLS Regression Trend Analysis > 6 7 8 9 Full (w/o NDs) > 1 With NDs > 10 11 12 tTest Proportion Sign Test Wilcoxon Signed Rank 144 ------- 2. The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. • When the Options button is clicked, the following window will be shown. Select Uncensored Proportion Opti Select Null Hypothesis Form ® Sample 1 Proportion. P <= PO (Fonn 1) O Sample 1 Proportion. P >= PO (Form 2) O Sample 1 Proportion. P = PO (Two Sided) Confidence Level Proportion. PO Action Level (For % Exceedances) 0.95 0.2 800 OK Cancel o Specify the Confidence level; default is 0.95. o Specify the Proportion level and a meaningful Action Level. o Select the form of Null Hypothesis; default is Sample 1 Proportion <= PO (Form 1). o Click on OK button to continue or on Cancel button to cancel the test. 145 ------- Example 9-lb (continued). Consider the WSR data set described in EPA (2006a). One Sample proportion test results are summarized as follows. Output for Single-Sample Proportion Test (Full Data without NDs) User Selected Options Date/Time of Computation 3/17/201310:29:33 P M From File WSR EPA (2006}chapter 9-USerjds OFF 95X Full Precision Confidence Coefficient User Specified Proportion 0.200 (PO of Exceedances of Action Level) Action/compliance Limit 800.000 Select Null Hypothesis Sample Proportion. P of Exceedances of Action Level >= User Specified Proportion (Form 2} Alternative Hypothesis Sample Proportion. P of Exceedances of Action Level cthe User Specified Proportion WSR1 One Sample Proportion Test Raw Statistics Number of Valid Observations Number of Distinct Observations Minimum 10 10 750 Maximum Mean Median SD 1161 925.7 SE of Mean Number of Exceedances Sample Proportion of Exceedances HO: Sanple Proportion >= 0.2 [Form 2) 136.7 43.24 B 0.8 Conclusion with Alpha = 0.05 Do Not Reject HO. Conclude P-Value > Alpha (0.05) ) 1 'roportion >= 02 9.1.1.3 Single-Sample Sign Test 1. Click Single Sample Hypothesis ^ Full (w/o NDs) > Sign test Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests ^ Goodness-of-Fit Tests ^ Single Sample Hypothesis ^ Two Sa m p 1 e Hy p oth esi s > Oneway AN OVA > OLS Regression Trend Analysis > 6 7 8 9 10 Full (w/o NDs) > With NDs > 11 12 tTest Proportion Sign Test Wilcoxon Signed Rank 146 ------- 2. The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. • When the Options button is clicked, the following window will be shown. Select Uncensored Sign Test Options Select Null Hypothesis Form <.._.'! Sample Median <= Action Level (Form 1) (j Sample Median >= Action Level (Form 2) O Sample Median >= Action Level + S (Form 2) • Sample Median = Action Level (Two Sided) Confidence Level Substantial Difference. S (Form 2) Action Level Cancel o Specify the Confidence Level; default choice is 0.95. o Specify meaningful values for Substantial Difference, S and Action Level. o Select the form of Null Hypothesis; default is Sample Median <= Action Level (Form 1). o Click on OK button to continue or on Cancel button to cancel the test. 147 ------- Example 9-lc (continued). Consider the WSR data set described in EPA (2006a). The Sign test results are summarized as follows. Output for Single-Sample Sign Test (Full Data without NDs) From Rle Full Precision Confidence Coefficient Substantial Difference Action Level Selected Null Hypothesis Alternative Hypothesis WSR EPA(200ffH;hapterS-USer.xls OFF 95% 0.000 5.000 Median = Action/compliance Limit (2 Sided Alternative) Median o Action/compliance Limit WSR2 One Sample Sign Test Raw Satisfies Number of Valid Observations 49 Number of Distinct Observations 44 Minimum 1.09 Maximum 7.5 Mean 5.048 Median 5.55 SD 1.775 SE of Mean 0.254 Number Above .Action Level 29 Number Equal Action Level 0 Number Below .Action Level 20 HO: Sample Median = 5 Large Sample Z Test Statistic 1.236 P-Value 0.199 Conclusion with Alpha = 0.05 Do Not Reject HO at the specified level of significance (0.05). Conclude Median = 5 P-Value > Alpha (0.05) 148 ------- 9.1.1.4 Single-Sample Wilcoxon Signed Rank (WSR) Test 1. Click Single Sample Hypothesis ^- Full (w/o NDs) ^- Wilcoxon Signed Rank CL 5.0 - [WSR EPA (2006)-chapter 9-U Statistical Tests | Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests *• Goodness-of-Fit Tests t Single Sample Hypothesis * Two Sample Hypothesis ^ Oneway AN OVA > OLS Regression Trend Analysis t 678 5 10 Full [w/o NDs) > With NDs > 11 12 tTest Proportion Sign Test Wilcoxon Signed Rank 2. The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. • When the Options button is clicked, the following window will be shown. Select Null Hypothesis Form • Mean/Median <= Action Level (Form 1) O Mean/Median >= Action Level (Form 2) O Mean/Median >= Action Level + S (Form 2) O Mean/Median = Action Level (Two Sided] Confidence Level Action Level OK Cancel o Specify the Confidence Level; default is 0.95. o Specify meaningful values for Substantial Difference, S, and Action Level. o Select form of Null Hypothesis; default is Mean/Median <= Action Level (Form 1). 149 ------- o Click on OK button to continue or on Cancel button to cancel the test. Example 9-ld (continued). Consider the WSR data set described in EPA (2006a). One Sample WSRtest results are summarized as follows. Output for Single-Sample Wilcoxon Signed Rank Test (Full Data without NDs) Confidence Coefficient 95% Substantial Difference 0.000 Action Level 800.000 Selected Null Hypothesis Mean/Median <= .Action Level (Form 1} Alternative Hypothesis Mean/Median > the Action Level WSR1 One Sample Wilcoxon Spied Rank Test Raw Statistics Number of Valid Observations 10 Number of Distinct Observations 10 Minimum 750 Maximum 1161 Mean 925.7 Median 883 SD 136.7 SE of Mean 43.24 Number Above .Action Level 8 Number Equal .Action Level G Number Below Action Level 2 T-plus 50 T-minus 5 HO: Sample Mean/Median <= 800 (Form 1) Exact Test Statistic 50 Critical Value {0.05} 45 P-Value 0.0098 Conclusion with Alpha = 0.05 Reject HO. Conclude Mean/Median > 800 P-Value < Alpha {0.05) 150 ------- 9.1.2 Single-Sample Hypothesis Testing for Data Sets with Nondetects Most of the one-sample tests such as the Proportion test and the Sign test on data sets with ND values assume that all ND observations lie below the specified action level, Ao. These single-sample tests are not performed if ND observations exceed the action levels. Single-sample hypothesis tests for data sets with NDs are shown in the following screen shot. 1. 2. Click on Single Sample Hypothesis ^- With NDs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of- Fit Tests ^ Single Sample Hypothesis > Two Sam pie Hypothesis > Oneway AN OVA > Trend Analysis > 6 Fi W 7 ill [w/o ND ith NDs Q 0 > > 9 10 11 12 Proportion Sign Test Wilcoxon Signed Rank Select the With NDs option • To perform a proportion test, click on Proportion from the drop-down menu. • To perform a sign test, click on Sign test from the drop-down menu. • To perform a Wilcoxon Signed Rank test, click on Wilcoxon Signed Rank from the drop-down menu list. 9.1.2.1 Single Proportion Test on Data Sets with NDs 1. Click Single Sample Hypothesis ^- With NDs ^- Proportion ProUCL 5.0 - Statistical Tests | Upper Limits/BTVs Outlier Tests > G o o d n ess- of - Fit Tests > Single Sample Hypothesis » Two Sample Hypothesis » Oneway AN OVA ^ OLS Regression Trend Analysis ^ UCLs/EPCs Windows Help 673 Full (w/o NDs) f Wrth NDs > 3 1C 11 12 Proportion Sign Test Wilcoxon Signed Rank 151 ------- 2. The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. • If hypothesis test has to be performed by using a Group variable, then select a group variable by clicking the arrow below the Select Group Column (Optional) button. This will result in a drop-down list of available variables. The user should select and click on an appropriate variable representing a group variable. This option has been used in the following screen shot for the single-sample proportion test. Available Variables Selected Variable Name Cu ID D Name Zn ID 1 Select Group Column (Optional) Options <- Sdect an action level [ OK Caned When the Options button is clicked, the following window will be shown. Select Censored Proportion Options Select Null Hypothesis Form @ Sample 1 Proportion. P <= PO (Form 1) O Sanple 1 Proportion. P >= PO (Form 2) O Sai^le 1 Proportion. P = PO (Two Sided) Confidence Level Proportion. PO Action Level (For % Bcceedances) tions OK Cancel o Specify the Confidence Level; default is 0.95. o Specify meaningful values for Proportion and the Action Level (=15 here). o Select form of Null Hypothesis; default is Sample 1 Proportion, P <= PO (Form 1). o Click on OK button to continue or on Cancel button to cancel the test. 152 ------- Example 9-2a. Consider the copper and zinc data set collected from two zones: Alluvial Fan and Basin Trough discussed in the literature (Helsel 2012, NADA in R [Helsel 2013]). This data set is used here to illustrate the one sample proportion test on a data set with NDs. The output sheet generated by ProUCL 5.1 is presented below. Output for Single-Sample Proportion Test (with NDs) by Groups: Alluvial Fan and Basin Trough User Selected Options Date/Time of Computation 3/1 a/2013 3:55:58 AM From File Zn-Cu-ND-dataxis Full Precision OFF Confidence Coefficient 35% User Specified Proportion Action Level " 5K :?D of Exceedances of Action Level) 15.000 Select Null Hypothesis Sample Proportion, P of Btceedances of Action Level c= User Specified Proportion (Form 1) Alternative Hypothesis Sample Proportion, P uf F_-ceedancer of Action Level > User Specified Proportion Zn (alluvial fan) One Sample Proportion Ted Note: All nondetects are treated as detects at values (eg.. DLs) included in Data Rle Raw Statistics Number of Valid Data 67 Number of Missing Observations 1 Number of Distinct Data 1 5 Numberof Non-Detects 16 Number of Detects 51 Percent Non-Detects 23.85% Minimum Non-detect 3 Maximum Non-detect 10 Minimum Detect 5 Maximum Detect 62D Mean of Detects 27.88 Median of Detects 11 SD of Detects 35.02 Number of Exceedances 24 Sample Proportion of Exceedances 0.358 ( HO: Sample Proportion <= 09 (Form 1) Large Sample i-Test Statistic -14.58 Critical Value (0.05) 1.645 P-Value 1 Conclusion with Alpha = 0 05 Do Not Reject HO. Conclude Sample Proportion <= 0.9 P-Value > Alpha (0 OS) Zn (basin trough) One Sample Proportion Test Note: All nondetects are treated as detects at values (eg. DLs) included in Data H Raw Statistics Number of Valid Data 50 Number of Distinct Data 20 Numberof Non-Detects 4 Number of Detects 46 Percent Non-Detects S.OOH Minimum Non-detect 3 Maximum Non-detect 10 Minimum Detect 3 Maximum Detect 90 Mean of Detects 23.13 SD of Detects 19.03 Number of Exceedances 27 Sample Proportion of Exceedances 0.54 e HO: Sample Proportion <= 0.9 (Form 1) Exact P-Value 1 Conclusion with Alpha = 0.05 Do Not Reject HO. Conclude Sample Proportion <= 0.9 P-Value > Alpha (0.05) 153 ------- 9.1.2.2 Single-Sample Sign Test with NDs 1. Click Single Sample Hypothesis ^ With NDs ^ Sign test 2. Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests » G o o d n ess- of - Fit Tests » Single Sample Hypothesis » Two Sample Hypothesis » Oneway AN OVA > OLS Regression Trend Analysis » 6 7 8 9 10 Full (w/o NDs) (• With NDs > 11 12 Proportion Sign Test Wilcoxon Signed Rank E The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. • When the Options button is clicked, the following window will be shown. Select Censored Sign Test Options Select Null Hypothesis Form O Sample Median <= Action Level (Form 1) O Sample Median >= Action Level (Form 2) • Sample Median = Action Level (Two Sided) Confidence Level Action Level OK Cancel o Specify the Confidence Level; default is 0.95. o Select an Action Level. o Select the form of Null Hypothesis; default is Sample Median <= Action Level (Form 1). o Click on OK button to continue or on Cancel button to cancel the test. 154 ------- Example 9-2b (continued). Consider the copper and zinc data set collected from two zones: Alluvial Fan and Basin Trough discussed above. This data set is used here to illustrate the Single-Sample Sign test on a data set with NDs. The output sheet generated by ProUCL 5.0 follows. Output for Single-Sample Sign Test (Data with Nondetects) Zn (alluvial fan) Selected Null Hypothesis .Alternative Hypothesis Median = Action/compliance Limrt (Two Sided Alternative} Median o Action/compliance Limit One Sample Sign Test Note All nondetects are treated as detects at values (e.g.. DU) included in Data Rle Raw Statistics Number of Valid Data 67 Number of Missing Observations 1 Number of Distinct Data 19 Number of Non-Detects 16 Number of Detects 51 Percent Non-Detects 23.SS% Minimum Non-detect 3 Maximum Non-detect Minimum Detect Maximum Detect Mean of Detects Median of Detects SD of Detects Number Above Action Level Number Equal Action Level Number Below Action Level 10 5 620 27.88 11 85.02 24 0 43 HO: Sample Median = 15 Standardized Test Value using Normal Appx. -2.321 P-Value 0.0203 Conclusion with Alpha = 0.05 Reject HO at the specified level of significance (0.05). Conclude Median o 15 P-Vdue < Alpha (0.05) 9.1.2.3 Single-Sample Wilcoxon Signed Rank Test with NDs 1. Click Single Sample Hypothesis ^- With NDs ^- Wilcoxon Signed Rank ProUCL 5.0 - [Zn-Cu-ND- Statistkal Tests I Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests > Single Sample Hypothesis ^ Two Sample Hypothesis » Oneway AN OVA > OLS Regression Trend Analysis > 6 7 & 5 Full (w/o NDs) > With NDs > 10 Proportion Sign Test 11 12 Wilcoxon Signed Rank 155 ------- 2. The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. • When the Options button is clicked, the following window will be shown. < Select Censored WSR Test Options Select Null Hypothesis Form Sample Mean/Median <= Action Level (Form 1) • Sample Mean/Median >= Action Level (Form 2) O Sample Mean/Median = Action Level (Two Sided) Confidence Level Action Level OK Cancel o Specify the Confidence Level; default is 0.95. o Specify an Action Level. o Select form of Null Hypothesis; default is Sample Mean/Median <= Action Level (Form 1). o Click on OK button to continue or on Cancel button to cancel the test. Example 9-2c (continued). Consider the copper and zinc data set collected from two zones: Alluvial Fan and Basin Trough discussed earlier in this chapter. This data set is used here to illustrate one sample Wilcoxon Signed Rank test on a data set with NDs. The output sheet generated by ProUCL 5.0 is provided as follows. Output for Single-Sample Wilcoxon Signed Rank Test (Data with Nondetects) One Sample Wilcoxon Signed Rank Test for Data Sets with Non-Detects User Selected Options Date/Time of Computation 3/1S/2013 1:43:46 PM From File Zn-Cu-ND-datajds Full Precision OFF Confidence Coefficient 95% Action Level 15.000 Selected Null Hypothesis Mean/Median >= Action Level (Form 2) Alternative Hypothesis Mean/Median =: the Action Level 156 ------- Output for Single-Sample Wilcoxon Signed Rank Test (Data with Nondetects)- continued Zn (basin trough) One Sample Wilcoxon Signed Rank Test Raw Statistics Number of Valid Data Number of Distinct Data Number of Non-Detects Number of Detects Percent Non-Detects Minimum Non-detect Maximum Non-detect Minimum Detect Maximum Detect Mean of Detects Median of Detects SD of Detects Median of Processed Data used in WSR Number Above Action Level Number Equal Action Level Number Below Action Level T-plus T-minus 50 20 4 46 8.00X 3 10 3 90 23.13 20 1903 1S.5 27 1 22 764 461 HO: Sample Median >= 15 (Form 2) Large Sample z-Test Statistic Critical Value (0.05) P-Value Conclusion with Alpha = 0.05 Do Not Reject HO. Conclude Mean/Meet; F -Value > Alpha (0.05) 1.269 -1.645 0898 n>= 15 Dataset contains multiple Non Detect values! All NDs are replaced by their respective DL/2 9.2 Two-Sample Hypotheses Testing Approaches The two-sample hypotheses testing approaches available in ProUCL are described in this section. Like Single-Sample Hypothesis, the Two-Sample Hypothesis options are available under the Statistical Tests module of ProUCL 5.0/ProUCL 5.1. These approaches are used to compare the parameters and distributions of two populations (e.g., Background vs. AOC) based upon data sets collected from those populations. Several forms (Form 1 and Form 2, and Form 2 with Substantial Difference, S) of the two- sample hypothesis testing approaches are available in ProUCL. The methods are available for full- uncensored data sets as well as for data sets with ND observations with multiple detection limits. • Full (w/o NDs) - performs parametric and nonparametric hypothesis tests on uncensored data sets consisting of all detected values. The following tests are available: o Student's t and Satterthwaite tests to compare the means of two populations (e.g. Background versus AOC). o F-test to the check the equality of dispersions of two populations. o Two-sample nonparametric Wilcoxon-Mann-Whitney (WMW) test. This test is equivalent to Wilcoxon Rank Sum (WRS) test. • With NDs - performs hypothesis tests on left-censored data sets consisting of detected and ND values. The following tests are available: o Wilcoxon-Mann-Whitney test. All observations (including detected values) below the highest detection limit are treated as ND (less than the highest DL) values. 157 ------- o Gehan's test is useful when multiple detection limits may be present. o Tarone-Ware test is useful when multiple detection limits may be present. The details of these methods can be found in the ProUCL Technical Guides (2013, 2015) and are also available in EPA (2002b, 2006a, 2009a, 2009b). It is emphasized that the use of informal graphical displays (e.g., side-by-side box plots, multiple Q-Q plots) should always accompany the formal hypothesis testing approaches listed above. This is especially warranted when data sets may consist of NDs with multiple detection limits and observations from multiple populations (e.g., mixture samples collected from various onsite locations) and outliers. Notes: As mentioned before, when one wants to use two-sample hypotheses tests on data sets with NDs, ProUCL assumes that samples from both of the groups have ND observations. This may not be the case, as data from a polluted site may not have any ND observations. ProUCL can handle such data sets; the user will have to provide a ND column (with 0 or 1 entries only) for the selected variable of each of the two samples/groups. Thus when one of the samples (e.g., site arsenic) has no ND value, the user supplies an associated ND column with all entries equal to T. This will allow the user to compare two groups (e.g., arsenic in background vs. site samples) with one of the groups having some NDs and the other group having all detected data. 9.2.1 Two-Sample Hypothesis Tests for Full Data Full (w/o NDs): This option is used to analyze data sets consisting of all detected values. The following two-sample tests are available in ProUCL 5.1. • Student's t and Satterthwaite tests to compare the means of two populations (e.g., Background versus AOC). • F-test is also available to test the equality of dispersions of two populations. • Two-sample nonparametric Wilcoxon-Mann-Whitney (WMW) test. • Student's!-Test o Based upon collected data sets, this test is used to compare the mean concentrations of two populations/groups provided the populations are normally distributed. The data sets are represented by independent random observations, XI, X2, . . ., Xn collected from one population (e.g., site), and independent random observations, Yl, Y2, . . . , Ym collected from another (e.g., background) population. The same terminology is used for all other two-sample tests discussed in the following sub-sections of this section. o Student's t-test also assumes that the spreads (variances) of the two populations are approximately equal. o The F-test can be used to the check the equality of dispersions of two populations. A couple of other tests (e.g., Levene 1960) are also available in the literature to compare the variances of two populations. Since the F-test performs fairly well, other tests are not included in the ProUCL software. For more details refer to ProUCL Technical Guides. 158 ------- • Satterthwaite t-Test This test is used to compare the means of two populations when the variances of those populations may not be equal. As mentioned before, the F-distribution based test can be used to verify the equality of dispersions of the two populations. However, this test alone is more powerful test to compare the means of two populations. • Test for Equality of two Dispersions (F-test) o This test is used to determine whether the true underlying variances of two populations are equal. Usually the F-test is employed as a preliminary test, before conducting the two- sample t-test for testing the equality of means of two populations. o The assumptions underlying the F-test are that the two-samples represent independent random samples from two normal populations. The F-test for equality of variances is sensitive to departures from normality. • Two-Sample Nonparametric WMW Test o This test is used to determine the comparability of the two continuous data distributions. This test also assumes that the shapes (e.g., as determined by spread, skewness, and graphical displays) of the two populations are roughly equal. The test is often used to determine if the measures of central locations (mean, median) of the two populations are significantly different. o The Wilcoxon-Mann-Whitney test does not assume that the data are normally or log- normally distributed. For large samples (e.g., > 20), the distribution of the WMW test statistic can be approximated by a normal distribution. Notes: The use of the tests listed above is not recommended on log-transformed data sets, especially when the parameters of interests are the population means. In practice, cleanup and remediation decisions have to be made in the original scale based upon statistics and estimates computed in the original scale. The equality of means in log-scale does not necessarily imply the equality of means in the original scale. 1. Click on Two Sample Hypothesis ^- Full (w/o NDs) Upper Limits/BTVs UCLs/EPCs ProUCLS.O- Windows Help Outlier Tests ^ G o o d n ess- of - Fit Tests ^ Single Sample Hypothesis » Two Sample Hypothesis > Oneway AN OVA > OLS Regression Trend Analysis ^ 6 7 ln-89 4600 8 9 MW5 MN9 9 22DD Full (w/o NDs) > With NDs ^ 1790 1730 10 11 MN-99 2200 12 D.MN-99 1 tTest Wilcoxon-Mann-Whitney 9T 2150 9 222^ 21 ED 222 D 1 0 159 ------- 2. Select the Full (w/o NDs) option • To perform a t-test, click on t Test from the drop-down menu. • To perform a Wilcoxon-Mann-Whitney, click on Wilcoxon-Mann-Whitney from the drop-down menu list. 9.2.1.1 Two-Sample t- Test without NDs 1. Click on Two Sample Hypothesis ^ Full (w/o NDs) ^ t Test 2. The Select Variables screen will appear. • Select variable(s) from the Select Variables screen. Available Variables tVarial Without Group Variable Name ID Well ID 0 Mn 1 MW-ID 2 Manganese 3 MW-89 5 MWS 8 MN9 9 Count 48 32 32 MN99 index Sample 1 Sample 2 Name • Using Group Variable Variable Group Variable Sample 1 Sample 2 Name Mn-89 ID 6 Count 32 MW-89 ( Count = 32 ) V Without Group Variable: This option is used when the sampled data of the variable (e.g., lead) for the two populations (e.g., site vs. background) are given in separate columns. With Group Variable: This option is used when sampled data of the variable (e.g., lead) for the two populations (e.g., site vs. background) are given in the same column. The values are separated into different populations (groups) by the values of an associated Group ID Variable. The group variable may represent several populations (e.g., background, surface, subsurface, silt, clay, sand, several AOCs, MWs). The user can compare two groups at a time by using this option. When the Group option is used, the user then selects a variable by using the Group Variable Option. The user should select an appropriate variable representing a group variable. The user can use letters, numbers, or alphanumeric labels for the group names. 160 ------- When the Options button is clicked, the following window will be shown. Select t Test Options Select Null Hypothesis Form • Sample 1 <= Sample 2 (Form 1) O Sample 1 >= Sample 2 (Form 2) O Sample 1 >= Sample 2 + S (Ftxm 2) O Sample 1 = Sample 2 (Two Sided) Select Confidence Coefficient O 99 9* O 99.5% O 99* O 97.5% (§) 95% O 90% OK o Specify a useful Substantial Difference, S value. The default choice is 0. o Select the Confidence Coefficient. The default choice is 95%. o Select the form of Null Hypothesis. The default is Sample 1 <= Sample 2 (Form 1). o Click on OK button to continue or on Cancel button to cancel the option. • Click on OK button to continue or on Cancel button to cancel the Sample 1 versus Sample 2 Comparison. Example 9-3. Consider the manganese concentrations data set collected from three wells: MW1, an upgradient well, and MW8 and MW9, two downgradient wells. The two-sample t-test results, comparing Mn concentrations in MW8 vs. MW9, are described as follows. 161 ------- Output for Two-Sample t-Test (Full Data without NDs) Confidence Coefficient Substantial Difference (S) Selected Null Hypothesis Alternative Hypothesis 95*i 0.000 Sample 1 Mean = Sample 2 Mean (Two Sided Alternative} Sample 1 Mean o Sample 2 Mean Sample 1 Data: Mn-89(8) Sample 2 Data: Mn-KS'3) Number of Valid Raw Statistics Sample 1 Observations 16 Number of Distinct Observations Sample I vs Minimum Maximum Mean Median SD SE of Mean 16 1270 4600 1998 1750 S3S.B 2C3.7 Sample 2 16 15 1050 3080 1968 2C55 500.2 125 Sample 2 Two-Sample t-Test HO: Mean of Sample 1 = Mean of Sample 2 t-Test Value 0.123 0.123 Method DF Pooled (Equal Variance) 30 Welch-Satterthvvalte (Unequal Variam 24.5 Pooled SD: 690.548 Conclusion with Alpha = O.D50 Student! (Pooled): Do Not Reject HD, Conclude Sample 1 = Sample 2 Welch-Satterthwaite: Do Not Reject HD. Conclude Sample 1 = Sample 2 Lower C.Val Upper C.Val t (0.025) t (0.975) P-Value -2.042 2.W2 C.SC3 -2.0S4 2.064 C<.BC3 Test of Equality of Variances Variance of Sample 1 Variance of Sample 2 703523 250190 Numerator DF Denominator DF 15 15 Conclusion with Alpha = 0.05 Two variances appear to be equal F-Test Value 2.812 P-Value 0.054 162 ------- 9.2.12 Two-Sample Wilcoxon-Mann-Whitney (WMW) Test without NDs 1. Click on Two Sample Hypothesis Testing ^ Full (w/o NDs) ^ Wilcoxon-Mann-Whitney Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests K Goodness-of- Fit Tests > Single Sample Hypothesis k Two Sample Hypothesis ¥• Oneway AN OVA > OLS Regression Trend Analysis K 6 1n-89 7 S 5 MW9 MN9 10 4600 9 2200 Full (w/o NDs) > With NDs > 11 MN-99 12 D_MN-99 2200 1 tTest Wilcoxon-Mann-Whitney 1790 9 2150 173. 9 2220 2150 1 2220 D 2. The Select Variables screen will appear. Available Variables • Without Group Variable Sample 1 Sample 2 Name ID Count X3 0 24 Name ID Y3 1 Count 25 Using Group Variable Variable Group Variable Sample 1 Sample 2 Name ID Count Select variable(s) from the Select Variables screen. Without Group Variable: This option is used when the data values of the variable (arsenic) are given in separate columns. With Group Variable: This option is used when data of the variable (arsenic) are given in the same column. The values are separated into different samples (groups) by the values of an associated Group Variable. When the Group option is used, the user then selects a group variable/ID by using the Group Variable Option. The user should select an appropriate variable representing a 163 ------- group variable. The user can use letters, numbers, or alphanumeric labels for the group names. Notes: ProUCL documents have been written using environmental terminology such as performing background versus site comparisons. However, all tests and procedures incorporated in ProUCL can be used on data sets from any other application. For other applications such as comparing a new treatment drug versus older treatment drug, the group variable may represent the two groups: Control Drug and New Drug. • When the Options button is clicked, the following window is shown. a~? Select Wilcoxon- Mann- Whitney Select Null Hypothesis Form O Sample 1 <= Sample 2 (Form 1) O Sample 1 >= Sample 2 (Form 2) O Sample 1 >= Sample 2 + S (Form 2) * Sample 1 = Sample 2 (Two Sided) Select Confidence Coefficient O 99 9* O 99.5% O 99% O 37.5% •:•:• 95% O 90% o Specify a Substantial Difference, S value. The default choice is 0. o Choose the Confidence Coefficient. The default choice is 95%. o Select the form of Null Hypothesis. The default is Sample 1<= Sample 2 (Form 1). o Click on OK button to continue or on Cancel button to cancel the selected options. • Click on OK to continue or on Cancel to cancel Sample 1 vs. Sample 2 comparison. 164 ------- Example 9- 4. The two-sample Wilcoxon Mann Whitney (WMW) test results on a data set with ties are summarized as follows. Output for Two-Sample Wilcoxon-Mann-Whitney Test (Full Data with ties) Selected Null Hypothesis Sample 1 Mean/Median Alternative Hypothesis Sample 1 Mean/Median = Sample 2 Mean/Median (Two Sided Memative) o Sample 2 Mean/Median Sample 1 Data: X3 Sample 2 Data: Y3 Raw Statistics Sample 1 Number erf Valid Observations Number of Distinct Observations Minimum Maximum Mean Median SD SE of Mean 24 i 5.687 31.2 17.38 17.56 7.421 1.515 Sample 2 25 19 1,85 79.06 39.8 44.63 19.39 3.878 HO: Mean/Median of Sample 1 = Mean/Median of Sample 2 Sample 1 Rank Sum W-Stat 396 WMW U-Stat % Standardized WMW U-Stat -4.0B3 Mean (U) 300 SD(U)-Mjties 49.97 Lower Approximate U-Stat Critical Value (0.025) -1 .96 Upper Approximate U-Stat Critical Value {0.975} 1 .96 P-Value (Adjusted for Ties) 4.4474E-5 Conclusion with Alpha = 0.05 Reject HO. Conclude Sample 1 o Sample 2 P Value < alpha (0.05) 9.2.2 Two-Sample Hypothesis Testing for Data Sets with Nondetects 1. Click Two Sample Hypothesis ^- With NDs •oUCL 5.0 - [Zn-Cu-ND-data-chapter 9-u Statistical Tests I Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests ^ Goodness-of-Fit Tests K Single Sample Hypothesis » Two Sample Hypothesis t- Oneway AN OVA ^ OLS Regression Trend Analysis » 6 7 a, 9 Full (w/o NDs) > With NDs * 10 11 12 Gehan Tarone-Ware Wilcoxon-Mann-Whitney 2. Select the With NDs option. A list of available tests will appear (shown above). • To perform a Wilcoxon-Mann-Whitney test, click on Wilcoxon-Mann-Whitney from the drop-down menu list. • To perform a Gehan test, click on Gehan from the drop-down menu. • To perform a Tarone-Ware test, click on Tarone-Ware from the drop-down menu. 165 ------- 9.2.2.1 Two-Sample Wilcoxon-Mann-Whitney Test with Nondetects 1. Click Two Sample Hypothesis ^- With NDs ^- Wilcoxon-Mann-Whitney 2. Statistical Tests | Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests t Single Sample Hypothesis t Two Sample Hypothesis > Oneway AN OVA > OLS Regression Trend Analysis ^ 6 7 8 9 10 Full (w/o NDs) > With NDs > 11 12 Gehan Ta rone- Ware Wilcoxon-Mann-Whitney The Select Variables screen shown below will appear. Available Variables • Without Group Variable Name ID Count » Sample 1 « " Sample 2 « O Using Group Vs » Variable « Croup Variable Sample 1 Sample 2 Options Name ID Site 1 Count 10 Name ID Background 0 liable Name ID OK Count 10 Count v Caned Select variable(s) from the Select Variables screen. Without Group Variable: This option is used when the data values of the variable (e.g., TCDD 2,3,7,8) for the site and the background are given in separate columns. With Group Variable: This option is used when data values of the variable (TCDD 2, 3, 7, 8) are given in the same column. The values are separated into different samples (groups) by the values of an associated Group Variable. When using this option, the user should select an appropriate variable representing groups such as AOC1, AOC2. AOC3 etc. When the Options button is clicked, the following window will be shown. 166 ------- a Select Wilcoxon-Mann-Whitne... Select Null Hypothesis Form '.._) Sample 1 <= Sample 2 (Form 1) » Sample 1 >= Sample 2 (Form 2) O Sample 1 = Sample 2 (Two Sided) Select Confidence Coefficient O 99.9% O 99-5* O 99* O 97.5% * 95% O 90% Cancel o Choose the Confidence Coefficient. The default choice is 95%. o Select the form of Null Hypothesis. The default is Sample 1 <= Sample 2 (Form 1). o Click on OK button to continue or on Cancel button to cancel the selected options. • Click on OK to continue or on Cancel to cancel the Sample 1 vs. Sample 2 comparison. Example 9-5. Consider a two sample data set with nondetects and multiple detection limits. Since the data sets have more than one detection limit, the WMW test is not recommended for this data set. However, sometimes, the users tend to use the WMW test on data sets with multiple detection limits. The WMW test results are summarized as follows. 167 ------- Output for Two-Sample Wilcoxon-Mann-Whitney Test (with Nondetects) Date/Time of Computation 3/1 8/201 3 6:43:04 PM From File WMW-NDs-Chapter&user_ajds Full Precision OFF Confidence Coefficient 95"; Selected Null Hypothesis Sample 1 Mean/Median s= Sample 2 Mean/Median (Form 2) Alternative Hypothesis Sample 1 Mean/Median : Sample 2 Mean/Median Sample 1 Data: Site Sample 2 Data: Background Raw Statistics Sample 1 Number of Valid Data 11 Number of Non-Detects Number of Detect Data Minimum Non-Detect Maximum Non-Detect Percent Non-detects Minimum Detect Maximum Detect Mean of Detects Median of Detects SD of Detects 3 8 4 11 27.27% 2 43 27 29.5 13.71 Sample 2 11 3 4 9 27.27% 1 27 15.5 16.5 9.196 WMW test is meant fora Single Detection Limit Case of Gehan or T-W test is suggested when multiple detection limits are pres All observations <- 1 1 (Max DL) are ranked the same WMW test is meant for a Single Detection Limit Case Use of Gehan or T-W test is suggested when multiple detection limits are present All observations <= 1 1 (Max DL) are ranked the same Wilco.on-Mar.n-Whitney (WMW) Test HO Mean/Median of Sample 1 >= Mean/Median of Sample 2 Sample 1 Rank Sum W-Stat 144.5 WMW U-Stat 78.5 Mean (U) 60.5 SD(U)-Aijties 15.22 WMW U-Stat Critical Value (0.05) 35 Standardized WMW U-Stat 1191 Approximate P-Value O.SS3 Conclusion with Alpha - 0.05 Do Not Reject HO. Conclude Sample 1 >= Sample 2 Notes: In the WMW test, all observations below the largest detection limit are considered as NDs (potentially including some detected values) and hence they all receive the same average rank. This action tends to reduce the associated power of the WMW test considerably. This in turn may lead to an incorrect conclusion. 9.2.2.2 Two-Sample Gehan Test for Data Sets with Nondetects 1. Click Two Sample Hypothesis ^- With NDs ^- Gehan Statistical Tests Upper LimitE/BTVs UCLs/EPCs Windows Help Outlier Tests > G o o d n ess- of - Fit Tests ^ Single Sample Hypothesis t Two Sample Hypothesis » Oneway AN OVA ^ OLS Regression Trend Analysis ^ 6 7 8 9 Full (w/o NDs) t With NDs > 10 11 12 Gehan Ta rone- Ware Wi 1 c oxo n - M a n n - Wh itn ey 168 ------- 2. The Select Variables screen will appear. Available Variables : Without Group Variable Count 118 Sample 1 Sample 2 Name ID Counl • Using Group Variable Variable Zn ID Counl Group Variable Zone (Counl-118) Sample 1 alluvial fan Sample 2 Select variable(s) from the Select Variables screen. Without Group Variable: This option is used when the data values of the variable (Zinc) for the two data sets are given in separate columns. With Group Variable: This option is used when data values of the variable (Zinc) for the two data sets are given in the same column. The values are separated into different samples (groups) by the values of an associated Group Variable. When using this option, the user should select a group variable representing groups/populations such as Zone 1, Zone2, ZoneS, etc. When the Options button is clicked, the following window will be shown. Select Gehan Options Select Null Hypothesis Form O Sample 1 <= Sample 2 (Form 1) • Sample 1 >= Sample 2 (Form 2) O Sample 1 = Sample 2 (Two Sided) Select Confidence Coefficient O 99-9* O 99 5% 99% O 97-5X ® 95% O 90% OK 169 ------- o Choose the Confidence Coefficient. The default choice is 95%. o Select the form of Null Hypothesis. The default is Sample 1 <= Sample 2 (Form 1). o Click on OK button to continue or on Cancel button to cancel selected options. • Click on the OK button to continue or on the Cancel button to cancel the Sample 1 vs. Sample 2 Comparison. Example 9-6a. Consider the copper and zinc data set collected from two zones: Alluvial Fan and Basin Trough discussed in the literature (Helsel 2012). This data set is used here to illustrate the Gehan two- sample test. The output sheet generated by ProUCL 5.0 (similar sheet is generated using ProUCL 5.1, therefore the following output is not replaced) follows. Output for Two-Sample Gehan Test (with Nondetects) Confidence Coefficient 95% Selected Null hypothesis Sample 1 Mean/Median >= Sample 2 Mean/Median (Form 2} Alternative Hypothesis Sample 1 Mean/Median : Sample 2 Mean/Median Sample 1 Data: Zn(alluvial fan) Sample 2 Data: Zn(basin trough) Raw Statistics Sample 1 Sample 2 67 1 16 51 3 10 Number of Valid Data Number of Missing Observations Number of Non-Detects Number of Detect Data Minimum Non-Detect Maximum Non-Detect Percent Non-detects 23.88% Minimum Detect 5 Maximum Detect Mean of Detects Median of Detects SD of Detects EC 0 4 46 3 10 8.00% 3 620 3D 27.8B 23.13 11 2D S5.D2 19.03 Sample 1 vs Sample 2 Gehan Test HO: Mean of Sample 1 >= Mean of background Gehan 2 Test Value -3.037 Critical z (0.05) -1.645 P-Value 0.0012 Conclusion with Alpha = 0.05 Reject HO. Conclude Sample 1 < Sample 2 170 ------- 9.2.2.3 Two-Sample Tarone-Ware Test for Data Sets with Nondetects The two-sample Tarone-Ware (T-W) test (1978) for data sets with NDs is new in ProUCL 5.0. 1. Click Two Sample Hypothesis Testing ^ Two Sample ^ With NDs ^ Tarone-Ware Statistical Tests Outlier Test Goodness-c Single Samf Upper Limits/BTVs UCLs/EPCs Windows Help : K f-Fit Tests > le Hypothesis » Two Sample Hypothesis ^ Oneway AN OVA > OLS Regression Trend Analysis > 6 7 8 9 10 11 12 Full (w/o NDs] > With NDs »> Gehan Tarone-Ware Wikoxon- Mann-Whitney The Select Variables screen will appear. Available Variables Without Group Variable Name Cu ID 0 Counl 118 Sample 1 Sample 2 ® Using Group Variable Variable Name ID Counl Zn 1 118 Group Variable Zone (Count-118) Sample 1 alluvial fan Sample 2 basin trough Select variable(s) from the Select Variables screen. Without Group Variable: This option is used when the data values of the variable (Cu) for the two data sets are given in separate columns. With Group Variable: This option is used when data values of the variable (Cu) for the two data sets are given in the same column. The values are separated into different samples (groups) by the values of an associated Group Variable. When using this 171 ------- option, the user should select a group variable/ID by clicking the arrow next to the Group Variable option for a drop-down list of available variables. The user selects an appropriate group variable representing the groups to be tested. When the Options button is clicked, the following window will be shown. Select Tarone-Ware Options Select Null Hypothesis Form * Sample 1 <= Sample 2 (Form 1} O Sample 1 >= Sample 2 (Form 2) i.J1 Sample 1 = Sample 2 [Two Sided) Select Confidence Coefficient O 99.9% O 99.5% O 99% O 97-5* '* 95% O 90% o Choose the Confidence Coefficient. The default choice is 95%. o Select the form of Null Hypothesis. The default is Sample 1 <= Sample 2 (Form 1). o Click on OK button to continue or on Cancel button to cancel selected options. Click on the OK button to continue or on the Cancel button to cancel the Sample 1 vs. Sample 2 Comparison. 172 ------- Example 9-6b (continued). Consider the copper and zinc data set used earlier. The data set is used here to illustrate the T-W two-sample test. The output sheet generated by ProUCL 5.0/ProUCL 5.1 is described as follows. Output for Two-Sample Tarone-Ware Test (with Nondetects) Confidence Coefficient 95% Selected Null Hypothesis Sample 1 Mean/Median <= Sample 2 Mean/Median (Form 1} Memative Hypothesis Sample 1 Mean/Median > Sample 2 Mean/Median Sample 1 Data: Zn(alluvial fan) Sample 2 Data: Zn(basin trough) Raw Statistics Sample 1 Sample 2 Number of Valid Data 67 50 Number of Missing Observations 1 0 Number of Non-Detects 16 4 Number of Detects 51 46 Minimum Non-Detect 3 3 Maximum Non-Detect 10 1fl Percent Non-detects 23.88% 8.00% Minimum Detect 5 3 Maximum Detect 620 M Mean of Detects 27,88 23.13 Median of Detects 11 20 SD of Detects 85.02 19.03 Sample 1 vs Sample 2 Tarone-Ware Test HO: Mean/Median of Sample 1 <= Mean/Median of Sample 2 TW Statistic -2.113 TW Critical Value (0.05) 1.645 P-Value 0.383 Conclusion with Alpha = 0.05 Do Not Reject HO, Conclude Sample 1 <= Sample 2 P-Value >= alpha (0.05) 173 ------- 174 ------- Chapter 10 Computing Upper Limits to Estimate Background Threshold Values Based Upon Full Uncensored Data Sets and Left- Censored Data Sets with Nondetects This chapter illustrates the computations of parametric and nonparametric statistics and upper limits that can be used as estimates of BTVs and other not-to-exceed values. The BTV estimation methods are available for data sets with and without ND observations. Technical details about the computation of the various limits can be found in the associated ProUCL 5.1 Technical Guide. For each selected variable, this option computes various upper limits such as UPLs, UTLs, USLs and upper percentiles to estimate the BTVs that are used in site versus background evaluations. Two choices are available to compute background statistics for data sets: • Full (w/o NDs) - computes background statistics for uncensored full data sets without any ND observation. • With NDs - computes background statistics for data sets consisting of detected as well as nondetected observations with multiple detection limits. The user specifies the confidence coefficient (probability) associated with each interval estimate. ProUCL accepts a CC value in the interval [0.5, 1), 0.5 inclusive. The default choice is 0.95. For data sets with and without NDs, ProUCL 5.0/ProUCL 5.1 can compute the following upper limits to estimate BTVs: • Parametric and nonparametric upper percentiles. • Parametric and nonparametric UPLs for a single observation, future or next k (> 1) observations, mean of next k observations. Here future k, or next k observations may represent k observations from another population (e.g., site) different from the sampled (background) population. • Parametric and nonparametric UTLs. • Parametric and nonparametric USLs. Note on Computing Lower Limits: In many environmental applications (e.g., groundwater monitoring), one needs to compute lower limits including: lower prediction limits (LPLs), lower tolerance limits (LTLs), or lower simultaneous limit (LSLs). At present, ProUCL does not directly compute a LPL, LTL, or a LSL. It should be noted that for data sets with and without nondetects, ProUCL outputs several intermediate results and critical values (e.g., khat, nuhat, K, d2max) needed to compute the interval estimates and lower limits. For data sets with and without NDs, except for the bootstrap methods, the same critical value (e.g., normal z value, Chebyshev critical value, or t-critical value) can be used to compute a parametric LPL, LSL, or a LTL (for samples of size >30 to be able to use Natrella's approximation in LTL) as used in the computation of a UPL, USL, or a UTL (for samples of size >30). Specifically, to compute a LPL, LSL, and LTL (n>30) the '+' sign used in the computation of the 175 ------- corresponding UPL, USL, and UTL (n>30) needs to be replaced by the '-' sign in the equations used to compute UPL, USL, and UTL (n>30). For specific details, the user may want to consult a statistician. For data sets without ND observations, the user may want to use the Scout 2008 software package (EPA 2009c) to compute the various parametric and nonparametric LPLs, LTLs (all sample sizes), and LSLs. 10.1 Background Statistics for Full Data Sets without Nondetects 1. Click Upper Limits/BTVs ^ Full (w/o NDs) Statistical Tests 3 Upper Limits/BTVs UCLs/EPCs Windows Help Full (w/o NDs) > With NDs > Normal Gamma Lognormal Non-Parametric All 10 11 12 13 2. Select Full (w/o NDs) To compute background statistics assuming the normal distribution, click on Normal from the drop-down menu list. To compute background statistics assuming the gamma distribution, click on Gamma from the drop-down menu list. To compute background statistics assuming the lognormal distribution, click on Lognormal from the drop-down menu list. To compute background statistics using distribution-free nonparametric methods, click on Non-Parametric from the drop-down menu list. To compute and see all background statistics available in ProUCL, click on the All option from the drop-down menu list. ProUCL will display data distribution, all parametric and nonparametric background statistics in an Excel type spreadsheet. The user may use this output sheet to select the most appropriate statistic to estimate a BTV. 176 ------- 10.1.1 Normal or Lognormal Distribution 1. Click Upper Limits/BTVs ^- Full (w/o NDs) ^- Normal or Lognormal ProUCL 5.0 - FCE-NDs-Blanks-data-BTVs-UCL-chaps10.xls] Upper Limits/BTVs I UCLs/EPCs Windows Help 2. Full (w/o NDs) > With NDs > Normal Gamma Lognormal Non-Parametric All 1D 11 12 13 The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • To compute BTV estimates by a group variable, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of available variables and select an appropriate group variable. When the Option button is clicked, the following window will be shown. Confidence Level Coverage Different or Future K Observations 1 I OK I Caned 177 ------- o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Coverage coefficient (for a percentile) needed to compute UTLs. Coverage represents a number in the interval (0.0, 1). The default choice is 0.95. Remember, a UTL is an upper confidence limit (e.g., with confidence level = 0.95) for a 95% (e.g., with coverage = 0.95) percentile. o Specify the Different or Future K Observations. The default choice is 1. It is noted that when K = 1, the resulting interval will be a UPL for a single future observation. In the example shown above, a value of K = 1 has been used. o Click on OK button to continue or on Cancel button to cancel this option. • Click on OK to continue or on Cancel button to cancel the Upper Limits/BTVs options. Example 10-la. Consider the real data set described in Example 1-1 of Chapter 1 collected from a Superfund site. Aluminum concentrations follow a normal distribution and manganese concentrations follow a lognormal distribution. The normal and lognormal distribution based estimates of BTVs are summarized in the following two tables. Aluminum - Output Screen for BTV Estimates Based upon a Normal Distribution (Full - Uncensored Data Set) User Selected Options Date/Time of Computation 3/18/2013 3:2S:21 PM From File SuperFundjds Full Precision OFF Confidence Coefficient 95'; Coverage 95% New or Future K Observations 1 General Satisfies Total Number of Observations 24 Minimum 171(3 Second Largest 15400 Maximum 1620-3 Mean 7789 Coefficient of Variation 0.547 Mean of logged Data 8.798 Number of Distinct Observations 24 First Quartie 405S Median 7010 Third Quartie 10475 SO 4264 Skewness 0.542 SD of logged Data 0.61 Critical Values for Background Threshold Values (BTVs) Tolerance Factor K (For UTL) 2.309 d2max for USL) 2.644 Shapiro Wilk Test Statistic 5'-, Shapiro Wilk Critical Value Ulliefors Test Statistic 5% Ulliefors Critical Value Normal GOF Test 0.939 0.916 0.109 0.1S1 Shapiro Wilk GOF Test Data appear Norma! at 5% Significance Level Lillief ors GOF Test Data appear Normal at 5% Significance Level Data appear Normal at 5% Significance Level Background Statistics Assuming Normal Distribution 95% UTL with 95% Coverage 176.3.5 95% UPL}) 15248 95% USL 19063 90% Percentile (i) 13254 95% Percentile h] 14803 99% Percentile fz) 17708 178 ------- Manganese -Output Screen for BTV Estimates Based upon a Lognormal Distribution (Full-Uncensored Data Set) Lognormal Background Statistics for Uncensored Full Data Sets User Selected Options From Rle SuperFundjds Full Precision Confidence Coefficient OFF 95% Coverage 95?. New or Future K Observations 1 Number of Bootstrap Operations 2000 Manganese General Statistics Total Number of Observations 24 Minimum 3.6 Second Largest 440 Maximum 530 Mean 113.8 Coefficient of Variation 1181 Mean of logged Data 4.192 Number of Distinct Observations 23 First Quartile 29 3 Median 71.25 Thiid Quartile SD Skewness SD of logged Data 122.5 134.5 2.17 1.084 Critical Values for Background Threshold Values (BTVs) Tolerance Factor K {For UTL) 2.309 d2max for USD 2.644 Lognormal GOF Test Shapiro Wilk Test Statistic 0.972 Shapiro Wilk Lognormal GOF Test 5% Shapiro Wilk Critical Value 0.916 Data appear Lognomial at 5'., Significance Level Ulliefors Test Statistic 012 Ljllietors Loyr.orm.al GOF Test 5% Ljlliefors Critical Value 0.131 Data appear Lognormal at 5% Significance Leve! Data appear Lognormal at 5X Significance Level Background Statistics assuming Lognormal Distribution 95%UTLwth 95%Coverage 808.1 95%UPL«) 440.6 95'iUSL 1162 90% Percentile (z) 265.4 95% Percentile (z) 393.5 99% Percentile (z) 823.5 10.1.2 Gamma Distribution 1. Click Upper Limits/BTVs ^ Full (w/o NDs) ^ Gamma Upper Limits/BTVs UCLs/EPCs Windows Help Full (w/o NDs) o With NDs i> Normal Gamma Lognormal Non-Parametric All 10 11 12 13 2. The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. 179 ------- Available Variables Selected Variables Name ID Aluminum 0 Areenic 1 Chromium 2 Iran 3 Lead 4 Thallium 6 Vanadium 7 Benzo[a)pyrene S Naphthalene 9 Benzo{a)pyrene ... 10 <^^^H > I » I « Name ID Manganese 5 <^^^^^H > Select Group Column (Optional) I I | Option. OK Cancel If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of variables, and select a proper group variable. When the Option button is clicked, the following window will be shown. Enter BTV level Options Confidence Level Coverage 0.95 095 Different or Future K Observations 1 Number of Bootstrap Operations 2000 OK Cancel o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Coverage level; a number in interval (0.0, 1). Default choice is 0.95. o Specify the Future K. The default choice is 1. o Specify the Number of Bootstrap Operations. The default choice is 2000. o Click on OK button to continue or on Cancel button to cancel the option. Click on OK to continue or on Cancel button to cancel the Upper Limits/BTVs options. 180 ------- Example 10-lb (continued). Manganese concentrations also follow a gamma distribution. The gamma distribution based BTV estimates are summarized in the following table generated by ProUCL 5.0. The Gamma GOF test is shown in the following figure. Gamma Q-Q Plot for Manganese Theoretical Quantiles of Gamma Distribution Mn.-in- MJ it,*HI k slai - 0.9531 Ihelaitat -119.4215 Slope = 1.1632 Intercept -1&5912 CnirHc)linn,R-0%l') ."nder-ton IjailimT'Sf Test Statistic = 0.575 CiihcalValue(0.05) = 0.771 Data appear Gamma D»tribut Gamma GOF Test for Manganese Data Set Manganese - Output Screen for BTV Estimates Based Upon a Gamma Distribution (Full-Uncensored Data Set) Manganese General Statistics Total Number of Observations Minimum 24 3.6 Second Largest 440 Maximum 530 Mean Number of Distinct Observations first Quartile Median 23 Coefficient of Variation Mean of logged Data 113.3 1.181 4192 29.3 71.25 Third Quartile 122.5 SD 134.5 Skewness SD of logged Data 2.17 Critical Values for Background Threshold Values (BTVs) Tolerance Factor K (For UTL) 2.309 d2max for USD 2.644 Gamma GOF Test A-D Test Statistic 0.575 Anderson-Darling Gamma GOF Test 5% A-D Critical Value D.771 Detected data appear Gamma Distributed at 5S Signrficance Level K-S Test Statistic 0168 Kclmogrov-Smirnoff Gamma GOF Test 5% K-S Critical Value 0.183 Detected data appear Gamma Distributed at 5% Signrficance Level Detected data appear Gamma Distributed at 5*4 Significance Level khat(MLE) Theta hat (MILE) nuhat(MLE) MLE Mean tolas corrected) Gamma Statistics 1.05B 107.6 50.76 113.8 k star bias corrected MLE) 0.953 Theta star (bias corrected MLE) 1194 nu star (bias corrected) 45.75 M LE Sd (bias corrected) 116.6 Background Statistics Assuming 95% Wilson Hiferty (WH) Approx. Gamma UPL 353.6 957. Hawkins Wrxley (HW) Approx. Gamma UPL 364.2 95% WH Approx. Gamma UTL with 95% Coverage 503.8 95% HW Approx. Gamma UTL with 95% Coverage 540.3 95XWHUSL 611.3 90% Percentile 95% Percentile 99% Percentile 265.2 346.8 537 95% HW USL 6723 181 ------- The mean manganese concentration is 113.8 with sd = 134.5, and the maximum value = 530. The UTL based upon a lognormal distribution is 808.1 which is significantly higher than the largest value of 530. It is noted that the sd of the log-transformed data is 1.084. By comparing BTV estimates computed using lognormal and gamma distributions, it is noted that the lognormal distribution based upper limits, UTL and UPL, are significantly higher than those based upon a gamma distribution confirming the statements made earlier that the use of a lognormal distribution tends to yield inflated values of the upper limits used to estimate environmental parameters (e.g., BTVs, EPCs). These upper limits are summarized as follows. UTL95-95 UPL95 Lognormal 808.1 440.6 Gamma (WH) 504 353.6 Gamma (HW) 540.3 364.2 Mean = 113.8, Max value = 530. 10.1.3 Nonparametric Methods 1. Click Upper Limits/BTVs ^ Full (w/o NDs) ^ Non-Parametric Upper Limits/BTVs UCLs/EPCs Windows Help Full (w/o NDs) > With NDs > 7 8 9 10 11 12 13 Normal Gamma Lognormal All 2. The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of variables, and select a proper group variable. • When the Option button is clicked, the following window will be shown. Enter BTV Level Options Confidence Level 0.95 Coverage 0.95 Number of Bootstrap Operations 2000 OK Cancel 182 ------- o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Coverage level; a number in the interval (0.0, 1). Default choice is 0.95. o Specify the Number of Bootstrap Operations. The default choice is 2000. o Click on the OK button to continue or on the Cancel button to cancel the option. • Click OK button to continue or Cancel button to cancel the Upper Limits/BTVs options. Example 10-2. Lead concentrations data set used in Example 1-1 does not follow a discernible distribution. Nonparametric BTV estimates are summarized as follows. ProUCL 5.1 also outputs the sample size needed to compute a nonparametric UTL needed to achieve the specified CC. Lead - Output Screen for Nonparametric BTVs Estimates (Full-Uncensored Data Set) General Statistics Total Number of Observations 24 Minimum 4.9 Second Largest 98.5 Maximum 109 Mean 22.49 Coefficient of Variation 1.193 Mean of logged Data 2.743 Number of Distinct Observations 18 First Quartile 10.43 Median 14 Third Quartile 19.25 SD 26.83 Skewness 2.665 SD of logged Data 0.771 d2max (for USL) 2.644 Critical Values for Background Threshold Values (BTVs) T olerance Factor K (For U T L] 2.309 Nonparametric Distribution Free Background Statistics Data do not follow a Discernible Distribution [005} Nonparametric Upper Limits for Background Threshold Values Order of Statistic, r 24 35% UTL with 95% Coverage 109 Approximate f 1.263 Approximate Actual Confidence Coefficient achieved by UTL 0.708 Approximate Sample Size needed to achieve specified CC 59 95% Percentile Bootstrap UTL with 35% Coverage 109 95% BCA Bootstrap UTL with 35% Coverage 109 95% UPL 106.4 30% Percentile 44.81 30% Chebyshev UPL 104.6 35% Percentile 91.72 35% Chebjishev UPL 141.8 33% Percentile 106.6 To compute nonparametric upper limits providing the specified coverage (e.g., 0.95), sizes of the data sets should be fairly large (e.g., > 59). For details, consult the associated ProUCL Technical Guide. In this example the sample size is only 24, and the confidence coefficient (CC) achieved by the nonparametric, UTL is only 0.71 which is significantly lower than the desired CC of 0.95. 183 ------- 10.1.4 All Statistics Option 1. Click Upper Limits/BTVs ^ Full (w/o NDs) ^ All ICL5.0 - [TCE-NDs-Blanks-data-BTVs-UCL-chaps1u.xl Upper Limits/BTVs I UCLs/EPCs Windows Help Full [w/o NDs) With NDs 2. Normal Gamma Lognormal Non-Parametric All 10 11 12 13 The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of variables, and select a proper group variable. • When the Option button is clicked, the following window will be shown. Enter BTV Level Options Confidence Level Coverage Number of Bootstrap Operations 2000 OK Cancel o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Coverage level; a number in the interval (0.0, 1). Default is 0.9. o Specify the Future K. The default choice is 1. o Specify the Number of Bootstrap Operations. The default choice is 2000. o Click on OK button to continue or on Cancel button to cancel the option. • Click on OK to continue or on Cancel button to cancel the Upper Limits/BTVs options. 184 ------- Example 10-lc (continued). The various BTV estimates based upon the manganese concentrations computed using the All option of ProUCL are summarized as follows. The All option computes and displays all available parametric and nonparametric BTV estimates. This option also informs the user about the distribution(s) of the data set. This option is specifically useful when one has to process many analytes (variables) without any knowledge about their probability distributions. Manganese Manganese - Output Screen for All BTVs Estimates (Full-Uncensored Data Set) General Statistics Total Number of Observations 24 Minimum 8.6 Second Largest 440 Maximum 53S Mean 113.8 Coefficient of Variation 1.181 Mean of logged Data 4.192 Number of Distinct Observations 23 First Quartile 29.3 Median 71.25 Third Quartile 122.5 SD 134.5 Skewness 2.17 SD of logged Data 1.084 Critical Values for Background Threshold Values (BTVs) Tolerance Factor K {For UTL) 2.309 d2max for USL) 2.644 Shapiro Wilk Test Statistic ; Shapiro Wilk Crtical Value Ljlliefors Test Statistic 5'i Ulliefors Crtical Value Normal GOF Test 0.637 0.916 0.233 0.181 Shapiro Wilk GOF Test Data Not Normal at 5% Significance Level Ulliefors GOF Test Data Not Normal at 5% Significance Level Data Not Normal at 5% Significance Level Background Statistics Assuming Normal Distribution 95% UTL with 35% Coverage 424.3 30% Percentile {z} 35% Percentile (z) 33% Percentile (z) 286.1 335 426.6 95%UPLJ) 349 95% USL 469.3 Gamma GOF Test A-D Test Statistic 0.575 Anderson-Darting Gamma GOF Test 5% A-D Critical Value 0.771 Detected data appear Gamma Distributed at 5% Significance Level K-S Test Statistic 0168 Kolmogrov Smirnoff Gamma GOF Test 5% K-S Critical Value 0.183 Detected data appear Gamma Distributed at 5% Significance Level Detected data appear Gamma Distributed at 5% Significance Level Gamma Statistics khat(MLE) 1.058 Theta hat {MLEj 107.6 nu hat (MLE) 50.76 P>1LE Mean ibias corrected) 113.8 k star (bias corrected MLE) 0.953 Theta star (bias corrected MLE} 119.4 nu star (bias corrected) 45.75 MLE Sd (bias corrected) 116.6 Background Statistics Assuming Gamma Distribution 35% Wilson HiKerty (W H) Approx. Gamma LI P L 353.« 95% Hawkins Wixley (HW) .Approx. Gamma UPL 364.2 95% WH Approx. Gamma UTL with 95% Coverage 503.8 35% HW Approx. Gamma UTL with 95% Coverage 54D.3 35%WHUSL 611.3 90% Percentile 2S5.2 95% Percentile 346.8 93% Percentile 537 35% HW USL 672.3 185 ------- Manganese - Output Screen for All BTVs Estimates - Continued (Full-Uncensored Data Set) Lognormal GOF Test Shapiro Wilk Test Statistic 0 972 Shapiro Wilk Lognormal GOF Test 5% Shapiro Wilk Critical Value 0.916 Data appear Lognomial at 5', Significance Level Llefors Test Statistic D 12 Lilliefors Lognoimal GOF Test 5% Ulliefors Cntical Value 0.181 Data appear Lognormal at 5% Significance Level Data appear Lognormal at 5% Significance Level Background Statistics assuming Lognormal Distribution 95% UTL with 95% Coverage 80B.1 95*4 UPLJ) 440.6 95%USL 1162 Nonparametric Distribution Free Background Statistics Data appear Gamma Distributed at 5% Significance Level 90% Percentile (z) 265.4 95% Percentile (z) 393.5 99% Percentile (z) 823.5 Nonparametric Upper Limits for Background Threshold Values Order of Statistic, r 24 Approximate f 1.263 95% Percentile Bootstrap UTL with 95% Coverage 530 95%UPL 507.5 mChebyshevUPL 525.5 95%ChebyshevlJPL 712 95% USL 530 95% UTL with 95% Coverage Confidence Coefficient (CC) achieved by UTL 95% BCA Bootstrap UTL with 95% Coverage 90% Percentile 95% Percentile 99% Percentile 530 0708 530 280 425 5C5.3 10.2 Background Statistics with NDs 1. Click Upper Limits/BTVs ^ With NDs Upper Limits/BTVs UCLs/EPCs Windows Help Full (w/o NDs) t With NDs t 7 8 9 Normal Gamma Lognormal Non-Parametric All 10 11 12 13 2. Select the With NDs option. To compute background statistics assuming the normal distribution, click on Normal from the drop-down menu list. To compute background statistics assuming the gamma distribution, click on Gamma from the drop-down menu list. 186 ------- • To compute background statistics assuming the lognormal distribution, click on Lognormal from the drop-down menu list. • To compute background statistics using distribution-free methods, click on Non- Parametric from the drop-down menu list. • To compute all available background statistics in ProUCL, click on the All option from the drop-down menu list. 10.2.1 Normal or Lognormal Distribution 1. Click Upper Limits/BTVs ^ With NDs ^ Normal or Lognormal ProUCL 5.0 - [TCE-NDs-Blanks-data-BTVs-UCL-chaps10.x Upper Limits/BTVs | UCLs/EPCs Windows Help Full (w/o NDs) * With NDs 8 Normal Gamma Lognormal Non-Parametric All 10 11 12 13 2. The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of variables, and select a proper group variable. • When the option button is clicked, the following window will be shown. "=! Enter BTV level Options Confidence Level Coverage Different or Future K Observations Number of Bootstrap Operations 2000 OK Caned Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. 187 ------- o Specify the Coverage level; a number in the interval (0.0, 1). Default choice is 0.95. o Specify the Future K. The default choice is 1. o Specify the Number of Bootstrap Operations. The default choice is 2000. o Click on the OK button to continue or on the Cancel button to cancel the option. • Click on OK to continue or on Cancel button to cancel the Upper limits/BTVs options. Example 10-3a. Consider a small real TCE data set of size «=12 consisting of 4 ND observations. The detected data set of size 8 follows a normal as well as a lognormal distribution. The BTV estimates using the LROS method, normal and lognormal distribution on KM estimates, and nonparametric Chebyshev inequality and bootstrap methods on KM estimates are summarized in the following two tables. It is noted that upper limits including UTL95-95 and UPL95 based upon the robust LROS method yield much higher values than the other methods including KM estimates in normal and lognormal equations to compute the upper limits. It is noted that the detected data also follows a gamma distribution. The gamma distribution (of detected data) based BTV estimates are described in the next section. TCE - Output Screen for BTV Estimates Computed Using Normal Distribution of Detected Data (Left-Censored Data Set with NDs) I User Selected Options From File TCE-NDs-Hanks-data-BTVs-UCL-chapsjds Full Precision OFF Confidence Coefficient 95% Coverage 95% Different or Future K Observations 2 TCE Total Number of Observations Number of Missing Observations Number of Detects Number of Distinct Detects Minimum Detect Maximum Detect Variance Detected Mean Detected Mean of Detected Logged Data General Satisfies U 0.75 9.29 9.732 2.941 0.634 Number of Distinct Observations 9 Number of Non-Detects 4 Number of Distinct Non-Defects 1 Minimum Non-Detect G.68 Maximum Non-Detect G.S8 Percent Non-Detects 33,33% SD Detected 3.12 SD of Detected Logged Data 0.978 Critical Values for Background Threshold Values (BTVs) Tolerance Factor K (For UTLj 2.736 d2maxforUSL) 2.285 Normal GOF Test on Detects Only Shapiro Wilk Test Statistic 0.765 Shapiro Wilk GOF Test 5% Shapiro Wilk Critical Value 0.818 Data Not Normal at 5% Significance Level Ulliefors Test Statistic 0.256 LHIiefors GOF Test 5% Uliefors Critical Value 0.313 Detected Data appear Normal at 5% Significance Level Detected Data appear Approximate Normal at 5% Significance Level 188 ------- TCE (continued) - Output Screen for BTV Estimates Computed Using Normal Distribution of Detected Data (Left-Censored Data Set with NDs) Kaplan Meier (KM) Background Satisfies Assuring Normal Distribution Mean 2.183 SD 2.61 9.323 95%KMUPLft} 7.CK7 S.167 95% KM UPLfor Mean of Next 2 Observations 5.768 14.03 90% KM Percentile (z) 5.533 S.481 99% KM Percentile {2} 8.26 95% UTL95% Coverage 95% KM UPLfor Next 2 Observations 95% KM Chebyshev UPL 95% KM Percentile (z) Output Screen for BTV Estimates Computed Using a Lognormal Distribution of Detected Data (Left-Censored Data Set with NDs) Lognormal Background Statistics for Data Sets with Non Detects User Selected Options From File Full Precision Confidence Coefficient Coverage Different or Future K Observations Number of Bootstrap Operations TCE-NDs-Hanks-data-BTVs-UCL-chapslOxIs OFF 95% 95*', 2 2000 TCE Total Number of Observations Number of Missing Observations Number of Detects Number of Distinct Detects Minimum Detect Maximum Detect Variance Detected Mean Detected Mean of Detected Logged Data General Statistics 12 2 0.75 9.29 9.732 2.941 0.634 Number of Distinct Observations 9 Number of Non-Defects 4 Number of Distinct Non-Detects 1 Minimum Non-Detect Q.68 Maximum Non-Detect G.GS Percent Non-Detects 33.33°; SD Detected 3.12 SD of Detected Logged Data 0.97S Critical Values for Background Threshold Values (BTVs) Tolerance Factor K (For UTL) 2.736 d2max for USL) 2.285 Lognormal GOF Test on Detected Observations Only Shapiro Wilk Test Statistic O.S65 Shapiro Wilk GOF Test 5% Shapiro Wilk Critical Value 0.81 S Detected Data appear Lognormal at 5'i Significance Level Lilliefors Test Statistic 0.258 Lilliefors GOF Test 5*% Uliefors Critical Value 0.313 Detected Data appear Lognormal at 5% Significance Level Detected Data appear Lognormal at 5% Significance Level 189 ------- Continued: Output Screen for BTV Estimates Computed Using a Lognormal Distribution of Detected Data (Left-Censored Data Set with NDs) Kaplan Meier (KM) Background Statistics Assuming Normal Distribution Mean 2.18S 95% UTL95% Coverage 9.329 95% KM UPLfor Next 2 Observations B.167 95% KM Chebyshev UPL 14.03 Background 95% KM Percentile (z) 6.4S1 95% KM USL 8.152 Lognormal ROS Statistics Assuming Mean in Original Scale 2.01 S SD in Original Scale 2.S3S 95% UTL.95% Coverage 50.54 95% Bootstrap (%) UTL95% Coverage 9.29 95 % UPLfor Next 2 Observations 25.78 90% Percentile (z) 5.606 99% Percentile (z) 27.2 Statistics using KM estimates on Logged SD 35% KM UPL ------- Set BTV level Options ^^^^^^^^^^^^^^^^^^^^^H Confidence Level Coverage Different or Future K Observations OK o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Coverage level; a number in the interval (0.0, 1). Default choice is 0.95. o Click on the OK button to continue or on the Cancel button to cancel option. • Click on OK to continue or on Cancel button to cancel the Upper Limits/BTVs options. Example 10-3b (continued). It is noted that the detected TCE data considered in Example 10-3 also follows a gamma distribution. The gamma distribution based upper limits are summarized as follows. TCE - Output Screen for BTV Estimates Computed Using Gamma Distribution of Detected Data (Left-Censored Data Set with NDs) TCE General Statistics Total Number of Observations 12 Number of Missing Observations 2 Number of Detects 8 Number of Distinct Detects 8 Minimum Detect 0.75 Maximum Detect 9.29 Variance Detected 9.732 Mean Detected 2.841 Mean of Detected Logged Data 0.634 Number of Distinct Observations 9 Number of Non-Delects 4 Number of Distinct Non-Detects 1 Minimum Non-Detect 0.68 Maximum Non-D elect 0.68 Percent N on-D etects 33.33% SD Detected 3.12 S D of D elected Logged D ata 0.978 Critical Values for B ackground T hreshold Values (BTVs) Tolerance Factor K (For UTL] 2.736 d2max (for USL) 2.285 G amma GOFTestsonD elected 0 bservations Or Jy A-D Test Statistic 0.624 Anderson-Darling GOF Test 5%A-D Critical Value 0.732 Detected data appear Gamma Distributed at 5% 5ignificance Level K-S Test Statistic 0.274 Kolmogomv SmirnovGOF 5XK-S Critical Value 0.3 Detected data appear Gamma Distributed at 5% Significance Level Detected data appear Gamma Distributed at 5% Significance Level 191 ------- TCE (continued) - Output Screen for BTV Estimates Computed Using Gamma Distribution of Detected Data (Left-Censored Data Set with NDs) Gamma Statistics on Detected Data Only khat(MLE) 1.265 Theta hat (MLE) 2.326 nu hat (MLE) 20.23 2.941 3.147 MLE Mean (bias corrected) MLE Sd (bias corrected) k star (bias corrected MLE) 0.874 Theta star (bias corrected MLE) 3.366 nu star (bias corrected) 13.98 95? Percentile of Chisquare (2k) 5.492 G amma ROSS tatistics using I mputed N on-Detects Minimum 0.01 Maximum 9.29 SD 2.377 k hat (MLE) 0.372 Theta hat (MLE) 5.274 nu hat (MLE) 8.938 MLE Mean (bias corrected) 1.964 95% Percentile of Chisquare (2k) 2.956 '35% Percentile 8.668 Mean Median CV k star (bias corrected MLE) Theta star (bias corrected MLE) nu star (bias corrected) MLE Sd (bias corrected) 90% Percentile 98% Percentile The following statistics are computed using Gamma ROS Statistics on Imputed Data Upper Limits using Wilson Hilferty (WH] and Hawkins Wixley (HW) Methods WH HW WH 95% Appro*. Gamma UTL with 95% Coverage 19.62 27.19 95% Appro*. Gamma UPL 9.793 35% Gamma USL 13.95 17.89 Estimates of Gamma Parameters using KM Estimates Mean (KM) 2.188 6.813 0.702 13.99 3.115 3.606 Variance (KM) k hat (KM) nu hat (KM) theta hat (KM) 80% gamma percentile (KM) 35% gamma percentile (KM) SD (KM) SE of Mean (KM) k star (KM) nu star (KM) theta star (KM) 30% gamma percentile (KM) 7.957 99% gamma percentile (KM) 1.964 0.845 1.465 0.335 5.865 8.037 3.394 5.709 16.26 HW 11.66 2.61 0.806 0.582 13.98 3.757 5.728 13.36 The following statistics are computed using gamma distribution and KM esbnates U pper Limits using Wilson H ilf eity (WH) and H awkins Wixley (HW) Methods WH HW WH HW 95% Appro*. Gamrna UTL with 95% Coverage 11.34 11.95 95%Approx. Gamma UPL 6.88 6.896 35% KM Garnma Percentile 5.955 5.902 85% Gamma USL 8.836 9.063 The detected data set does not follow a normal distribution based upon the S-W test, but follows a normal distribution based upon the Lilliefors test. Since the detected data set is of small size (=8), the normal GOF conclusion is suspect. The detected data follow a gamma distribution. There are several NDs reported with a low detection limit of 0.68, therefore, GROS method may yield infeasible negative imputed values. Therefore, the use of a gamma distribution on KM estimates is preferred for computing the BTV estimates. The gamma KM UTL95-95 (HW) =11.34, and gamma KM UTL95-95 (WH) = 11.95. Any one of these two limits can be used to estimate the BTV. 192 ------- 1'0.2.3 Nonparamethc Methods (with NDs) 1. Click Upper Limits/BTVs^ With NDs ^ Non-Parametric 2. >.0 - [TCE-NDs-Blanks-data-BTVs-UCL-chaps10.xls] Upper Li mits/BTVs UCLs/EPCs Windows Help Full (w/o NDs) > With NDs > 7 8 S 10 11 12 13 Normal Gamma Lognormal Non-Parametric All The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of variables, and select a proper group variable. • When the Option button is clicked, the following window will be shown. : BTV level Optic Confidence Level Coverage Different or Future K Observations OK Cancel o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Coverage level; a number in interval (0.0, 1). Default choice is 0.95. o Click on the OK button to continue or on the Cancel button to cancel the option. Click on OK to continue or on Cancel button to cancel the Upper Limit/BTVs option. 193 ------- Example 10-3c (continued). The nonparametric upper limits based the TCE data considered in Example 10-3 are summarized in the following table. TCE - Output Screen for Nonparametric BTV Estimates (Left-Censored Data Set with NDs) TCE General Statistics Total Number of Observations 12 Number of Missing Observations 2 Number of Detects 8 Number of Distinct Detects 8 Minimum Detect 0.75 Maximum Detect 9.29 Variance Detected 9.732 Mean Detected 2.941 Mean of Detected Logged Data 0.634 Number of Distinct Observations Number of Non-Detects 4 Number of Distinct Non-Detects 1 Minimum Non-Delect 0.68 Maximum Non-Detect 0.68 Percent Non-Detects 33.33% SD Detected 3.12 SD of Detected Logged Data 0.978 Critical Values for Background Threshold Values (BTVs) Tolerance Factor K (For UTL] 2.736 Nonparametric Distribution Free Background Statistics d2max(forUSL) 2.285 Data appear to follow a Discernible Distribution at 5% Signficance Level Kaplan Meier [KM] Background Statistics Assuming Normal Disbrfajtjon Mean 2.188 SD 2.61 35% UTL95S Coverage 9.329 95% KM UPL (t) 7.067 35% KM Chebjishev UPL 14.03 90% KM Percentile (2) 5.533 95S KM Percentile (z) 6.481 99% KM Percentile (:) 8.26 95%KMU8L 8.152 N onparametric Upper Limits for B TVs(no distinction made between detects and nondetects) Order of Statistic, r 12 95X UTL with95% Coverage 9.29 Approximate f 0.632 Approximate Actual Confidence Coefficient achieved by UTL 0.46 Approximate Sample Size needed to achieve specified CC 59 95% UPL 9.29 95KUSL 9.29 95% KM Chebyshev UPL 14.03 10.2.4 All Statistics Option 1. Click Upper Limits/BTVs ^ With NDs ^ All ProUCL 5.0 - [TCE-NDs-Blanks-data-BTVs-UCL-chaps1u.xls] Upper Limits/BTVs |_UCLs/EPCs Windows Help Full (w/o NDs) With NDs B Normal Gamma Lognormal Non-Parametric All 1C 11 12 13 2. The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. 194 ------- If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of variables, and select a proper group variable. When the Option button is clicked, the following window will be shown. Enter BTV level Options Confidence Level 0.95 I Coverage 0-95 Different or Future K Observations 1 Number of Bootstrap Operations 2000 I OK Cancel o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Coverage level; a number in the interval (0.0, 1). Default choice is 0.95. o Specify the Future K. The default choice is 1. o Click on the OK button to continue or on the Cancel button to cancel the option. • Click on OK to continue or on Cancel button to cancel the Upper Limits/BTVs option. Example 10-3d (continued). BTV estimates using the All option for the TCE data are summarized as follows. The detected data set is of small size («=8) and follows a gamma distribution. The gamma GOF Q-Q plot based upon detected data is shown in the following figure. The relevant statistics have been high-lighted in the output table provided after the gamma GOF Q-Q plot. Gamma Q-Q Plot (Statistics using Detected Data) for TCE lotalNumbclofDjta 14 Slope = 1.0' Intercept = -0.1201 Conelation.R = 0.3M3 Anderson-Daring Test Critical Value(Q.05] = 11732 Data appeal Gamma D iiliibuted • Best Fil Line Theoretical Quantiles of Gamma Distribution 195 ------- TCE - Output Screen for All BTV Estimates (Left-Censored Data Set with NDs) TCE General Statistics Total Number of Observations T 12 Number of Distinct Observations r 9 Number of Defects' 8 Number of Distinct Detects r 8 Minimum Detect' 0.75 Maximum Detectr 9.29 Variance Detected r 9.732 Number of Missing Observations f 2 Mean Detected ' 2.941 Mean of Detected Logged Data ' 0.634 Number of Non-Detects Number of Distinct Non-Detects r 1 Minimum Non-Detect' 0.6S Maximum Non-Detect r 0.68 Percent Non-Detects r 33.33°i SD Detected ' 3.12 SD of Detected Logged Data ' 0.978 Critical Values lor Background Threshold Values (BTVs) Tolerance Factor K (For UTL)r 2.736 d2rnaxfforUSL)'' 2.285 Normal GOF Test on Detects Only Shapiro Wilk Test Statistic " 0.765 Shapiro Wilk GOF Test 5% Shapiro Wilk Critical Value r 0.818 Data Not Normal at 5°. Significance Level Lilliefors Test Statistic r 0.256 Ljlliefors GOF Test 5% Ljlliefors Critical Value r 0.313 Detected Data appear Normal at 55; Significance Level Detected Data appear Approximate Normal at 5% Significance Level Kaplan Meier (KM) Background Statistics Assuming Normal Distribution Meanr 2.188 SD' 2.61 95% UTL35% CoverageT 9.329 955; KM UPL«r 7.067 95%KMCnebysnevUPLr 14.03 90% KM Percentile (z)r 5.533 955; KM Percentile (z)r 6.481 99% KM Percentile (z)r 8.26 95%KMUSLr S.I 52 ' ' Gamma GOF Tests on Detected Observations Only A-D Test Statistic ' 0.624 Anderson-Darling GOF Test 5% A-D Critical Value * 0.732 Detected data appear Gamma Distributed at 55; Significance Level K-S Test Statistic r 0.274 Kolmogrov Smirnoff GOF 5*; K-S Critical Value ' 0.3 Detected data appear Gamma Distributed at 55; Significance Level Detected data appear Gamma Distributed at 5% Significance Level Gamma Statistics on Detected Data Only khat{MLE)r 1.265 Thetahat{MLE)r 2.326 nuhat(MLE)r 20.23 MLE Mean (bias corrected)' 2.941 r k star (bias corrected MLE)' 0.874 Theta star Ibias corrected MLE)r 3.366 nu star (bias corrected)r 13.98 MLE Sd (bias corrected)r 3.147 95% Percentile of Chisquare (2k)T 5.492 196 ------- TCE (continued) - Output Screen for All BTV Estimates (Left-Censored Data Set with NDs) The following statistics are computed using Gamma ROS Statistics on Imputed Data Upper Limits using Wilson Hilferty (WH) and Hawkins Wixley (HW) Methods WH HW T 95% Appro*.Gamma UTLwith 95%Coverage'' 19.62 r 27.19 95% Approx. Gamma UPL r 95% Gamma USLr 13.95 T 17.89 r r WH 9.793 HW 11.66 The following statistics are computed using gamma distribution and KM estimates Upper Limits using Wilson Hrferty (WH) and Hawkins Wbdey (HW) Methods k hat (KM)' 0.702 nu hat (KM)' 16.86 WH HW r WH HW 95%Approx. Gamma UTL with 95% Coverage' 11.34 * 11.95 957. Approx. Gamma UPLr 6.SS ' 6.896 95% Gamma USL r S.S36 r 9.063 r r r Lognormal GOF Test on Detected Observations Only Shapiro Wilk Test Statistic ' O.S65 Shapiro Wilk GOF Test 5% Shapiro Wilk Critical Value r 0,818 Detected Data appear Lognoimal at 5% Significance Level Ulliefors Test Statistic ' 0.258 Lilliefors GOF Test 5% Lilliefors Critical Value r 0.313 Detected Data appear Lognormal at 5% Significance Level Detected Data appear Lognormal at 5% Significance Level Background Lognormal ROS Statistics Assuming Lognormal Distribution Using Imputed Non-Defects Mean in Original Scale r 2.018 SD in Original Scale ' 2.S38 95% UTL95% Coverage r 50.54 95% Bootstrap {%) UTL95% Coverage r 9.29 90%Percentile(z)r 5.606 99%Percentile(z)r 27.2 Mean in Log Scale r -0.214 SD in Log Scale T 1.512 95% BCAUTL95% Coverage r 9.29 95%UPL95% Coverage ' 15.25 KM SD of Logged Data r 0.888 95% KM UPL (Lognormal)' 7.06 95*/. KM Percentile Lognormal (z)' 5.7S4 95% KM USL (Lognormal)T 10.21 Nonparametric Llppper Limits for BTVs(no distinction made between detects and nondetects) Order of Statistic. r 12 95% UTL with 95% Coverage 9.29 .Approximate f D.632 95% UPL 9.29 Confidence Coefficient ICC) achieved by UTL 95% USL 0.46 9.29 Note: Even though the data set failed the Shapiro-Wilk test of normality, based upon Lilliefors test it was concluded that the data set follows a normal distribution. Therefore, instead of saying that the data set does not follow a normal distribution, ProUCL outputs that the data set follows an approximate normal distribution. In practice the two tests can lead to different conclusions, especially when the data set is of small size. In such instances, the user may want to select a distribution (if any) passing both of the GOF tests. It is also suggested that the user supplements test results with graphical displays to derive the final conclusion. As noted, detected data follow a gamma as well as a lognormal distribution. The various upper limits using Gamma ROS and Lognormal ROS methods and Gamma and Lognormal distribution on KM estimates are summarized as follows. 197 ------- Summary of Upper Limits Computed using Gamma and Lognormal Distribution of Detected Data Sample Size = 12, No. of NDs = 4, % NDs = 33.33, Max Detect = 9.29 Upper Limits Mean (KM) Mean (ROS) UPL95 (ROS) UTL95-95 (ROS) UPL95 (KM) UTL95-95 (KM) Gamma Distribution Result 2.188 1.964 9.79 19.62 6.88 11.34 Reference/ Method of Calculation - - WH- ProUCL(ROS) WH- ProUCL(ROS) WH - ProUCL (KM- Gamma) WH - ProUCL (KM- Gamma) Lognormal Distribution Result 0.29 2.018 13.63 50.54 7.06 15.25 Reference/ Method of Calculation Logged - Helsel(2012),EPA(2009)- LROS Helsel(2012),EPA(2009)- LROS KM-Lognormal EPA (2009) KM- Lognormal EPA(2009) Note: All computations have been performed using the ProUCL software. In the above table, methods proposed/described in the literature have been cited in the Reference Method of Calculation column. The statistics summarized above demonstrate the merits of using the gamma distribution based upper limits to estimate decision parameters (BTVs) of interest. These results summarized in the above tables suggest that the use of a gamma distribution cannot be dismissed just because it is easier to use a lognormal distribution to model skewed data sets as stated by some practitioners. 198 ------- 199 ------- Chapter 11 Computing Upper Confidence Limits (UCLs) of Mean Based Upon Full-Uncensored Data Sets and Left-Censored Data Sets with Nondetects Several parametric and nonparametric UCL methods for full-uncensored and left-censored data sets consisting of ND observations with multiple DLs are available in ProUCL 5.1. Methods such as the Kaplan-Meier (KM) and regression on order statistics (ROS) methods incorporated in ProUCL can handle multiple detection limits. For details regarding the goodness-of-fit tests and UCL computation methods available in ProUCL, consult the ProUCL Technical Guides, Singh, Singh, and Engelhardt (1997); Singh, Singh, and laci (2002); and Singh, Maichle, and Lee (2006). In ProUCL 5.0/ProUCL 5.1, two choices are available for computing UCL statistics: • Full (w/oNDs): Computes UCLs for full-uncensored data sets without any nondetects. • With NDs: Computes UCLs for data sets consisting of ND observations with multiple DLs or reporting limits (RLs). • For full data sets without NDs and also for data sets with NDs, the following options and choices are available to compute UCLs of the population mean. o The user specifies a confidence level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o The program computes several nonparametric UCLs using the CLT, adjusted CLT, Chebyshev inequality, jackknife, and bootstrap re-sampling methods. o For the bootstrap method, the user can select the number of bootstrap runs (re-samples). The default choice for the number of bootstrap runs is 2000. o The user is responsible for selecting an appropriate choice for the data distribution: normal, gamma, lognormal, or nonparametric. It is desirable that user determines data distribution using the Goodness-of-Fit test option prior to using the UCL option. The UCL output sheet also informs the user if data are normal, gamma, lognormal, or a non- discernible distribution. Program computes statistics depending on the user selection. o For data sets, which are not normal, one may try the gamma UCL next. The program will offer you advice if you chose the wrong UCL option. o For data sets, which are neither normal nor gamma, one may try the lognormal UCL. The program will offer you advice if you chose the wrong UCL option. 200 ------- o Data sets that are not normal, gamma, or lognormal are classified as distribution-free nonparametric data sets. The user may use nonparametric UCL option for such data sets. The program will offer you advice if you chose the wrong UCL option. o The program also provides the All option. By selecting this option, ProUCL outputs most of the relevant UCLs available in ProUCL. The program informs the user about the distribution of the underlying data set, and offers advice regarding the use of an appropriate UCL. For lognormal data sets, ProUCL can compute 90%, 95%, 97.5%, and 99% Land's statistic-based H-UCL of the mean. For all other methods, ProUCL can compute a UCL for any confidence coefficient (CC) in the interval (0.5, 1.0), 0.5 inclusive. If you have selected a distribution, then ProUCL will provide a recommended UCL method for 0.95, confidence level. Even though ProUCL can compute UCLs for any confidence coefficient level in the interval (0.5, 1.0), the recommendations are provided only for 95% UCL; as EPC term is estimated by a 95% UCL of the mean. Notes: Like all other methods, it is recommended that the user identify a few low probability (coming from extreme tails) outlying observations that may be present in the data set. Outliers distort statistics of interest including summary statistics, data distributions, test statistics, UCLs and BTVs. Decisions based upon distorted statistics may be misleading and incorrect. The objective is to compute decision statistics based upon the majority of the data set representing the main dominant population. The project team should decide about the disposition (to include or not to include) of outliers before computing estimates of EPCs and BTVs. To determine the influence of outliers on UCLs and background statistics, the project team may want to compute statistics twice: once using the data set with outliers, and once using the data set without outliers. Note on Computing Lower Confidence Limits (LCLs) of the Mean: In several environmental applications, one needs to compute a LCL of the population mean. At present, ProUCL does not directly compute LCLs of mean. It should be pointed out that for data sets with and without NDs, except for the bootstrap methods, gamma distribution (e.g., samples of sizes <50), and H-statistic based LCL of mean, the same critical value (e.g., normal z value, Chebyshev critical value, or t-critical value) are used to compute a LCL of mean as used in the computation of the UCL of mean. Specifically, to compute a LCL, the '+' sign used in the computation of the corresponding UCL needs to be replaced by the '-' sign in the equation used to compute that UCL (excluding gamma, lognormal H-statistic, and bootstrap methods). For specific details, the user may want to consult a statistician. For data sets without nondetect observations, the user may want to use the Scout 2008 software package (EPA 2009c) to directly compute the various parametric and nonparametric LCLs of mean. 201 ------- 11.1 UCLs for Full (w/o NDs) Data Sets 11.1.1 Normal Distribution (Full Data Sets without NDs) 1. Click UCLs/EPCs ^ Full (w/o NDs) ^ Normal ^^^^^^^^^^^^^H ;s ISM Simulator Graphs Statistical Tests Upper Limrts/BTVs UCLs/EPCs I Windows Help 2. D TCE 1 D_TCE 2 3 4 5 Full (w/o NDs) > With NDs t 0.81 1 0.68 0 0.68 0 0.95 1 0.68 51 51 61 0.68 61 9.29 | 1.9 0.88 2.98 0.75 5.97 Normal Gamma Lognormal Non-Parametric All 12 13 The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of available variables to select a group variable. :lect Variables Available Variables Selected Variables Name ID Name TCE ID 0 Select Group Column (Optional) When the Option button is clicked, the following window will be shown. 202 ------- Select Confidence Lev Confidence Level o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Click on OK button to continue or on Cancel button to cancel the option. • Click on OK to continue or on Cancel to cancel the UCL computation option. Example 11-1. Consider the data used in Example 1-1 collected from a Superfund site; vanadium concentrations follow a normal distribution. The normal distribution based 95% UCLs of mean are summarized in the following table. Vanadium - Output Screen for Normal Distribution (Full Data w/o NDs) Normal UCL Statistics for Uncensored Full Data Sets User Selected Options Date/Time of Computation 3/25/2013 3:53:05 PM From File SuperFundjds Full Precision OFF Confidence Coefficient 95V. Vanadium General Statistics Total Number of Observations 20 Minimum 7.2 Maximum 32 SD 3.075 Number of Distinct Observations 17 Number of Missing Observations 0 Mean 17.34 Median 16.5 SD of logged Data 0.492 Coefficient of Variation 0.4S8 Skewness 0.429 Normal GOF lest Shapiro Wilk Test Statistic 0.925 Shapiro Wilk GOF Test 5!i Shapiro Wilk Critical Value 0.905 Data appear Normal at 5% Significance Level ulliefore Test Statistic 0.146 LJIIiefors GOF Test 5% Lilliefore Critical Value 0.198 Data appear Normal at 5« Significance Level Data appear Normal at 5% Significance Level Assuming Normal Distribution 95% Normal UCL SE:= Students4 UCL 20.4S 95% UCLs (Adjusted for Skewness) 95% Adjusted-CLT UCL (Chen-1995) 20.49 95'i Modified4 UCL (Johnson-1978) 20.49 Suggested UCL to Use 95'i Student's-t UCL 20.46 203 ------- 11.1.2 Gamma, Log normal, Nonparamethc, All Statistics Option (Full Data without NDs) 1. Click UCLs/EPCs ^ Full (w/o NDs) ^ Gamma, Lognormal, Non-Parametric, or All 2. Upper Limits/BTVs 1 5 UCLs/EPCs Windows Help Full (w/o NDs] > With NDs > Normal Gamma Lognormal Non-Parametric All 12 13 The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of available variables, and select a proper group variable. • When the Option button is clicked, the following window will be shown. sleet UCL Option Confidence Level Number of Bootstrap Operations 2000 OK Cancel o Specify the Confidence Level; a number in the interval [0.5, 1), 0.5 inclusive. o Specify the Number of Bootstrap Operations (runs). Default choice is 2000. o Click on OK button to continue or on Cancel button to cancel the UCLs option. • Click on OK to continue or on Cancel to cancel the selected UCL computation option. Example 11-2: This skewed data set of size n=25 with mean=44.09 was used in Chapter 2 of the Technical Guide. The data follows a lognormal and a gamma distribution. The data are: 0.3489, 0.8526, 2.5445, 2.5602, 3.3706, 4.8911, 5.0930, 5.6408, 7.0407, 14.1715, 15.2608, 17.6214, 18.7690, 23.6804, 25.0461, 31.7720, 60.7066, 67.0926, 72.6243, 78.8357, 80.0867, 113.0230, 117.0360, 164.3302, and 169.8303. UCLs based upon Gamma, Lognormal, Non-parametric, and All options are summarized in the following tables. 204 ------- Output Screen for Gamma Distribution Based UCLs (Full [w/o NDs]) General Statistics Total Number of Observations 25 Minimum Maximum SD Coefficient of Variation 0.349 163.8 51.34 1.164 Number of Distinct Observations Number of Missing Observations Mean Median SD of logged Data 25 0 4409 18.77 1.SB 1.294 Gamma GOF Test A-D Test Statistic 0.374 Anderson-Darting Gamma GOF Test 5% A-D Critical Value 0.794 Data appear Gamma Distributed at 5% Significance Level K-S Test Statistic 0.113 Kblmogrov-Smimoff Gamma GOF Test 5% K-S Critical Value 0.183 Data appear Gamma Distributed at 5'i Significance Level Data appear Gamma Distributed at 5% Significance Level Gamma Statistics k hat (MLE) 0.643 Tneta hat (MLE) 68.58 nu hat (MLE) 32.15 MLE Mean (bias corrected) 44.09 Adjusted Level of Significance 0.0395 k star (bias corrected MLE) 0.592 Theta star (bias corrected MLE) 74.42 nu star Ibias corrected) 29.62 M LE Sd (bias corrected) 57.28 Approximate Chi Square Value (D.05) 1 8.2 Adjusted Chi Square Value 17.59 Assuming Gamma Distribution 95% Approximate Gamma UCl (use when n> =50) 71.77 95V. Adjusted Gamma UCL (use when nc50) 74.27 Suggested UCL to Use 95% Adjusted Gamma UCL 74.27 Output Screen for Lognormal Distribution Based UCLs (Full [w/o NDs]) Total Number of Observations Minimum Maximum SO] Coefficient of Variation General Statistics 25 0.349 169.8 51.34 1.164 Number of Distinct Observations Number of Missing Observations Mean Median Std. Error of Mean Skewness Lognomal GOF Test Shapiro Wilk Test Statistic 0.943 Shapiro Wilk Lognomal GOF Test 5% Shapiro Wilk Critical Value 0.918 Data appear Lognormal at 5'i Significance Level Lilliefors Test Statistic 0.135 Lilliefors Lognormal GOF Test 5% Lilliefors Critical Value D.177 Data appear Lognormal at 55 Significance Level Data appear Lognormal at 5% Significance Level Lognormal Statistics Minimum of Logged Data -1.053 Maximum of Logged Data 5.135 Mean of logged Data SD of logged Data 95% Chebyshev (MVUE) UCL 35/i Chebyshev (MVUE) UCL Assuming Lognormal Distribution 951'. H-UCL 229.2 176.3 323 25 0 4409 18.77 10.27 1.234 2.835 1.68 Ml Chebyshev (MVU E) UCL 140.6 97.5% Chebyshev (MVUE) UCL 225.8 Suggested UCL to Use Data appear Gamma. May want to try Gamma Distribution 205 ------- Output Screen for Nonparametric UCLs (Full [w/o NDs]) Nonparametric Distribution Free UCLs 95%CLTUCL 60.9! 95% Standard Bootstrap UCL 60.* 95% Halls Bootstrap UCL G2.1- 95%JackknifeUCL * 95% Bootstrap-t UCL ) 95% Percentile Bootstrap UCL 61.66 65 61.42 95% BCA Bootstrap UCL 63.63 90% ChebyshevfMean, Sd) UCL 74.89 95% ChebyshevfMean 97.5% ChebyshevfMean , Sd) UCL 1 08.2 99%Chebyshev(Mean Sd} UCL Sd) UCL 3S.35 146.3 Suggested UCL to Use Data appear Gamma. May want to try Gamma_Distribution Output Screen for All Statistics Option (Full [w/o NDs]) Total Number of Observations General Statistics 25 Minimum Maximum SD Coefficient of Variation Shapiro Wilk Test Statistic 5?; Shapiro Wilk Critical Value Lilliefons Test Statistic 5% Lilliefons Critical Value Normal GOF Test 0.799 0.918 0.245 0.177 Shapiro Wilk GOF Test Data Not Normal at 55i Significance Level Lilliefors GOF Test Data Not Normal at 5% Significance Level C.34S 169.8 51.34 1.164 Number of Missing Observations Mean Median Std. Error of Mean Skewness 0 44.09 18.77 10.27 1.294 Data Not Normal at 5% Significance Level Assuming Normal Distribution 95% Normal UCL 957. Students4 UCL 61.66 95% UCLs (Adjusted for Skewness) 95% Adjusted-CLT UCL (Chen-1995) 63.82 95% Modified-t UCL (Johnson-197S) 62.1 A-D Test Statistic 5% A-D Critical Value K-S Test Statistic 5% K-S Critical Value Gamma GOF Test 0.374 Anderson-Darling Gamma GOF Test 0.794 Detected data appear Gamma Distributed at 5% Significance Level 0.113 Kblmogrov-Smirnoff Gamma GOF Test 0.183 Detected data appear Gamma Distributed at 5% Significance Level Detected data appear Gamma Distributed at 5% Significance Level 206 ------- Continued: Output Screen for All Statistics Option (Full [w/o NDs]) khat(HLE) Thetahat(MLE) nu hat (MLE) MLE Mean Ibias corrected) Gamma Statistics G.W3 68.58 32.15 fijoa Adjusted Level of Significance 0.0395 k star bias corrected MLE) Theta star (bias corrected MLE) nu star (bias corrected) MLE Sd (bias corrected) Approximate Chi Square Value (0.05) Adjusted Chi Square Value Assuring Gamma Distribution 95% Approximate Gamma UCL (use when n>=50) 71." Lognormal GOF Test Shapiro Wilk Test Statistic 0.948 Shapiro Wilk Lognoimal GOF Test 5% Shapiro Wilk Critical Value 0.918 Data appear Lognormal at 5% Significance Level LJIIiefors Test Statistic 0.135 Lilliefors Lognonnal GOF Test 5% Lilliefors Critical Value 0.177 Data appear Lognormal at 5% Significance Level Data appear Lognonnal at 5% Significance Level Lognormal Statistics Minimum of Logged Data -1.053 Maximum of Logged Data 5.135 Mean of logged Data SD of logged Data Assuming Lognormal Distribution 95% H-UCL 229.2 95% Chebyshev (MVU E) UCL 176.3 99% Chebyshev (MVU E) UCL 323 Nonparametric Distribution Free UCL Statistics Data appear to follow a Discernible Distribution at 5% Significance Level Nonparametric Distribution Free UCLs 95%CLTUCL 60.98 95% Standard Bootstrap UCL 60.45 35% Halls Bootstrap UCL 63.51 95% BCA Bootstrap UCL 64.96 90% Chebyshev (Mean, Sd) UCL 74.89 97.5% Chebyshev{Mean, Sd) UCL 108.2 0.592 74.42 29.62 57.28 18.2 17.59 55% Adjusted Gamma UCL [use when rK5D) 74.27 2.835 1.68 90% Chebyshev fMVUE) UCL 140.6 97.5% Chebyshev (MVUE) UCL 225.S 95%JackknifeUCL 61.66 95% Bootstrap^ UCL 65.83 95% Percentile Bootstrap UCL 61,84 95% ChebyshevfMean, Sd) UCL 88.85 99% Chebyshev(Mean, Sd) UCL 146.3 Suggested UCL to Use 95% Adjusted Gamma UCL 74.27 Notes: Once again, the statistics summarized above demonstrate the merits of using the gamma distribution based UCL of mean to estimate EPCs. The use of a lognormal distribution tends to yield unrealistic UCLs without practical merit (e.g., Lognormal UCL = 229.2 and the maximum = 169.8 in the above example). The results summarized in the above tables suggest that the use of a gamma distribution (when a data set follows a gamma distribution) cannot be dismissed just because it is easier (Helsel and Gilroy 2012) to use a lognormal distribution to model skewed data sets. 207 ------- Number of valid samples represents the total number of samples minus (-) the missing values (if any). The number of unique or distinct samples simply represents number of distinct observations. The information about the number of distinct values is useful when using bootstrap methods. Specifically, it is not desirable to use bootstrap methods on data sets with only a few distinct values. 11.2 UCL for Left-Censored Data Sets with NDs 1. Click UCLs/EPCs ^ With NDs .tatistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help 3 4 5 Full (w/o NDs) f Wrth NDs t 3 10 11 12 13 Norma Gamma Lognormal Non-Parametric All Choose the Normal, Gamma, Lognormal, Non-Parametric, or All option. The Select Variables screen (Chapter 3) will appear. • Select a variable(s) from the Select Variables screen. • If needed, select a group variable by clicking the arrow below the Select Group Column (Optional) to obtain a drop-down list of available variables, and select a proper group variable. The selection of this option will compute the relevant statistics separately for each group that may be present in the data set. • When the Option button is clicked, the following window will be shown. Confidence Level Number of Bootstrap Operations 2000 I OK Cancel o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Specify the Number of Bootstrap Operations (runs). Default choice is 2000. o Click on OK button to continue or on Cancel button to cancel the UCLs option. Click on OK to continue or on Cancel to cancel the selected UCL computation option. 208 ------- Example 11-3. This real data set of size n=55 with 18.8% NDs (=10) is also used in Chapters 4 and 5 of the ProUCL Technical Guide. The minimum detected value is 5.2 and the largest detected value is 79000, sd of detected logged data is 2.79 suggesting that the data set is highly skewed. The detected data follow a gamma as well as a lognormal distribution. It is noted that GROS data set with imputed values follows a gamma distribution and LROS data set with imputed values follows a lognormal distribution (results not included). The lognormal Q-Q plot based upon detected data is shown in the following figure. The various UCL output sheets: normal, nonparametric, gamma, and lognormal generated by ProUCL are summarized in tables following the lognormal Q-Q plot on detected data. The main results have been high-lighted in the output screen provided after the lognormal GOF Q-Q plot. Lognormal Q-Q Plot (Statistics using Detected Data) for A-DL FWwtNDi-H* n ]«ISL*ihr.=aiW Cmc4Vik»IDQ5|-am D*si«>«a Loarnyiwl Theoretical QuanWes (Standard Normal) Output Screen for UCLs based upon Normal, Lognormal, and Gamma Distributions (of Detects) A-DL General Statistics Total Number of Obseivations ' 55 Number of Detects' 45 Number of Distinct Detects ' 45 Minimum Detect ' 5.2 Maximum Detect '79000 Variance Detects I5.954E»B Mean Detects '1 D556 Median Detects '1940 Skevmess Detects ' 2.632 Mean of Logged Detects ' 7.031 Number of Distinct Observations ' Number of Non-Detects ' Number of Distinct Non-Detects ' Minimum Non-Detect ' Maximum Non-Detect ' Percent Non-Detects ' SD Detects ' CV Detects ' Kurtosis Detects ' SD of Logged Detects ' 53 10 8 3.3 124 18.18% 19S86 1.884 6.496 2.788 Normal GOF Test on Detects Only Shapiro Wilk Test Statistic ' 0.575 St. Shapiro Wilk Critical Value ' 0.945 Shapiro Wilk GOF Test Detected Data Not Normal at 5% Significance Level Ulliefors Test Statistic ' 0.298 LJIIiefore GOF Test 5% Ulliefors Critical Value ' 0.132 Detected Data Not Normal at 5% Significance Level Detected Data Not Noraal at 55 Significance Level Kaplan-Meier (KM) Statistics using Normal Critical Values and other Nonparametric UCLs Mean'8638 Standard Error of Mean ' 2488 S D '18246 55% KM (BCA) UCL '13562 95% KM () LICL '12802 95% KM (Percerrtile Bootstrap) UCL '13040 95% KM (z) UCL'12731 95% KM Bootstrap! UCL'15221 90% KM Chebyshev UCL '16102 95% KM Chebyshev UCL '19483 97.5% KM Chebyshev UCL '24176 99% KM Chebyshev UCL '33394 209 ------- Continued: Output Screen for UCLs based upon Normal, Lognormal, and Gamma Distributions (of Detects) Gamma GOF Tests on Detected Observations Only A-D Test Statistic T 0.591 Anderson-Darling GOF Test 5% A-D Critical Value ' 0.86 Detected data appear Gamma Distributed at 5% Significance Level K-S Test Statistic ' 0.115 Kdmogrov-Smimoff GOF 5% K-S Critical Value ' 0.143 Detected data appear Gamma Distributed at 5% Significance Level Detected data appear Gamma Distributed at 5% Significance Level Gamma Statistics on Detected Data Only khat{MLE)r 0.307 Thetahat(MLE)r34333 nuhat{MLE)r 27.67 MLE Mean (bias corrected) "10556 k star (bias corseted MLE)r 0.302 Theta star (bias corrected MLE) r34380 nu star (bias corrected) 27.16 MLE Sd (bias corrected) "1 5216 Gamma Kaplan-Meier (KM) Statistics khat(KM)r 0.224 nu hat (KM) Approximate Chi Square Value (24.66, a)' 14.35 Adjusted Chi Square Value (24.66, p) 95% Gamma Approximate KM-UCL (use when n>=5D) "14844 95*/; Gamma Adjusted KM-UCL (use when n<50) GROS Statistics using imputed NDs MinimumT 0.01 Maximum "7SOOQ SD "18415 k hat (MLE)r 0.18 Theta hat (MLE) "47915 nuhat(MLE)r 19,83 MLE Mean (bias corrected)T8637 Approximate Chi Square Value (20.08, a)' 10.91 95% Gamma Approximate UCL (use when n>=50) "15896 Mean Median CV k star (bias corrected MLE) Theta star (bias corrected MLE) nu star (bias corrected) MLE Sd Ibias corrected) Adjusted Level of Significance (3) Adjusted Chi Square Value (20.08. p) 95% Gamma Adjusted UCL (use when n<50) 24.66 14.14 r15066 r8637 T 588 2.132 0.183 r47314 20.08 "20215 0.0456 10.73 '16167 . ) Lognormal GOF Test on Detected Observations Only Shapiro Wilk Test Statistic r 0.939 Shapiro Wilk GOF Test 5% Shapiro Wilk Critical Value ' 0.945 Detected Data Not Lognormal at 5% Significance Level Lilliefors Test Statistic T 0.104 LJIIiefors GOF Test 5% LJIIiefons Critical Value ' 0.132 Detected Data appear Lognormal at 5% Significance Level Detected Data appear Approximate Lognormal at 5% Significance Level Logrormal ROS Statistics Using Imputed Non Detects Mean in Original Scale r 8638 SD in Original Scale r1S414 95% t UCL (assumes normality of ROS data) "12793 95% BCA Bootstrap UCL "14069 95% H-UCL (Log ROS) "1855231 Mean in Log Scale T 5.983 SD in Log Scale r 3.391 95% Percentile Bootstrap UCL'13090 95%Bootstrap! UCL"15524 UCLs using Lognormal Distribution and KM Estimates when Detected data are Lognormally Distributed KM Mean flogged)" 6.03 95% H-UCL {KM -Log) 1173988 KM SD (logged)' 3.286 95% Critical H Value (KM-Log)' 5.7 KM Standard Error of Mean flogged)" 0.449 " " Suggested UCL to Use 95% KM (Chebyshev) UCL "19483 95% GROS Approximate Gamma UCL "15896 95% Approximate Gamma KM-UCL"!4844 " " 210 ------- Detected data follow a gamma as well as a lognormal distribution. The various upper limits using Gamma ROS and Lognormal ROS methods and Gamma and Lognormal distribution on KM estimates are summarized in the following table. Upper Confidence Limits Computed using Gamma and Lognormal Distributions of Detected Data Sample Size = 55, No. of NDs=10, % NDs = 18.18% Upper Limits Min (detects) Max (detects) Mean (KM) Mean (ROS) UCL95 (ROS) UCL (KM) Gamma Distribution Result 5.2 79000 8638 8637 15896 14844 Reference/ Method of Calculation - - - - ProUCL5.0-GROS ProUCL 5.0 - KM-Gamma Lognormal Distribution Result 1.65 11.277 6.3 8638 14863 12918 1173988 Reference/ Method of Calculation Logged Logged Logged - bootstrap-t on LROS, ProUCL 5.0 percentile bootstrap on LROS,Helsel(2012) H-UCL, KM mean and sd on logged data, EPA (2009) All computations have been performed using the ProUCL software. In the above table, methods proposed/described in the literature have been cited in the Reference Method of Calculation column. The results summarized in the above table re-iterate that the use of a gamma distribution cannot be dismissed just because it is easier to use a lognormal distribution to model skewed data sets. These results also demonstrate that for skewed data sets, one should use bootstrap methods which adjust for data skewness (e.g., bootstrap-1 method) rather than using percentile bootstrap method. 211 ------- Chapter 12 Sample Sizes Based Upon User Specified Data Quality Objectives (DQOs) and Power Assessment One of the most frequent problems in the application of statistical theory to practical applications, including environmental projects, is to determine the minimum number of samples needed for sampling of reference/background areas and survey units (e.g., potentially impacted site areas, areas of concern, decision units) to make cost-effective and defensible decisions about the population parameters based upon the sampled discrete data. The sample size determination formulae for estimation of the population mean (or some other parameters) depend upon certain decision parameters including the confidence coefficient, (1-a) and the specified error margin (difference), A from the unknown true population mean, fj,. Similarly, for hypotheses testing approaches, sample size determination formulae depend upon pre- specified values of the decision parameters selected while describing the data quality objectives (DQOs) associated with an environmental project. The decision parameters associated with hypotheses testing approaches include Type I (false positive error rate, a) and Type II (false negative error rate, /?=!-power) error rates; and the allowable width, A of the gray region. For values of the parameter of interest (e.g., mean, proportion) lying in the gray region, the consequences of committing the two types of errors described above are not significant from both human health and cost-effectiveness point of view. Note: Initially, the Sample Sizes module was incorporated in ProUCL 4.0/ProUCL 4.1. Not many changes have been made in ProUCL 5.0/ProUCL 5.1 except those described below. Therefore, many screenshots generated using an earlier 2010 version of ProUCL have been used in the examples described in this chapter. Both parametric (assuming normality) and nonparametric (distribution free) sample size determination formulae as described in guidance documents (MARS SIM 2000, EPA 2002c and 2006a) have been incorporated in the ProUCL software. Specifically, the DQOs Based Sample Sizes module of ProUCL can be used to determine sample sizes to estimate the mean, perform parametric and nonparametric single-sample and two-sample hypothesis tests, and apply acceptance sampling approaches to address project needs of the various CERCLA and RCRA site projects. The details can be found in Chapter 8 of the ProUCL Technical Guide and in EPA guidance documents (EPA 2006a, 2006b). New in ProUCL 5.0/ProUCL 5.1: The Sample size module in ProUCL 5.0/ProUCL 5.1 can be used at two different stages of a project. Most of the sample size formulae require some estimate of the population standard deviation (variability). Depending upon the project stage, a standard deviation: 1) represents a preliminary estimate of the population (e.g., study area) variability needed to compute the minimum sample size during the planning and design stage; or 2) represents the sample standard deviation computed using the data collected without considering DQOs process which is used to assess the power of the test based upon the collected data. During the power assessment stage, if the computed sample size is larger than the size of already collected data set, it can be inferred that the size of the collected data set is not large enough to achieve the desired power. The formulae to compute the sample sizes during the planning stage and after performing a statistical test are the same except that the estimates of standard deviations are computed/estimated differently. Planning stage before collecting data: Sample size formulae are commonly used during the planning stage of a project to determine the minimum sample sizes needed to address project objectives (estimation, hypothesis testing) with specified values of the decision parameters (e.g., Type I and II errors, width of 212 ------- gray region). During the planning stage, since the data are not collected a priori, a preliminary rough estimate of the population standard deviation (to be expected in sampled data) is obtained from other similar sites, pilot studies, or expert opinions. An estimate of the expected standard deviation along with the specified values of the other decision parameters are used to compute the minimum sample sizes needed to address the project objectives during the sampling planning stage; the project team is expected to collect the number of samples thus obtained. The detailed discussion of the sample size determination approaches during the planning stage can be found in MARSSIM 2000 and U.S. EPA 2006a. Power assessment stage after performing a statistical method: Often, in practice, environmental samples/data sets are collected without taking the DQOs process into consideration. Under this scenario, the project team performs statistical tests on the available already collected data set. However, once a statistical test (e.g., WMW test) has been performed, the project team can assess the power associated with the test in retrospect. That is for specified DQOs and decision errors (Type I error and power of the test [=l-Type II error]), using the sample standard deviation computed based upon the already collected data, the minimum sample size needed to perform the test for specified values of the decision parameters is computed. • If the computed sample size obtained using the sample variance is less than the size of the already collected data set used to perform the test, it may be determined that the power of the test has been achieved. However, if the sample size of the collected data is less than the minimum sample size computed in retrospect, the user may want to collect additional samples to assure that the test achieves the desired power. • It should be pointed out that there could be differences in the sample sizes computed in two different stages due to the differences in the values of the estimated variability. Specifically, the preliminary estimate of the variance computed using information from similar sites could be significantly different from the variance computed using the available data already collected from the study area under investigation which will yield different values of the sample size. Sample size determination methods in ProUCL can be used for both stages. The only difference will be in the input value of the standard deviation/variance. It is user's responsibility to input a correct value for the standard deviation during the two stages. File Edit Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation F Name Worksheet it> General Statistics ^ mputed NDs using ROS Methods > DQOs Based Sample Sizes t WMW^ith NDsxIs ASHALL7groupsxls Box Plot Full.gst Box Plot Full_a.gst Box Plot Full_b.gst 3 4 5 6 7 8 5 8" 7| 17] 12J 20 15 25 18 34 21j~~ 35 3 F°\ D-Ste 4 5 Estimate Mean Hypothesis Tests ^ Acceptance Sampling 1 1 1 1 1 1 6 7 8 9 1 Single Sample Tests * Two Sample Tests ^ 0 11 tTest Proportion Sign Test Wilcoxon Sign sd Rank 213 ------- 12.1 Estimation of Mean 1. Click Stats/Sample Sizes^ DQOs Based Sample Sizes ^ Estimate Mean •a File Edit Navigation F Name Worksheet jds Well 10 Jds WMW-with NDsxl ASHALL7groupsj< Box Plot Full.gst Stats/Sample Sizes | Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help General Statistics > Imputed NDs using ROS Methods ^ DQOs Based Sample Sizes > s s 2 3 4 5 5 7 12 8 — 17 3 J9"5' D_Ste 4 5 Estimate Mean Hypothes s Tests ^ Acceptance Sampling 2d| 1 1 6 7 2. The following options window is shown. Select Sample Size Options Confidence Level Allowable Error Margin in Mean Estimate Estimate of Standard Deviation I OK • Specify the Confidence Coefficient. Default is 0.95. • Specify the Estimate of standard deviation. Default is 3. • Specify the Allowable Error Margin in Mean Estimate. Default is 10. • Click on OK button to continue or on Cancel button to cancel the options. Output Screen for Sample sizes for Estimation of Mean (CC = 95%, sd = 25, Error Margin = 10) S ample 5 ize f 01C si imation of M ean Based on Specified Values of Decision Par ameters/DQ Os [Data Quafty Objectives) Date/Time of Computation 2/26/201012:12:37 PM User Selected Options Confidence Coefficient 35% Allowable Error Margin 10 Estimate of Standard Deviation 25 95% Confidence Coefficient: proximate Minimum Sample Size 26 214 ------- 12.2 Sample Sizes for Single-Sample Hypothesis Tests 12.2.1 Sample Size for Single-Sample t-Test 1. Click DQOs Based Sample Sizes ^- Hypothesis Tests ^- Single Sample Tests ^-1 Test ab1 File Edit Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation FJ General Statistics ^ Name r nputed NDs using ROS Methods > Wo*Sheetjls 1 DQOs Based Sample Sizes t WMW-with NDsxIs ASHALL7groupsxls Box Plot Full.gst Box Plot Full_a gst Box Plot Full_b.gst 2 4 S 6 7 8 5 8 7 17 12 20 15 25 18 34 21 35 3-56789 10 11 J^" OS* | Estimate Mean Hypothesis Tests > Single Sample Tests ¥ Acceptance Sampling Two Sample Tests t 1 1 1 T[ tTest Proportion Sign Test Wilcoxon Signed Rank 1 1 The following options window is shown. False Rejection Rate [Alpha] False Acceptance Rate [Beta] O 0.005 O 0.010 O O.025 (I> 0.050 O O.100 O 0.150 O 0.200 O 0.250 [0.5%] [1.0%] [25%] [5.0%] [10.%] [15%] [20.%] [25.%] O 0.005 O 0.010 O 0.025 O 0.050 ® 0.100 O 0.150 O 0.200 O 0.250 [0.5%] [10%] [2.5%] [5.0%] [10.%] [15%] [20.%] [25%] Estimate of Population SD Preliminary Estimate (planning stage) Sample SD using collected data (to assess power) Width of Gray Region [Delta] OK Cancel • Specify the False Rejection Rate (Alpha, a). Default is 0.05. • Specify the False Acceptance Rate (Beta, P). Default is 0.1. • Specify the Estimate of standard deviation. Default is 3. • Specify the Width of the Gray Region (Delta, A). Default is 2. • Click on OK button to continue or on Cancel button to cancel the options. 215 ------- Output Screen for Sample Sizes for Single-Sample t-Test (« = 0.05, /? = 0.2, sd = 10.41, A = 10) Example from EPA 2006a (page 49) ! Sample Sizes for Single Sample (Test Based on Specified Values of Decision Parameteis/DQOs (Data Quafty Objectives) D ate/T ime of Computation 2/26/2010 12:41:58 PM User Selected Options False Rejection Rate [Alpha] 0.05 False Acceptance R ate [B eta] 0.2 Width of Q ray R egion [D elta] 10 EstimateofStandardDeviation 10.41 Approximate Minimum Sample Size Single Sided Alternative Hypothesis: Two Sided Alternative Hypothesis: 11 12.2.2 Sample Size for Single-Sample Proportion Test 1. Click DQOs Based Sample Sizes ^- Hypothesis Tests ^- Single Sample Tests ^- Proportion «J File Edit Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation F Name Worksheet xls Well 10xls WMW-wth NDsxl ASHALL7groupsjt Box Plot Full .gst Box Plot Full_a.gst Box Plot Full_b. gst General Statistics ^ Imputed NDs using ROS Methods > DQOs Based Sample Sizes 3 S 2 3 4 5 6 7 T 17 12 20 15 25 18 34 > *sro 3 D_Site 4 5 Estimate Mean HypothessTests t Acceptance Sampling 1 1 1 1 1 1 6 7 8 9 10 11 Single Sample Tests ^ Two Sample Tests ^ tTest Sign Test VYilcoxon Signed Rank 2. The following options window is shown. Single Sample Proportion Test Sample Size Options False Rejection Rate [Alpha] False Acceptance Rate [Beta] O 0.005 [0.5%] O 0.010 [i.o%] O 0.025 [2.5%] ® 0.050 [5.0%] O 0.100 [10.%] O 0.150 [15.%] O 0.200 [20.%] O 0.250 [25.%] Desirable Proportion [PO] Preliminary Estimate (planning stage) Sample Proportion using collected data Qo assess power) 0.3 O 0.005 [0.5%] O 0.010 [1.0%] O 0.025 [2.5%] O 0.050 [5.0%] ® 0.100 [10.%] O 0.150 [15.%] O 0.200 [20.%] O 0.250 [25.%] Width of Gray Region [Delta] 0.15 216 ------- • Specify the False Rejection Rate (Alpha, a). Default is 0.05. • Specify the False Acceptance Rate (Beta, /?). Default is 0.1. • Specify the Desirable Proportion (PO). Default is 0.3. • Specify the Width of the Gray Region (Delta, A). Default is 0.15. • Click on OK button to continue or on Cancel button to cancel the options. Output Screen for Sample Size for Single-Sample Proportion Test (« = 0.05, /? = 0.2, PO = 0.2, A 0.05) Example from EPA 2006a (page 59) i S ample S izes for Single S ample Proportion Test Based on Specified Values of Decision Parameters/DQ Os (Data Quafty Objectives) DateATime of Computation 2/26^2010 12:50:52 PM User Selected Options False Rejection Rate [Alpha] 0.05 False Acceptance Rate [Beta] 0.2 Width of Gray Region [Delta] 0.05 Proportion/Action Level [PO] 0.2 Approximate Minimum Sample Size R ight S ided Alternative Hypothesis: 413 Left S ided Alternative Hypothesis: 363 Two Sided Alternative Hypothesis: man[471,528) 72.2.3 Sample Size for Single-Sample Sign Test 1. Click DQOs Based Sample Sizes ^ Hypothesis Tests^- Single Sample Tests^- Sign Test - [WMW-i a^ File Edit Navigation F Name Worksheet* Well 10* WMW-wth NDsxl ASHALL7graupsx Box Plot Full.gst Box Plot Full_a.gst Stats/Sample Sizes General Statist! mputed NDs u Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help :: t sing ROS Methods ^ DQOs Based Sample Sizes t s s 2 3 4 5 6 7 5 7 12 15 18 8 17 20 25 3? 3 ^=r:i C_Site 4 5 6 Estimate Mean Hypothes s Tests > Acceptance Sampling 1 1 1 1 7 a 9 1 0 11 Single Sample Tests > Two Sample Tests > tTest Proportion Sign Test Wilcoxon Signed Rank 2. The following options window is shown. 217 ------- Single Sample Sign Test Sample Size Options False Rejection Rate [Alpha] False Acceptance Rate [Beta] O 0.005 O 0.010 O 0.025 ® 0.050 O 0.100 O 0.150 O 0.200 O 0.250 [0.5%] [10%] [2.5%] [5.0%] [10.%] [15.%] [20.%] [25.%] O 0.005 O 0.010 O 0.025 O 0.050 'ft 0.100 O 0.150 O 0.200 O 0.250 [0.5%] [10%] [2.5%] [5.0%] [10.%] [15.%] [20.%] [25.%] Estimate of Population SD Preliminary Estimate (planning stage) Sample SD using collected data (to assess power) Width of Gray Region [Delta] OK • Specify the False Rejection Rate (Alpha, a). Default is 0.05. • Specify the False Acceptance Rate (Beta, P). Default is 0.1. • Specify the Estimate of standard deviation. Default is 3 • Specify the Width of the Gray Region (Delta, A). Default is 2. • Click on OK button to continue or on Cancel button to cancel the options. Output Screen for Sample Sizes for Single-Sample Sign Test (Default Options) I Sample Sizes for Single Sample Sign Test Based on Specified Values of Decision Paiameteis/DQOs [Data Qua% Objectives) Date.'Tirne of Computation 2/26/2010 12:15:27 PM User Selected Options False Rejection Rate [Alpha] 0.05 False Acceptance Rate [Beta] 0.1 Width of Bray Region [Delta] 2 Estimate of Standard Deviation 3 Approximate Minimum Sample Size Single Sided Alternative Hypothesis: 35 Two Sided Alternative Hypothesis: 43 218 ------- 72.2.4 Sample Size for Single-Sample Wilcoxon Signed Rank Test 1. Click DQOs Based Sample Sizes ^- Hypothesis Tests ^- Single Sample Tests ^- Wilcoxon Signed Rank . 5.0 - [WMW-with NDs.xls] File Edit Navigation F Stats/Sample Sizes Graphs Statistical Tests Upper Limte/BTVs UCLs/EPCs Windows Help Name Worksheet jds Well 10:ds WMW^ith NDsxIs ASHALL7groupsjds Box Plot Full.flst Box Plot Full_a.gst Box Plot Full_b.gst General Statistics Imputed NDs using ROS Methods DQOs Based Sample Sizes 17 12 15 18 D Site Estimate Mean Hypothesis Tests Acceptance Sampling 10 Single Sample Tests Two Sample Tests 20 25 34 tTest Proportion Sign Test Wilcoxon Signed Rank 2. The following options window is shown. Single Sample Wilcoxon Signed Rank Test Sample Size Options False Rejection Rate [Alpha] O 0.005 [0.5%] O 0.010(1.0%] O 0.025 [2.5%] (?) 0.050 [50%] O 0.100 [10%] O 0.150(15.%] O 0.200 [20 %] O 0.250 [25.%] Estimate of Population SD Preliminary Estimate (planning stage) Sample SD using collected data (to assess power) False Acceptance Rate [Beta] O 0.005 [0.5%] O 0.010(1 0%] O 0.025 [2.5%] 0.050 [5.0%] ® 0.100(10%] O 0.150(15.%] O 0.200 [20.%] O 0.250 [25 %] Width of Gray Region [Delta] Specify the False Rejection Rate (Alpha, a). Default is 0.05. Specify the False Acceptance Rate (Beta, P). Default is 0.1. Specify the Estimate of standard deviation of WSR Test Statistic. Default is 3 Specify the Width of the Gray Region (Delta, A). Default is 2. Click on OK button to continue or on Cancel button to cancel the options. 219 ------- Output Screen for Sample Sizes for Single-Sample WSR Test (« = 0.1, /? = 0.2, sd = 130, A = 100) Example from EPA 2006a (page 65) Date/Time of Computation User Selected Options False Rejection Rate [Alpha] False Acceptance Rate [Beta] Width of Gray Region [Delta] Estimate of Standard Deviation Sample Sizes toi Single Sample WilcoKon Signed Rank Test Based on Specified Values of Decision Parameters/DQOs (Data Quafty Objectives) 2/26/20101:13:58PM Single Sided Alternative Hypothesis: Two Sided Alternative Hypothesis: 0.1 0.2 100 130 Approximate Minimum Sample Size 10 14 12.3 Sample Sizes for Two-Sample Hypothesis Tests 12.3.1 Sample Size for Two-Sample t-Test 1. Click DQOs Based Sample Sizes ^ Hypothesis Tests ^ Two Sample Tests ^ t Test CL5,0-[WMW-withNC File Edit I Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation F Name WofcSheeWs Well Hbds WMW-wBi NDsjt ASHALL7graupsj Box Plot Full .git Box Plot Full_a.gsl Rnx Pint Fi ill h nsl General Statistics ^ Imputed NDs using ROS Methods > DQOs Based Sample Sizes ^ 5 S 2 3 4 5 6 * 7 12 15 * 17 20 25 3 f9"* D_Ste 4 5 6 Estimate Mean Hypothesis Tests ^ Acceptance Sampling 7 8 9 10 11 12 Single Sample Tests t Two Sample Tests > 1 1 1 1 ties, | Wilcoxon-Mann-Whitney The following options window is shown. Two Sample t Test Sample Size Options False Rejection Rate [Alpha] O 0.005 [0.5%] O 0.010 [i.o%] O O.025 [2.5%] ® 0.050 [5.0%] O o.ioo [io.%] O O.150[15.%] O 0.200 [20.%] O O.250 [25.%] Pooled Estimate of Population SD Preliminary Estimate (planning stage) Sample SD using collected data (to assess power) 3 False Acceptance Rate [Beta] O 0.005 [0.5%] O 0.010(1.0%] O 0.025 [2.5%] O O.050 [5.0%] ® 0.100 [10.%] O O.150[15.%] O 0-200 [20.%] O 0.250 [25.%] Width of Gray Region [Delta] 220 ------- • Specify the False Rejection Rate (Alpha, a). Default is 0.05. • Specify the False Acceptance Rate (Beta, P). Default is 0.1. • Specify the Estimate of standard deviation. Default is 3 • Specify the Width of the Gray Region (Delta, A). Default is 2. • Click on OK button to continue or on Cancel button to cancel the options. Output Screen for Sample Sizes for Two-Sample t-Test (« = 0.05, /? = 0.2, sp = 1.467, A = 2.5) Example from EPA 2006a (page 68) i Sample Sizes for Two Sample (Test B ased on Specified Values of Decision Parameters/DQ0s [Data Qunity Objectives) Date/Time of Computation 2/26/20101:17:57 PM User Selected Options False R ejection R ate [Alpha] 0.05 False Acceptance Rate [Beta] 0.2 Width of Gray Region [Delta] 2.5 E stirnate of Pooled SD 1.467 Approximate Minimum Sample Size Single Sided Alternative Hypothesis: 5 Two Sided Alternative Hypothesis: 7 72.3.2 Sample Size for Two-Sample Wilcoxon Mann-Whitney Test 1. Click DQOs Based Sample Sizes ^ Hypothesis Tests ^ Two Sample Tests ^- Wilcoxon-Mann-Whitney DUCL 5.0 - [WMW-with NDs.> File Edit Stats/Sample Sizes Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation F Name Work Sheet jds Well 10.xb WMW-wlh NDsxl ASHALL7groupsi Box Plot Full.gst Box Plot FulLa.gst Rnv Pint Fl ill h nit General Stat sties * Imputed NDs using ROS Methods ^ DQOs Based Sample Sizes > s s 2 3 S 6 5 S 7 17 12 20 15 25 34567 f ro DSite | Estimate Mean Hypothesis Tests * Acceptance Sampling S Single SampleTests > Two SampleTests > 1 1 1 1 9 10 11 tTest Wilcoxon-Mann-Whitney 12 2. The following options window is shown. 221 ------- Two Sample Wilcoxon Mann-Whitney Test Sample Size Options False Rejection Rate [Alpha] O 0.005 [0.5%] O 0.010 [1.0%] O 0.025 [2.5%] ® 0.050 [5.0%] O 0.100 [10.%] O 0.150 [15.%] O 0.200 [20.%] O 0.250 [25.%] Pooled Estimate of Population SD Preliminary Estimate (planning stage) Sample SD using collected data (to assess power) False Acceptance Rate [Beta] O 0.005 [0.5%] O 0.010 [1.0%] O 0.025 [25%] O 0.050 [5.0%] ® 0.100 [10.%] O 0.150 [15.%] O 0.200 [20.%] O 0.250 [25.%] Width of Gray Region [Delta] • Specify the False Rejection Rate (Alpha, a). Default is 0.05. • Specify the False Acceptance Rate (Beta, P). Default is 0.1. • Specify the Estimate of standard deviation of WMW Test Statistic. Default is 3 • Specify the Width of the Gray Region (Delta, A). Default is 2. • Click on OK button to continue or on Cancel button to cancel the options. Output Screen for Sample Sizes for Single-Sample WMW Test (Default Options) i S ample S izes for T wo S ample Witc oxon- M ann -Whitney Test Based on Specified Values of Decision Paiamelets/DOOs [Data Quaify Objectives) Date/Time of Computation 2/26V2010 12:18:47 PM User Selected Options False Rejection Rate [Alpha] 0.05 False Acceptance Rate [Beta] 0.1 Width of G ray R egion [D elta] 2 Estimate of Standard Deviation 3 Approximate Minimum Sample Size Single Sided Alternative Hypothesis: 46 Two Sided Alternative Hypothesis: 56 222 ------- 12.4 Sample Sizes for Acceptance Sampling 1. Click DQOs Based Sample Sizes ^- Acceptance Sampling r^ File Edit Navigation F Name Worksheet jds Well Iflxls WMW-with NDsjd ASHALL7groupsa Stats/Sample Sizes Graphs Statistical Tests General Statistics » I Imputed NDs using ROS Methods > rk DQOs Based Sample Sizes > | s s 2 3 4 5 a 71 17 Upper Limits/BTVs 3 9ro D_Site 4 UCLs/EPCs Windows Help 567 Estimate Mean Hypothesis Tests > Acceptance Sampling 2. The following options window is shown. OptionsSampteSizeAcceptance Confidence Coefficient [C Pre-specffied Proportion [P] of non-comforming items/drums 0.05 Number of Allowable non-conforming items/drums I o • Specify the Confidence Coefficient. Default is 0.95. • Specify the Proportion [P] of non-conforming items/drums. Default is 0.05. • Specify the Number of Allowable non-conforming items/drums. Default is 0. • Click on OK button to continue or on Cancel button to cancel the options. Output Screen for Sample Sizes for Acceptance Sampling (Default Options) ! Acceptance Sampling for Pie-specified Proportion of Non-confonwig Items Based on Specified Values of Decision Parameters/DHOs Date/Time of Computation 2/26^2010 12:20:34 PM User Selected Options Confidence Coefficient 0.95 Pre-specified proportion of non-conforming items in the lot 0.05 Number of allowable non-conforming items in the lot 0 Approximate Minimum Sample Size Exact Binomial/Beta Distribution 59 Approximate Chisquare Distribution (Tukey-Scheffe) 59 223 ------- Chapter 13 Analysis of Variance Oneway Analysis of Variance (ANOVA) is a statistical technique that is used to compare the measures of central tendencies: means or medians of more than two populations/groups. Oneway ANOVA is often used to perform inter-well comparisons in groundwater monitoring projects. Classical Oneway ANOVA is a generalization of the two-sample t-test (Hogg and Craig 1995); and nonparametric ANOVA, Kruskal- Wallis test (Hollander and Wolfe 1999) is a generalization of the two- sample Wilcoxon Mann Whitney test. Theoretical details of Oneway ANOVA are given in the ProUCL Technical Guide. Oneway ANOVA is available under the Statistical Tests module of ProUCL 5.0/ProUCL 5.1. It is advised to use these tests on raw data in the original scale without transforming the data (e.g., using a log- transformation) . B^ File Edit Stats/Sample Sizes Graphs Navigation Panel Name Work Sheet jds Well Kbds WMW-with NDsjds ASHALL7graups;------- 2. The Select Variables screen will appear. • Select the variables for testing. • Select a Group variable by using the arrow under the Group Column option. • Click OK to continue or Cancel to cancel the test. Example 13-la. Consider Fisher's (1936) 3 species (groups) Iris flower data set. Fisher collected data on sepal length, sepal width, petal length and petal width for each of the 3 species. Oneway ANOVA results with conclusions for the variable sepal-width (sp-width) are shown as follows: Output for a Classical Oneway ANOVA Date/Time of Computation From Rle Full Precision Classical Oneway ANOVA 3/26/2013 10:45:03 AM FULLIRIS-rdsjds OFF sp -width Group Obs Mean 1 50 3.428 2 50 2.77 3 50 2.974 Grand Statistics (All data) 150 3.057 SD Variance 0379 0144 0.314 0.0985 0322 0.104 0.436 0.19 Classical One-Way Analysis of Variance Table Source SS DOF MS Between Groups 11.34 2 5.872 Within Groups 16.96 147 0.115 Total 28.31 149 Pooled Standard Deviation 0.34 R-Sq 0.401 V.R.(FStat) P-Value 49.16 0 Note: A p-value •:- 0.05 (or some other selected level) suggests that there are significant deferences in mean/median characteristics of the various groups at 0.05 or other selected level of significance A p-value > 0.05 (or other selected level) suggests that mean/median characteristics of the various groups are comparable 225 ------- 13.2 Nonparametric ANOVA Nonparametric Oneway ANOVA or the Kruskal-Wallis (K-W) test is a generalization of the Mann- Whitney two-sample test. This is a nonparametric test and can be used when data from the various groups are not normally distributed. 1. Click Oneway ANOVA ^ Nonparametric a^ File Edit Stats/Sample Sizes Graphs Navigation Panel Name Worksheet :ds Wdl 10jds WHW-with NDsxIs ASHALL7groupsjds Box Plot Full.gst Box Plot Full_a.gst 1 2 3 4 5 6 0 Backgroun H 1 4 7 12 15 Statistical Tests | Upper Limits/BTVs UCLs/EPCs Windows Help Outlier Tests I Goodness-of-Fit Tests > Single Sample Hypothesis > Two Sample Hypothesis ^ Oneway ANOVA ^ OLS Regression Trend Analysis > 5 6 7 Classical Nonparametric Like classical Oneway ANOVA, nonparametric ANOVA also requires that the data file used should follow the data format as shown above; the data file should consist of a group variable defining the various groups to be evaluated using the Oneway ANOVA module. 2. The Select Variables screen will appear. • Select the variables for testing. • Select the Group variable. • Click OK to continue or Cancel to cancel the test. Example 13-lb (continued). Nonparametric Oneway ANOVA results with conclusion for sepal-length (sp-length) are shown as follows. Output for a Nonparametric ANOVA Nonparametric Oneway ANOVA (Kruskal-Wallis Test) Date/Time of Computation 3/26/2013 11:11:32 AM From Be FULLI Full Precision OFF sp4ength Group 1 2 3 Overall K-W (H-Stat) 96.76 96.34 50 50 50 150 5 5.9 6.5 5.8 DOF 2 2 P-Value 0 0 29.64 82.65 114.2 -9142 1.425 7.716 75.5 (Approx. Chisquare) (Adjusted for Ties) Hole: A p-value <= 0.05 (or some other selected level) suggests that there are significant differences in mean/median characteristics of the various groups at 0.05 or other selected level of significance A p-value > 0.05 (or other selected level) suggests that mean/median characteristics of the various groups are comparable- 226 ------- 227 ------- Chapter 14 Ordinary Least Squares of Regression and Trend Analysis The OLS of regression and trend tests are often used to determine trends potentially present in constituent concentrations at polluted sites, especially in GW monitoring applications. The OLS regression and two nonparametric trend tests: Mann-Kendall test and Theil-Sen test are available under the Statistical Tests module of ProUCL 5.0/ProUCL 5.1. The details of these tests can be found in Hollander and Wolfe (1999) and Draper and Smith (1998). Some time series plots, which are useful in comparing trends in analyte concentrations of multiple groups (e.g., monitoring wells), are also available in ProUCL. The two nonparametric trend tests: M-K test and Theil-Sen test are meant to identify trends in time series data (data collected over a certain period of time such as daily, monthly, quarterly, etc.) with distinct values of the time variable (time of sampling events). If multiple observations are collected/reported at a sampling event (time), one or more pairwise slopes used in the computation of the Theil-Sen test may not be computed (become infinite). Therefore, it is suggested that the Theil-Sen test only be used on data sets with one measurement collected at each sampling event. If multiple measurements are collected at a sampling event, the user may want to use the average (or median, mode, minimum or maximum) of those measurements resulting in a time series with one measurement per sampling time event. Theil-Sen test in ProUCL has an option which can be used to average multiple observations reported for the various sampling events. The use of this option also computes M-K test statistic and OLS statistics based upon averages of multiple observations collected at the various sampling events. New in ProUCL 5.1: In addition to slope and intercept of the nonparametric Theil-Sen (T-S) trend line, ProUCL 5.1 computes residuals based upon the T-S trend line. The trend tests in ProUCL software also assume that the user has entered data in chronological order. If the data are not entered properly in chronological order, the graphical trend displays may be meaningless. Trend Analysis and OLS Regression modules handle missing values in both response variable (e.g., analyte concentrations) as well as the sampling event variable (called independent variable in OLS). 14.1 Simple Linear Regression 1. Click Statistical Tests ^ OLS Regression. CL 5.0 - [WMW-with N[ of File Edit Stats/Sample Sizes Graphs Navigation Panel Name Worksheet.^ WeJIDjds WMW™th NDs:xis ASHALL7groupsxls Box Plot Full.gst Box Plot Full_a.gst Rnv Pint Fi ill h net 1 2 3 4 5 6 0 Backgroun 1 4 5 7 12 15 Statistical Tests | Upper Limits/BTVs Outlier Tests ^ Goodness-of-Fit Tests ^ Single Sample Hypothesis ^ Two Sample Hypothesis ^ Oneway ANOVA ^ OLS Regression Trend Analysis > UCLs/EPCs Windows Help 5 6 7 8 9 10 11 228 ------- 2. The Select Regression Variables screen will appear. • Select the Dependent Variable and the Independent Variable for the regression analysis. Available Variables Name ID Time (days)-! 0 BTEXConc. @... 1 Time (days>2 2 BTEXConc. @... 3 Time [days)-3 4 BTEXConc. @... 5 Time (daysH 6 BTEXConc. @... 7 Time (days}-5 8 BTEXConc. @... 9 < > » « Dependent Variable Name ID MW-28 1 1 < > Independent Variable « Option Name ID Time(days)-6 10 < > Select Group Column (Optional) V OK Cancel Select a group variable (if any) by using the arrow below the Select Group Column (Optional). The analysis will be performed separately for each group. When the Options button is clicked, the following options window will appear. Regression Options EH Display Intervals Confidence Level 095 @ Display Regression Table I I Display Diagnostics Graphics Options 0 Display XY Plot XY Plot Title Classical Regression 0 Display Confidence Interval 0 Display Prediction Interval OK Select Display Intervals for the confidence limits and the prediction limits of each observation to be displayed at the specified Confidence Coefficient. The interval estimates will be displayed in the output sheet. 229 ------- o Select Display Regression Table to display Y-hat, residuals and the standardized residuals in the output sheet. o Select "XY Plot" to generate a scatter plot display showing the regression line. o Select Confidence Interval and Prediction Interval to display the confidence and the prediction bands around the regression line. o Click on OK button to continue or on Cancel button to cancel the option. • Click OK to continue or Cancel to cancel the OLS Regression. o The use of the above options will display the following graph on your computer screen which can be copied using the Copy Chart (To Clipboard) in a Microsoft documents (e.g., word document) using the File ^-Paste combination. o The above options will also generate an Excel-Type output sheet. A partial output sheet is shown below following the OLS Regression Graph. Example 14-la. Consider analyte concentrations, X collected from a groundwater (GW) monitoring well, MW-28 over a certain period of time. The objective is to determine if there is any trend in GW concentrations, X of the MW-28. The OLS regression line with inference about slope and intercept are shown in the following figure. The slope and its associated p-value suggest that there is a significant downward trend in GW concentrations of MW-28. OLS Regression Graph without Regression and Prediction Intervals Classical Regression Slope Intercept Je(Reg) je (Slope] 2.163.7246 0.8432 •09183 372.6302 00000 00000 2117 SUndaldizedS 5.0756 Approximate p-vatje O.OOfH 4246 '"--.... Time (days)-6 230 ------- OLS Regression Graph with Regression and Prediction Intervals Classical Regression •1.6372 3,1637245 OB432 P-valtie (Reg) P-value (Slope) Mann-Kendall S SDoJS Confidence Coefficient Red - Prediction Inierva -135.0000 26.4009 •S075S 160 260 360 460 560 660 760 860 Time (days)-6 960 1060 116C Partial Output of OLS Regression Analysis Ordinary Least Squares Linear Regression Output Sheet User Selected Options Date/Time of Computation From File Full Precision Paramater intercept Tme (daysJ-6 3/27/2013 11:51:45 AM Trend-MW-data-use jds OFF Number Reported W-values) 1 8 Dependendant Variable MW-28 Independent Variable Time (dsys}f Regression Estimates and Inference Table Estimates Std. Error T^alues p^/alues 2164 -1.637 165.3 13.09 5.793E-10 0.176 -9.276 7.7292E-8 OLS ANOVA Table Source of Variation SS DOF MS Regression 11952431 1 11952431 Error 2222368 16 138898 Total 14174799 17 R Square 0.843 .Adjusted R Square 0.833 Obs 1 2 3 4 5 6 7 8 9 Y Vector 2880 2117 1633 1845 1706 1719 1065 831.8 920.6 Sqrt(HSE) = Scale 372.7 Regression Table Yhat Residuals Res/Scale 2164 716.3 1.922 2028 B9.17 0.239 1900 -267.6 -0.718 1748 97.13 0.261 1587 118.2 0.317 1307 411.1 1.103 1154 -SB. 55 -0.23B 1009 -177.7 -0.477 1009 -B8.87 -0.238 F-Value 8605 P-Value 0.0000 231 ------- Verifying Normality of Residuals: As shown in the above partial output, ProUCL displays residuals including standardized residuals on the OLS output sheet. Those residuals can be imported (copying and pasting) in an excel file to assess the normality of those OLS residuals. The parametric trend evaluations based upon the OLS slope (significant slope, confidence interval and prediction interval) are valid provided the OLS residuals are normally distributed. Therefore, it is suggested that the user assesses the normality of OLS residuals before drawing trend conclusions using a parametric test based upon the OLS slope estimate. When the assumptions are not met, one can use graphical displays and nonparametric trend tests (e.g., T-S test) to determine potential trends present in a time series data set. 14.2 Mann-Kendall Test 1. Click Statistical Tests ^ Trend Analysis ^ Mann-Kendall. ^L b.U - llrend-MW-data-uiapi4.xlsJ Statistical Tests I Upper Limits/ETVs UCLs/EPCs Windows Help Outlier Tests G o o d n ess- of - Fit Tests Single Sample Hypothesis Two Sample Hypothesis Oneway ANOVA OLS Regression Trend Analysis 8 1C 11 12 Mann-Kendall Theil-Sen Time Series Plot > 2. The Select Trend Event Variables screen will appear. Available Variables Name ID Selected Variable Values/Measured Data Name MW-2S ID 1 Optional EventfTime Not Required - Index data will be generated for graphics Select Group Column (Optional) 232 ------- Select the Event/Time variable. This variable is optional to perform the Mann-Kendall (M-K) Test; however, for graphical display it is suggested to provide a valid Event/Time variable (numerical values only). If the user wants to generate a graphical display without providing an Event/Time variable, ProUCL generates an index variable to represent sampling events. Select the Values/Measured Data variable to perform the trend test. Select a group variable (if any) by using the arrow below the Select Group Column (Optional). When a group variable is chosen, the analysis is performed separately for each group represented by the group variable. When the Options button is clicked, the following window will be shown. Select Mann Kendall Options Confidence Level 0.95 Graphics Options 0 Display Graphics 0 Display Theil-Sen Trend Line 0 Display OLS Regression Line Title for Graph Mann-Kendall Trend Test o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Select the trend lines to be displayed: OLS Regression Line and/or Theil-Sen Trend Line. If only Display Graphics is chosen, a time series plot will be generated. o Click on OK button to continue or on Cancel button to cancel the option. • Click OK to continue or Cancel to cancel the Mann-Kendall test. 233 ------- 14- Ib (Continued). The M-K test results are shown in the following figure and in the following M-K test output sheet. Based upon the M-K test, it is concluded that there is a statistically significant downward trend in GW concentrations of the MW-28. Mann Kendall Test Trend Graph displaying all Selected Options Mann-Kendall Trend Test O.OMMI SiwMtad Deviate rt S 26 4MB dVaijfrtS -SOTS Mann-Kendall Trend Test Output Sheet Mann-Kendall Trend Test Analysis User Selected Options Date/Time of Computation 3/27/7013 12:19:26 PM From File Tnend-MW-data-Chap14jds Full Precision OFF Confidence Coefficient 0.95 Level of Significance 0.05 MW-28 General Statistics Number of Events Number Values Reported In} Minimum Maximum Mean Geometric Mean Median Standard Deviation Mam-Kendall Test Test Value ------- 14.3 Theil - Sen Test To perform the Theil-Sen test, the user is required to provide numerical values for a sampling event variable (numerical values only) as well as values of a characteristic (e.g., analyte concentrations) of interest observed at those sampling events. 1. Click Statistical Tests ^ Trend Analysis ^ Theil-Sen. aJ File Edit Stats/Sample Sizes Graphs Navigation Panel Name Worksheet xis Well 10xls WMW^ith NDsjds ASHALL7groups*is Box Plot Full.gst Box Plot Full_a.gsl Box Plot Full_b.gst 1 2 3 4 5 G 7 8 0 Backgroun r~ 4 5 7 12 15 18 Statistical Tests Upper Limto/BTVs UCLs/EPCs Windows Help Outlier Tests t Goodness-of-Fit Tests t Single Sample Hypothesis > Two Sample Hypothesis > Oneway ANOVA t OLS Regression Trend Analysis ^ 34 1 21 35 1 1 1 5 6 7 Mann-Kendall Theil-Sen Time Series Plot » 2. The Select Variables screen will appear. Available Variables Name ID Selected Event/Time Event/Time Data Name Time (daysX ID D Selected Variable Values/Measured Data Name MW-28 ID 1 Select Group Column (Optional) Select an Event/Time Data variable. Select the Values/Measured Data variable to perform the test. Select a group variable (if any) by using the arrow below the Select Group Column (Optional). When a group variable is chosen, the analysis is performed separately for each group represented by the group variable. When the Options button is clicked, the following window will be shown. 235 ------- Confidence Level Graphics Options 0 Display OLS Regression Line 0 Display Theil-Sen Trend Line Title for Graph Theil-Sen Trend Line and Ol o Specify the Confidence Level; a number in the interval (0.5, 1), 0.5 inclusive. The default choice is 0.95. o Select the trend lines to be displayed: OLS Regression Line and/or Theil-Sen Trend Line. o Click on OK button to continue or on Cancel button to cancel the option. • Click OK to continue or Cancel to cancel the Theil-Sen Test. 14-lc (continued). The Theil-Sen test results are shown in the following figure and in the following Theil-Sen test Output Sheet. It is concluded that there is a statistically significant downward trend in GW concentrations of MW-28. Theil-Sen test results and residuals are summarized in tables following the trend graph shown below. Theil-Sen Test Trend Graph displaying all Selected Options Theil-Sen Trend Line and OLS Regression Line Theil Sen T------- Theil-Sen Trend Test Output Sheet Date/Time of Computation 3/27/20132:19:55 PM From Rle Trend-MW-data-Chap14jds Full Precision OFF Confidence Coefficient 0.95 Level of Significance 0.05 MW-28 General Statistics Number of Events Number Values Reported In} Minimum Maximum Mean Geometric Mean Median Standard Deviation 18 18 1.7 2880 8-64.6 174.8 628.2 913.1 Approximate inference for Theil-Sen Trend Test Mann-Kendall Statistic (S) -137 Standard Deviation of S 26.4 Standardized Value of S -5.151 Approximate p-value 1.2930E-7 Number of Slopes 153 Theil-Sen Slope -1.705 Theil-Sen Intercept 1917 M2 98.21 One-sided 95% upper limit of Slope -1.365 95% LCL of Slope (0.025) -2.222 95% UCL of Slope (0.975) -1.268 Statistically significant evidence of a decreasing trend at the specified level of significance. Then-Sen Trend Test Estimates and Residuals Notes: As with other statistical test statistics, trend test statistics: M-K test statistic, OLS regression and Theil-Sen slopes may lead to different trend conclusions, hi such instances it is suggested that the user supplements statistical conclusions with graphical displays. Averaging of Multiple Measurements at Sampling Events: hi practice, when multiple observations are collected/reported at one or more sampling events (times), one or more pairwise slopes may become infinite, resulting in a failure to compute the Theil-Sen test statistic. In such cases, the user may want to 237 ------- pre-process the data before using the Theil-Sen test. Specifically, to assure that only one measurement is available at each sampling event, the user pre-processes the time series data by computing average, median, mode, minimum, or maximum of the multiple observations collected at those sampling events. The Theil-Sen test in ProUCL 5.0/ProUCL 5.1 provides the option of averaging multiple measurements collected at the various sampling events. This option also computes M-K test and OLS regression statistics using the averages of multiple measurements collected at the various sampling event. The OLS regression and M-K test can be performed on data sets with multiple measurements taken at the various sampling time events. However, often it is desirable to use the averages (or median) of measurements taken at the various sampling events to determine potential trends present in a time-series data set. 14-lc (continued). The data set used in Example 14-lc has some sampling events where multiple observations were taken. Theil-Sen test results based upon averages of multiple observations is shown as follows. The data set is included in the ProUCL Data directory which comes with ProUCL 5.1. Theil-Sen Test Trend Graph displaying all Selected Options Multiple Observations Taken at Some Sampling Events Have Been Averaged Theil-Sen Trend Line and OLS Regression Line T heil-S en T lend Analysis Level ol Significance OLS Regression Line (Blue] OLSRegiessionSlope OLS Regression Intercept T heil-S en T lend Line (Red) Theil Sen Slope Theil-Sen Intercept -1.8539 2,183.2196 Ml 32.6972 M2 72.3028 LCLol9lope 2.3073 UCLofSlope -1.3503 Statistically significant evidence oi trend at the specified level ol sigrafcance. Time (days)-6 14.4 Time Series Plots This option of the Trend Analysis module can be used to determine and compare trends in multiple groups over the same period of time. This option is specifically useful when the user wants to compare the concentrations of multiple groups (wells) and the exact sampling event dates are not be available (data only option). The user may just want to graphically compare the time-series data collected from multiple groups/wells during several quarters (every year, every 5 year, etc.). When the user wants to use this module using the data/event option, each group (e.g., well) defined by a group variable must have the same number of observations and should share the same sampling event values. That is the number of sampling events and values (e.g., quarter ID, year ID, etc.) for each group (well) must be the same for this option to work. However, the 238 ------- exact sampling dates (not needed to use this option) in the various quarters (years) do not have to be the same as long as the values of the sampling quarters/years (1,3,5,6,7,9,..) used in generating time-series plots for the various groups (wells) match. Using the geological and hydrological information, this kind of comparison may help the project team in identifying non-compliance wells (e.g., with upward trends in constituent concentrations) and associated reasons. 1. Click Statistical Tests ^ Trend Analysis ^ Time Series Plots /IW89-C jraphs 2 MW-ID Statistical Tests | Upper Limrts/BTVs UCLs/EPCs Windows Help Outlier Tests > Goodness-of-Fit Tests > Single Sample Hypothesis > Two Sample Hypothesis t Oneway ANOVA > OLS Regression Trend Analysis > 1 460 8 1 547 & 1 605 8 6 f-Mn-89 4600 2760 1270 1860 OgD 7 Mann-Kenda Thai-Sen 8 MW9 9 9 9 9 i Time Series Plot > 1610 < H jnn i 9 MN9 2200 2340 2340 2420 2150 10 11 MN-99 2200 2340 2340 ' 1 2420 215C 2220 2220 2050 2050 Data Only 2060 Event/Data 1770 2. When the Data Only option is clicked, the following window is shown: : Trend Data Variable Selected Variable Values/Measured Data Mn-GW MW-ID Manganese MW-E9 GW-Mn-B9 MW9 MN9 MN-99 Name ID Select Group Column (Optional) This option is used on the measured data only. The user selects a variable with measured values which are used in generating a time series plot. The time series plot option is specifically useful when data come from multiple groups (monitoring wells during the same period of time). Select a group variable (is any) by using the arrow shown below the Group Column (Optional). 239 ------- * Select Trend Data Variable - n Available Variables Name ID Well ID 0 MW-ID 2 Manganese 3 MW-B9 5 GW-Mn-89 6 MW9 8 MN9 9 MN-99 11 index 14 Selected Variable Values/Measured Data » « < Options Name ID Mn-GW 1 felect Group Column (Optional) OK Cancel When the Options button is clicked, the following window will be shown. Confidence Coefficient 0.95 Set Event/Index Label Set Initial Start Value Event Plot Graphs Together 0 Group Graphs Must select a Group Column All Groups the Same Size! 0 Display Thei I -Sen Trend Line Minimal Theil-Sen Stats Provided D Display OLS Regression Line 1 Event/Index Set Event/Index Increments 1 Greater Than Zero [0] Title for Graph Time-Series Trend Analysis The user can opt to display graphs for each group individually or for all groups together on the same graph by selecting the Group Graphs option. The user can also display the OLS line and/or the Theil-Sen line for all groups displayed on the same graph. The user may pick an initial starting value and an increment value to display the measured data. All statistics will be computed using the data displayed on the graphs (e.g., selected Event values). o Input a starting value for the index of the plot using the Set Initial Start Value. o Input the increment steps for the index of the plot using the Set Index/Event Increments. o Specify the lines (Regression and/or Theil-Sen) to be displayed on the time series plot. o Select Plot Graphs Together option for comparing the time series trends for more than one group on the same graph. 240 ------- If this option is not selected but a Group Variable is selected, different graphs will be plotted for each group. Click on OK button to continue or on Cancel button to cancel the Time Series Plot. 3. When the Event/Data option is clicked, the following window is shown: Available Variables Name Well ID MVi'-ID Manganese MW-S9 GW-Mn-39 MW9 HNS MN-99 ID 0 2 3 5 6 S 5 11 Opt Selected Event/Time Event/Time Data Name index ID 14 Selected Variable Values/Measured Data Name Mn-GW ID 1 Select Group Column (Optional) Select a group variable (is any) by using the arrow shown below the Group Column (Optional). This option uses both the Measured Data and the Event/Time Data. The user selects two variables; one representing the Event/Time variable and the other representing the Measured Data values which will be used in generating a time series plot. When the Options button is clicked, the following window will be shown. elect Time Series fidence Coefficient 0.95 Display OLS Regression Line Display Thei l-Sen Trend Line Plot Graphs Together EH Group Graphs Must select a Group Column All Groups the Same Size! Title for Graph Time-Series Trend Analysis 241 ------- The user can select to display graphs individually or together for all groups on the same graph by selecting the Plot Graphs Together option. The user can also display the OLS line and/or the Theil-Sen line for all groups displayed on the same graph. o Specify the lines (Regression and/or Theil-Sen) to be displayed on the time series plot. o Select Plot Graphs Together option for comparing time series trends for more than one group on the same graph. o If this option is not selected but a Group Variable is selected, different graphs will be plotted for each group. o Click on OK button to continue or on Cancel button to cancel the options. • Click OK to continue or Cancel to cancel the Time Series Plot. Notes: To use this option, each group (e.g., well) defined by a group variable must have the same number of observations and should share the same sampling event values (if available). That is the sampling events (e.g., quarter ID, year ID, etc.) for each group (well) must be the same for this option to work. Specifically, the exact sampling dates within the various quarters (years) do not have to be the same as long as the sampling quarters (years) for the various wells match. Example 14-2. The following graph has three (3) time series plots comparing manganese concentrations of the three GW monitoring wells (1 upgradient well [MW1] and 2 downgradient wells [MW8 and MW9]) over the period of 4 years (data collected quarterly). Some trend statistics are displayed in the side panel. Output for a Time Series Plot - Event/Data Option by a Group Variable (1, 8, and 9) Time-Series Trend Analysis 496 478 508 469 475 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 11.0 12.0 13.0 14.0 15.0 16.0 17.0 index Tiroe-SeriesTiend Analysis ttEvents/Time Periods OLS Regression Line [Bkjel Theil-SenTlend Line [Red) 1 LSRegiession Slope -6.9618 LSReglessionlntercept 561.5500 heil-SenSlope -7.8333 heilSenlntetcept 568.5333 LS Regression Slope -42.8371 LSRegiessionlnlocept 2.362.7500 heil-SenSlope -4.7619 heil-Senlnt<*cept 1.790.4762 LS Regression Slope 40.8382 LSRegiessionlnleicept 2.315.2500 heil SenSlope -72.5000 heil-SenlnleicepI 2.671.2500 242 ------- 243 ------- Chapter 15 Background Incremental Sample Simulator (BISS) Simulating BISS Data from a Large Discrete Background Data The Background Incremental Sample Simulator (BISS) module was incorporated in ProUCLS.O at the request of the Office of Superfund Remediation and Technology Innovation (OSRTI). However, this module is currently under further investigation and research, and therefore it is not available for general public use. This module has been retained in ProUCL 5.1. This module may be released in a future version of the ProUCL software, along with strict conditions and guidance for how it is applied. The main text for this chapter is not included in this document for release to general public. Only a brief placeholder write-up is provided here. The following scenario describes the Site or project conditions under which the BISS module could be useful: Suppose there is a long history of soil sample collection at a Site. In addition to having a large amount of Site data, a robust background data set (at least 30 samples from verified background locations) has also been collected. Comparison of background data to on-Site data has been, and will continue to be, an important part of this project's decision-making strategy. All historical data is from discrete samples, including the background data. There is now a desire to switch to incremental sampling for the Site. However, guidance for incremental sampling makes it clear that it is inappropriate to compare discrete sample results to incremental sample results. That includes comparing a Site's incremental results directly to discrete background results. One option is to recollect all background data in the form of incremental samples from background DUs that are designed to match Site DUs in geology, area, depth, target soil particle size, number of increments, increment sample support, etc. If project decision-making uses a BTV strategy to compare Site DU results one at a time against background, then an appropriate number (the default is no less than 10) of background DU incremental samples would need to be collected to determine the BTV for the population of background DUs. However, if the existing discrete background data show background concentrations to be low (in comparison to Site concentrations) and fairly consistent (relative standard deviation, RSD <1), there is a second option described as follows. When a robust discrete background data set that meets the above conditions already exists, the following is an alternative to automatically recollecting ALL background data as incremental samples. Step 1. Identify 3 background DUs and collect at least 1 incremental sample from each for a minimum of 3 background incremental samples. Step 2. Enter the discrete background data set (n > 30) and the >3 background incremental samples into the BISS module (the BISS module will not run unless both data sets are entered). • The BISS module will generate a specified (default is 7) simulated incremental samples from the discrete data set. • The module will then run a t-test to compare the simulated background incremental data set (e.g., with n = 7) to the actual background incremental data set (n > 3). 244 ------- o If the t-test finds no difference between the 2 data sets, the BISS module will combine the 2 data sets and determine the statistical distribution, mean, standard deviation, potential UCLs and potential BTVs for the combined data set. Only this information will be supplied to the general user. The individual values of the simulated incremental samples will not be provided. o If the t-test finds a difference between the actual and simulated data sets, the BISS module will not combine the data sets nor provide a BTV. o In both cases, the BISS module will report summary statistics for the actual and simulated data sets. Step 3. If the BISS module reported out statistical analyses from the combined data set, select the BTV to use with Site DU incremental sample results. Document the procedure used to generate the BTV in project reports. If the BISS module reported that the simulated and actual data sets were different, the historical discrete data set cannot be used to simulate incremental results. Additional background DU incremental samples will need to be collected to obtain a background DU incremental data set with the number of results appropriate for the intended use of the background data set. The objective of the BISS module is to take advantage of the information provided by the existing background discrete samples. The availability of a large discrete data set collected from the background areas with geological formations and conditions comparable to the Site DU(s) of interest is a requirement for successful application of this module. There are fundamental differences between incremental and discrete samples. For example, the sample supports of discrete and incremental samples are very different. Sample support has a profound effect on sample results so samples with different sample supports should not be compared directly, or compared with great caution. Since incremental sampling is a relatively new approach, the performance of the BISS module requires further investigation. If you would like to try this strategy for your project, or if you have questions, contact Deana Crumbling, crumbling.deana(g),epa.gov. 245 ------- Chapter 16 Windows The Windows Menu performs typical Windows program options. File Edit State/Sample Sizes BIS Simulator Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs 3UCL 5.0 Navigation Panel Name Click on the Window menu to reveal the drop-down options shown above. The following Window drop-down menu options are available: • Cascade option: arranges windows in a cascade format. This is similar to a typical Windows program option. • Tile option: resizes each window vertically or horizontally and then displays all open windows. This is similar to atypical Windows program option. • The drop-down options list also includes a list of all open windows with a check mark in front of the active window. Click on any of the windows listed to make that window active. This is especially useful if you have many windows (e.g., >40) open; the navigation panel only holds the first 40 windows. 246 ------- Chapter 17 Handling the Output Screens and Graphs 17.1 Copying and Saving Graphs Graphs can be copied into Word, Excel, or PowerPoint files in two ways. 1. Click the Copy Chart (To Clipboard) shown below; a graph must be present to be copied to the clipboard. File ^ Copy Chart (To Clipboard) File Edit Stats/Sample Sizes BIS Simulator Graphs Statistical Tests New Open Single File Sheet Open Excel File with Multiple Sheets Close Save Save As... Save Graph Copy Chart (To Clipboard) Page Setup Print Print Preview Exit Once the user has clicked Copy Chart (To Clipboard), the graph is ready to be imported (pasted) into most Microsoft office applications (e.g., Word, Excel, and PowerPoint) by clicking the Edit ^- Paste option in those Microsoft applications as shown below. TheJI-Sen Trend Line and OLS Regression Line 247 ------- 2. Graphs can be saved using the Save Graph Option in the Navigation Panel as a Bitmap file with .bmp extension. The user can import the saved bitmap file into a desired document such as a word document or a PowerPoint presentation by using the Copy and Paste options available in the selected Microsoft application. File ^- Save Graph •ff N <:>, w w A Br Bf Bi '.S Tl Tr File Edit Stats/Sample Sizes BIS Simulator Graphs Statistical Tests Upf New Open Single File Sheet Open Excel File with Multiple Sheets Close Save Save As.., Save Graph Copy Chart (To Clipboard) Page Setup Print Print Preview Exit 17.2 Printing Graphs 1. Click the graph you want to print in the Navigation Panel. Zn-Cu-ND-data-chapter 9- Box Plot wNDs gsi Box Plot wNDs_agst Q-Q Plot for Cu Theoretical Quantiles (Standard Normal) Mean-4815 Sd-4.E3S Elope -4.045 Intercept -4.815 basin tiou^i tt-sa Mean-5.49 Sd-4.92S Elope - 4.534 Ititeicept = 5.49 Correlation fi - 0 90 248 ------- Click File ^- Page Setup. New Open Single File Sheet Open Excel File with Multiple Sheets Close Save Save As.,. Save Graph Copy Chart (To Clipboard) Page Setup Print Print Preview Check the button next to Portrait or Landscape (shown below), and click OK. In some cases, with larger headings and captions, it may be desirable to use the Landscape printing option. Paper Letter Auto Orientation '•' Portrait O Landscape Margins indies) Left: D.5 Right: |0.5 Top: J0.5 Bottom: |o.5 Click File ^ Print to print the graph, and File before printing. Print Preview to preview (optional) the graph File Edit Stats/Sample Sizes BIS Simulator Graphs M '!', VI Vv A Bi a B< Vv Tl A New Open Single File Sheet Open Excel File with Multiple Sheets Close Save Save As... Save Graph Copy Chart (To Clipboard) Page Setup Print Print Preview Exit i 249 ------- 17.3 Making Changes in Graphs using Tools and Properties ProUCL uses a couple of development tools such as FarPoint spread (for Excel type input and output operations) and ChartFx (for graphical displays). ProUCL generates box plots using the built-in box plot feature in ChartFx. The programmer has no control over computing various statistics (e.g., Ql, Q2, Q3. IQR) using ChartFx. So box plots generated by ProUCL can differ slightly from box plots generated by other programs (e.g., Excel). Box plots generated using ChartFx round values to the nearest integer. For increased precision of graphical displays (all graphical displays generated by ProUCL), the user can use the process described as follows. Position your mouse cursor on the graph and right-click, a popup menu will appear. Position the mouse on Properties and right-click; a windows form labeled Properties will appear. There are three choices at the top; General, Series and Y-Axis. Position the mouse cursor over the Y-Axis choice and left-click. You can change the number of decimals to increase the precision, change the step to increase or decrease the number Y-Axis values displayed and/or change the direction of the label. To show values on the plot itself, position your mouse cursor on the graph and right-click; a popup menu will appear. Position the mouse on Point Labels and right-click. There are other options available in this popup menu including changing font sizes. 17.4 Printing Non-graphical Outputs 1. Click/Highlight the output you want to save or print in the Navigation Panel. File Edit BIS Simulator Graphs Statistical Tests Upper Limits/BTVs UCLs/EPCs Windows Help Navigation Panel Name Worksheet xis Well 10xls WMW^th NDsxis ASHALL7groLjpsxis Box Plot Full.gst Box Plot Full_a.gst Box Plot Full_b.gst WorkSheet_axls Theil-Senxls Trend Test.gst WorkSheet_bxis 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 A B C D E F G H Thei I Sen Trend Test Analysis User Selected Options D atsTi me of Computation 8C7Q01 3 9:23:00 PM From File Well 10.xls Full Precision OFF Average Replicates Replicates at sampling events will be averaged! Confidence Coefficient 0.95 Level ofSignificance 0.05 U(mg/L) General Statistics NumberofEvents 8 Number Values Observations Number Values Missing Number of Values Reported In) Number of Values After Averaging Number of Replicates Minimum Maximum Mean Geometric Mean Median Standard Deviation Mann-Kendall Statistics TestValue(S) Tabulated p- value Standard Deviation of S Standa rdized Value of S 15 7 8 8 0 0.584 1.78 0.853 0.799 0.724 O39 12 0.089 8.083 1.361 250 ------- 2. Click File ^- Print or File ^- Print Preview if you wish to see the preview before printing. File Edit BIS Simulator Graphs Statisti New Open Single File Sheet Open Excel File with Multiple Sheets Close Save Save As... Print Print Preview Exit 17.5 Saving Output Screens as Excel Files ProUCL 5.0/ProUCL 5.1 saves output files and data files as Excel files with .xls or .xlsx extensions. 1. Click on the output you want to save in the Navigation Panel List. 2. Click File ^- Save or File ^- Save As Edit BIS Simulator Graphs Statist! V', New Open Single File Sheet Open Excel File with Multiple Sheets Close Save Save As.., Print Print Preview Exit Enter the desired file name you want to use, and click Save, and save the file in the desired folder using your browser as shown below. 251 ------- ' © - t M Save As ^ Anita Singh ^ Deslctop ^ sprfnd2012 > Danang-AFB-33 v C Search Danang-AFB-33 p Organize1* Newfolder |EE •* '' -i Favorites E Desktop 4!- Downloads ..'_ Recent places W. Desktop ^ Libraries *^ Homegroup f-^ Anita Singh £ Anita Singh 3^' Computer Qfi Network Filename: Save as type: * Hide Folders A Name Date modified Type @3 Trend Results-by-source-treat-Averaged Data 6/19/2013 5:32 PM Microsoft Office E... S] Trend Results- by-source- Averaged Data 6/19/2013 5:30 PM Microsoft Office E,,, »13 Trend Results-Averaged Data 6/19/2013 3:56 PM Microsoft Office E... Gamma-UCLs v Excel Files (.xlsx) v Save Cancel 252 ------- Chapter 18 Summary and Recommendations to Compute a 95% UCL for Full Uncensored and Left-Censored Data Sets with NDs This chapter briefly summarizes recommendations and the process to compute upper confidence limits of the population mean based upon data sets with and without ND observations. The recommendations are made based upon the simulation studies summarized in Singh, Singh, and Engelhardt (1997, 1999); Singh, Singh, and laci (2002); Singh and Singh (2003); and Singh, Maichle, and Lee (2006). Some details can be found in Chapters 2 and 4 of the associated ProUCL 5.1 Technical Guide. Depending upon the data size, data distribution (e.g., normal, gamma, lognormal, and nonparametric), and data skewness, ProUCL suggests using one or more 95% UCL to estimate the population mean. The project team collectively should determine which of the suggested UCLs will be most appropriate for their project. If needed, the user may want to consult a statistician for additional insight. 18.1 Computing UCL95s of the Mean Based Upon Uncensored Full Data Sets • Formal GOF tests and GOF Q-Q plots are used first to determine the data distribution so that appropriate parametric or nonparametric UCL95s can be computed. • For a normally or approximately normally distributed data set, the user is advised to use Student's t-distribution-based UCL of the mean. Student's t UCL or modified-t-statistic based UCL can be used to estimate the EPC when the data set is symmetric (e.g., skewness = is smaller than 0.2- 0.3) or mildly skewed; that is, when a or a is less than 0.5. In practice, for mildly skewed data sets (with sd of logged data <0.5), all parametric UCLs computation methods available in ProUCL tend to yield comparable results. For gamma or approximately gamma distributed data sets, the user is advised to: 1) use the approximate gamma UCL when k>l and n >50; 2) use the adjusted gamma UCL when k>\ and «<50; 3) use the bootstrap-t method or Hall's bootstrap method when k < 1 and the sample size, n < 15-20; 4) use approximate gamma UCL for k < 1 and sample size, n > 50; and 5) use the adjusted gamma UCL (if available) for k < 1 and sample size, 50> n > 15. If the adjusted gamma UCL is not available (e.g., when an unusual CC level such as 0.935 is selected), then use the approximate gamma UCL as an estimate of the EPC. When the bootstrap-t method or Hall's bootstrap method yields an erratic inflated UCL (e.g., when outliers are present) result, the UCL may be computed using the adjusted gamma UCL (if available) or the approximate gamma UCL. For lognormally distributed data sets, ProUCL recommends a UCL computation method based upon the sample size, n, and standard deviation of the log-transformed data, a. These suggestions are summarized in Table 2-10 of the ProUCL 5.1 Technical Guide. For nonparametric data sets, which are not normally, lognormally, or gamma distributed, a nonparametric UCL is used to estimate the EPC. Methods used to estimate EPC terms based upon nonparametric data sets are summarized in Table 2-11 of the ProUCL 5.1 Technical Guide. For example for mildly skewed nonparametric data sets of smaller sizes (e.g., <30), one may use a modified-t UCL or BCA bootstrap UCL; and for larger samples one may use a CLT-UCL, 253 ------- adjusted-CLT UCL, or a BCA bootstrap UCL. These nonparametric UCLs computation methods do not provide desired coverage to the mean for moderately skewed to highly skewed data sets. • For moderately skewed to highly skewed nonparametric data sets, the use of a Chebyshev (Mean, Sd) UCL is suggested. It is noted that for extremely skewed data sets (e.g., with a exceeding 3.0), even a Chebyshev inequality-based 99% UCL of the mean fails to provide the desired coverage (e.g., 0.95) of the population mean. • For highly skewed data sets with a exceeding 3.0, 3.5, pre-processing the data is suggested. It is very likely that the data consist of outliers and/or come from multiple populations. The population partitioning methods may be used to identify mixture populations present in the data set. For defensible conclusions, the decision statistics such as EPC terms may be computed separately for each of the identified sub-population present in the mixture data set. 18.2 Computing UCLs Based Upon Left-Censored Data Sets with Nondetects The parametric maximum likelihood estimation (MLE) methods (e.g., Cohen 1991) and expectation maximization (EM) method (Gleit 1985) assume normality or lognormality of data sets and tend to work only when the data set has NDs with only one detection limit. These days, due to modern analytical tools and equipment, an environmental data sets consists of NDs with multiple detection limits. Since it is not easy to verify (perform goodness-of-fit) the distribution of a left-censored data set consisting of detects and NDs with multiple detection limits, some poor performing estimation methods including the parametric MLE and EM methods and the winsorization method are not retained in ProUCL 5.0/ProUCL 5.1. In ProUCL, emphasis is given to the use of nonparametric UCL computation methods and hybrid parametric methods based upon KM estimates which account for data skewness in the computation of UCL95. Avoid the use of transformations (to achieve symmetry) while computing upper limits based upon left-censored data sets. It is not easy to correctly interpret the statistics computed in the transformed scale. Moreover, the results and statistics computed in the original scale do not suffer from transformation bias. Like full uncensored data sets, when the standard deviation of the log-transformed data becomes >1.0, avoid the use of a lognormal model even when the data appear to be lognormally distributed. Its use often results in unrealistic statistics of no practical merit (Singh, Singh, and Engelhard 1997; Singh, Singh, and laci 2002). It is also recommended to identify potential outliers representing observations coming from population(s) different from the main dominant population and investigate them separately. Decisions about the disposition of outliers should be made by all interested members of the project team. • It is recommended to avoid the use of the DL/2 (t) UCL method, as the DL/2 UCL does not provide the desired coverage (for any distribution and sample size) for the population mean, even for censoring levels as low as 10%, 15%. This is contrary to the conjecture and assertion (e.g., EPA 2006a) made that the DL/2 method can be used for lower (e.g., < 20%) censoring levels. The coverage provided by the DL/2 (t) method deteriorates fast as the censoring intensity increases. The DL/2 (t) method is not recommended by the authors or developers of this text and ProUCL software. • The use of the KM estimation method is a preferred method as it can handle multiple detection limits. Therefore, the use of KM estimates is suggested to compute the decision statistics based upon methods which adjust for data skewness. Depending upon the data set size, distribution of the detected data, and data skewness, the various nonparametric and hybrid KM UCL95 methods including KM (BCA), bootstrap-t KM UCL, Chebyshev KM 254 ------- UCL, Gamma-KM UCL based upon the KM estimates provide good coverages for the population mean. All of these methods are available in ProUCL 5.1. 255 ------- REFERENCES Aitchison, J. and Brown, J.A.C. 1969. The Lognormal Distribution, Cambridge: Cambridge University Press. Anderson, T.W. and Darling, D. A. 1954. Test of goodness-of-fit. Journal of American Statistical Association, Vol. 49, 765-769. Bain, L.J., and Engelhardt, M. 1991. Statistical Analysis of Reliability and Life Testing Models, Theory and Methods. 2nd Edition. Dekker, New York. Bain, L.J. and Engelhardt, M. 1992. Introduction to probability and Mathematical Statistics. Second Edition. Duxbury Press, California. Barber, S. and Jennison, C. 1999. Symmetric Tests and Confidence Intervals for Survival Probabilities and Quantiles of Censored Survival Data. University of Bath, Bath, BA2 7AY, UK. Barnett, V. 1976. Convenient Probability Plotting Positions for the Normal Distribution. Appl. Statist., 25, No. 1, pp. 47-50, 1976. Barnett, V. and Lewis T. 1994. Outliers in Statistical Data. Third edition. John Wiley & Sons Ltd. UK. Bechtel Jacobs Company, LLC. 2000. Improved Methods for Calculating Concentrations used in Exposure Assessment. Prepared for DOE. Report # BJC/OR-416. Best, D.J. and Roberts, D.E. 1975. The Percentage Points of the Chi-square Distribution. Applied Statistics, 24: 385-388. Best, D.J. 1983. A note on gamma variate generators with shape parameters less than unity. Computing, 30(2): 185-188, 1983. Blackwood, L. G. 1991. Assurance Levels of Standard Sample Size Formulas, Environmental Science and Technology, Vol. 25, No. 8, pp. 1366-1367. Blom, G. 1958. Statistical Estimates and Transformed Beta Variables. John Wiley and Sons, New York. Bowman, K. O. and Shenton, L.R. 1988. Properties of Estimators for the Gamma Distribution, Volume 89. Marcel Dekker, Inc., New York. Bradu, D. and Mundlak, Y. 1970. Estimation in Lognormal Linear Models. Journal of the American Statistical Association, 65, 198-211. Chen, L. 1995. Testing the Mean of Skewed Distributions. Journal of the American Statistical Association, 90, 767-772. Choi, S. C. and Wette, R. 1969. Maximum Likelihood Estimation of the Parameters of the Gamma Distribution and Their Bias. Technometrics, Vol. 11, 683-690. 256 ------- Cochran, W. 1977. Sampling Techniques, New York: John Wiley. Cohen, A. C., Jr. 1950. Estimating the Mean and Variance of Normal Populations from Singly Truncated and Double Truncated Samples. Ann. Math. Statist., Vol. 21, pp. 557-569. Cohen, A. C., Jr. 1959. Simplified Estimators for the Normal Distribution When Samples Are Singly Censored or Truncated. Technometrics, Vol. 1, No. 3, pp. 217-237. Cohen, A. C., Jr. 1991. Truncated and Censored Samples. 119, Marcel Dekker Inc. New York, NY 1991. Conover W.J.. 1999. Practical Nonparametric Statistics, 3rd Edition, John Wiley & Sons, New York. D'Agostino, R.B. and Stephens, M.A. 1986. Goodness-of-Fit Techniques. Marcel Dekker, Inc. Daniel, Wayne W. 1995. Biostatistics. 6th Edition. John Wiley & Sons, New York. David, H.A. and Nagaraja, H.N. 2003. Order Statistics. Third Edition. John Wiley. Department of Navy. 2002a. Guidance for Environmental Background Analysis. Volume 1 Soil. Naval Facilities Engineering Command. April 2002. Department of Navy. 2002b. Guidance for Environmental Background Analysis. Volume 2 Sediment. Naval Facilities Engineering Command. May 2002. Dixon, W.J. 1953. Processing Data for Outliers. Biometrics 9: 74-89. Draper, N.R. and Smith, H. 1998. Applied Regression Analysis (3rd Edition). New York: John Wiley & Sons. Dudewicz, E.D. and Misra, S.N. 1988. Modern Mathematical Statistics. John Wiley, New York. Efron, B. 1981. Censored Data and Bootstrap. Journal of American Statistical Association, Vol. 76, pp. 312-319. Efron, B. 1982. The Jackknife, the Bootstrap, and Other Resampling Plans, Philadelphia: SIAM. Efron, B. and Tibshirani, R.J. \993.An Introduction to the Bootstrap. Chapman & Hall, New York. El-Shaarawi, A.H. 1989. Inferences about the Mean from Censored Water Quality Data. Water Resources Research, 25, pp. 685-690. Fisher, R. A. 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics 7 (2): 179-188. Fleischhauer, H. and Korte, N. 1990. Formation of Cleanup Standards Trace Elements with Probability Plot. Environmental Management, Vol. 14, No. 1. 95-105. Gehan, E.A. 1965. A Generalized Wilcoxon Test for Comparing Arbitrarily Singly-Censored Sample. Biometrika 52, 203-223. 257 ------- Gerlach, R. W., and J. M. Nocerino. 2003. Guidance for Obtaining Representative Laboratory Analytical Subsamples from Particulate Laboratory Samples. EPA/600/R-03/027. www. epa. gov/esd/tsc/image s/particulate .pdf. Gibbons. 1994. StatisticalMethods for Groundwater'Monitoring. John Wiley &Sons. Gilbert, R.O. 1987. Statistical Methods for Environmental Pollution Monitoring. VanNostrand Reinhold, New York. Gilliespie, B.W., Chen, Q., Reichert H., Franzblau A., Hedgeman E., Lepkowski J., Adriaens P., Demond A., Luksemburg W., and Garabrant DH. 2010. Estimating population distributions when some data are below a limit of detection by using a reverse Kaplan-Meier estimator. Epidemiology, Vol. 21, No. 4. Gleit, A. 1985. Estimation for Small Normal Data Sets with Detection Limits. Environmental Science and Technology, 19, pp. 1206-1213, 1985. Grice, J.V., and Bain, L. J. 1980. Inferences Concerning the Mean of the Gamma Distribution. Journal of the American Statistical Association. Vol. 75, Number 372, 929-933. Gu, M.G., and Zhang, C.H. 1993. Asymptotic properties of self-consistent estimators based on doubly censored data. Annals of Statistics. Vol. 21, 611-624. Hahn, J. G. and Meeker, W.Q. 1991. Statistical Intervals. A Guide for Practitioners. John Wiley. Hall, P. 1988. Theoretical comparison of bootstrap confidence intervals. Annals of Statistics, 16, 927- 953. Hall, P. 1992. On the Removal ofSkewness by Transformation. Journal of Royal Statistical Society, B 54, 221-228. Hardin, J.W. and Gilbert, R.O. 1993. Comparing Statistical Tests for Detecting Soil Contamination Greater Than Background. Pacific Northwest Laboratory, Battelle, Technical Report # DE 94-005498. Hawkins, D. M., and Wixley, R. A. J. 1986. A Note on the Transformation of Chi-Squared Variables to Normality. The American Statistician, 40, 296-298. Hayes, A. F. 2005. Statistical Methods for Communication Science, Lawrence Erlbaum Associates, Publishers. Helsel, D.R. 2005. Nondetects and Data Analysis. Statistics for Censored Environmental Data. John Wiley and Sons, NY. Helsel, D.R. 2012. Statistics for Censored Environmental Data Using Minitab and R. Second Edition. John Wiley and Sons, NY. Helsel, D.R. 2102a. Practical Stats Webinar on ProUCL v4. The Unofficial User Guide; October 15, 2012. Helsel, D.R. 2013. Nondetects and Data Analysis for Environmental Data, NADA in R 258 ------- Helsel, D.R. and E. J. Gilroy. 2012. The Unofficial Users Guide to ProUCL4. Amazon, Kindle Edition. Hinton, S.W. 1993. A Log-Normal Statistical Methodology Performance. ES&T Environmental Sci. Technol., Vol. 27, No. 10, pp. 2247-2249. Hoaglin, B.C., Mosteller, F., and Tukey, J.W. 1983. Understanding Robust and Exploratory Data Analysis. John Wiley, New York. Holgresson, M. and Jorner U. 1978. Decomposition of a Mixture into Normal Components: a Review. Journal of Bio-Medicine. Vol. 9. 367-392. Hollander M & Wolfe DA (1999). Nonparametric Statistical Methods (2nd Edition). New York: John Wiley & Sons. Hogg, R.V. and Craig, A. 1995. Introduction to Mathematical Statistics; 5th edition. Macmillan. Huber, P.J. 1981, Robust Statistics, John Wiley and Sons, NY. Hyndman, R. J. and Fan, Y. 1996. Sample quantiles in statistical packages, American Statistician, 50, 361-365. Interstate Technology Regulatory Council (ITRC). 2012. Incremental Sampling Methodology. Technical and Regulatory Guidance, 2012. Interstate Technology Regulatory Council (ITRC). 2013 Groundwater Statistics and Monitoring Compliance. Technical and Regulatory Guidance, December 2013. Interstate Technology Regulatory Council (ITRC). 2015. Decision Making at Contaminated Sites. Technical and Regulatory Guidance, January 2015. Johnson, N.J. 1978. Modified-t-Tests and Confidence Intervals for Asymmetrical Populations. The American Statistician, Vol. 73, 536-544. Johnson, N.L., Kotz, S., and Balakrishnan, N. 1994. Continuous Univariate Distributions, Vol. 1. Second Edition. John Wiley, New York. Johnson, R.A. and D. Wichern. 2002. AppliedMultivariate Statistical Analysis. 6th Edition. Prentice Hall. Kaplan, E.L. and Meier, O. 1958. Nonparametric Estimation from Incomplete Observations. Journal of the American Statistical Association, Vol. 53. 457-481. Kleijnen, J.P.C., Kloppenburg, G.L.J., and Meeuwsen, F.L. 1986. Testing the Mean of an Asymmetric Population: Johnson's Modified-t Test Revisited. Commun. in Statist.-Simula., 15(3), 715-731. Krishnamoorthy, K., Mathew, T., and Mukherjee, S. 2008. Normal distribution based methods for a Gamma distribution: Prediction and Tolerance Interval and stress-strength reliability. Technometrics, 50, 69-78. 259 ------- Kroese, D.P., Taimre, T., and Botev Z.I. 2011. Handbook of Monte Carlo Methods. John Wiley & Sons. Kruskal, W. H., and Wallis, A. 1952. Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47, 583-621. Kupper, L. L. and Hafher, K. B. 1989, How Appropriate Are Popular Sample Size Formulas? The American Statistician, Vol. 43, No. 2, pp. 101-105 Kunter, M. J., C. J. Nachtsheim, J. Neter, and Li W. 2004. Applied Linear Statistical Methods. Fifth Edition. McGraw-Hill/Irwin. Laga, J., and Likes, J. 1975, Sample Sizes for Distribution-Free Tolerance Intervals Statistical Papers. Vol. 16, No. 1.39-56 Land, C. E. 1971. Confidence Intervals for Linear Functions of the Normal Mean and Variance. Annals of Mathematical Statistics, 42, pp. 1187-1205. Land, C. E. 1975. Tables of Confidence Limits for Linear Functions of the Normal Mean and Variance. In Selected Tables in Mathematical Statistics, Vol. Ill, American Mathematical Society, Providence, R.I., pp. 385-419. Levene, Howard. 1960. Robust tests for equality of variances. In Olkin, Harold, et alia. Stanford University Press, pp. 278-292. Lilliefors, H.W. 1967. On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown. Journal of the American Statistical Association, 62, 399-404. Looney and Gulledge. 1985. Use of the Correlation Coefficient with Normal Probability Plots. The American Statistician, 75-79. Manly, B.F.J. 1997. Randomization, Bootstrap, and Monte Carlo Methods in Biology. Second Edition. Chapman Hall, London. Maronna, R.A., Martin, R.D., and Yohai, V.J. 2006, Robust Statistics: Theory and Methods, John Wiley and Sons, Hoboken, NJ. Marsaglia, G. and Tsang, W. 2000. A simple method for generating gamma variables. ACM Transactions on Mathematical Software, 26(3):363-372. Millard, S. P. and Deverel, S. J. 1988. Nonparametric statistical methods for comparing two sites based on data sets with multiple nondetect limits. Water Resources Research, 24, pp. 2087-2098. Millard, S.P. and Neerchal, M.K. 2002. Environmental Stats for S-PLUS. Second Edition. Springer. Minitab version 16. 2012. Statistical Software. 260 ------- Molin, P., and Abdi H. 1998. New Tables and numerical approximations for the Kolmogorov- Smirnov/Lilliefors/ Van Soest's test of normality. In Encyclopedia of Measurement and Statistics, Neil Salkind (Editor, 2007). Sage Publication Inc. Thousand Oaks (CA). Natrella, M.G. 1963. Experimental Statistics. National Bureau of Standards, Hand Book No. 91, U.S. Government Printing Office, Washington, DC. Noether, G.E. 1987 Sample Size Determination for some Common Nonparametric Tests, Journal American Statistical Assoc., 82, 645-647 Perrson, T., and Rootzen, H. 1977. Simple and Highly Efficient Estimators for A Type I Censored Normal Sample. Biometrika, 64, pp. 123-128. Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1990. Numerical Recipes in C, The Art of Scientific Computing. Cambridge University Press. Cambridge, MA. R Core Team, 2012. R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/. Rosner, B. 1975. On the detection of many outliers. Technometrics, 17, 221-227. Rosner, B. 1983. Percentage points for a generalized ESD many-outlier procedure. Technometrics, 25, 165-172. Rousseeuw, P.J. and Leroy, A.M. 1987. Robust Regression and Outlier Detection. John Wiley. Royston, P. 1982. An extension of Shapiro and Wilk's W test for normality to large samples. Applied Statistics, 31, 115-124. Royston, P. 1982a. Algorithm AS 181: The W test for Normality. Applied Statistics, 31, 176-180. Shacklette, H.T, and Boerngen, J.G. 1984. Element Concentrations in Soils and Other Surficial Materials in the Conterminous United States, U.S. Geological Survey Professional Paper 1270. Scheffe, H., and Tukey, J.W. 1944. A formula for Sample Sizes for Population Tolerance Limits. The Annals of Mathematical Statistics. Vol 15, 217. Schulz, T. W. and Griffin, S. 1999. Estimating Risk Assessment Exposure Point Concentrations when Data are Not Normal or Lognormal. Risk Analysis, Vol. 19, No. 4. Scheffe, H., and Tukey, J.W. 1944. A formula for Sample Sizes for Population Tolerance Limits. The Annals of Mathematical Statistics. Vol 15, 217. Schneider, B.E. and Clickner, R.P. 1976. On the Distribution of the Kolmogorov-Smirnov Statistic for the Gamma Distribution with Unknown Parameters. Mimeo Series Number 36, Department of Statistics, School of Business Administration, Temple University, Philadelphia, PA. Schneider, B. E. 1978. Kolmogorov-Smirnov Test Statistic for the Gamma Distribution with Unknown Parameters, Dissertation, Department of Statistics, Temple University, Philadelphia, PA. 261 ------- Schneider, H. 1986. Truncated and Censored Samples from Normal Populations. Vol. 70, Marcel Dekker Inc., New York, 1986. She, N. 1997. Analyzing Censored Water Quality Data Using a Nonparametric Approach. Journal of the American Water Resources Association 33, pp. 615-624. Shea, B. 1988. Algorithm AS 239: Chi-square and Incomplete Gamma Integrals. Applied Statistics, 37: 466-473. Shumway, A.H., Azari, A.S., Johnson, P. 1989. Estimating Mean Concentrations Under Transformation for Environmental Data with Detection Limits. Technometrics, Vol. 31, No. 3, pp. 347-356. Shumway, R.H., R.S. Azari, and M. Kayhanian. 2002. Statistical Approaches to Estimating Mean Water Quality Concentrations with Detection Limits. Environmental Science and Technology, Vol. 36, pp. 3345-3353. Sinclair, A.J. 1976. Applications of Probability Graphs in Mineral Exploration. Association of Exploration Geochemists, Rexdale Ontario, p 95. Singh, A. 1993. Omnibus Robust Procedures for Assessment of Multivariate Normality and Detection of Multivariate Outliers. In Multivariate Environmental Statistics, Patil G.P. and Rao, C.R., Editors, pp. 445-488. Elsevier Science Publishers. Singh, A. 2004. Computation of an Upper Confidence Limit (UCL) of the Unknown Population Mean Using ProUCL Version 3.0. Part I. Download from: www.epa.gov/nerlesdl/tsc/issue.htm Singh, A., Maichle, R., and Lee, S. 2006. On the Computation of a 95% Upper Confidence Limit of the Unknown Population Mean Based Upon Data Sets with Below Detection Limit Observations. EPA/600/R-06/022, March 2006. http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm Singh, A. and Nocerino, J.M. 1995. Robust Procedures for the Identification of Multiple Outliers. Handbook of Environmental Chemistry, Statistical Methods, Vol. 2.G, pp. 229-277'. Springer Verlag, Germany. Singh, A. and Nocerino, J.M. 1997. Robust Intervals for Some Environmental Applications." The Journal of Chemometrics and Intelligent Laboratory Systems, Vol 37, 55-69. Singh, A. and Nocerino, J.M. 2002. Robust Estimation of the Mean and Variance Using Environmental Data Sets with Below Detection Limit Observations, Vol. 60, pp 69-86. Singh, A.K. and Ananda. M. 2002. Rank kriging for characterization of mercury contamination at the East Fork Poplar Creek, Oak Ridge, Tennessee. Environmetrics, Vol. 13, pp. 679-691. Singh, A. and Singh, A.K. 2007. ProUCL Version 4 Technical Guide (Draft). Publication EPA/600/R- 07/041. January, 2007. http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm Singh, A. and Singh, A.K. 2009. ProUCL Version 4.00.04 Technical Guide (Draft). Publication EPA/600/R-07/041. February, 2009. http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm 262 ------- Singh, A.K., Singh, A., and Engelhardt, M. 1997. The Lognormal Distribution in Environmental Applications. Technology Support Center Issue Paper, 182CMB97. EPA/600/R-97/006, December 1997. Singh, A., Singh A.K., and Engelhardt, M. 1999, Some Practical Aspects of sample Size and Power Computations for Estimating the Mean of Positively Skewed Distributions in Environmental Applications. Office of Research and Development. EPA/006/S-99/006. November 1999. http://www.epa.gov/esd/tsc/images/325cmb99rpt.pdf Singh, A., Singh, A.K., and Flatman, G. 1994. Estimation of Background Levels of Contaminants. Math Geology, Vol. 26, No, 3, 361-388. Singh, A., Singh, A.K., and laci, R.J. 2002. Estimation of the Exposure Point Concentration Term Using a Gamma Distribution, EPA/600/R-02/084, October 2002. Stephens, M. A. 1970. Use of Kolmogorov-Smirnov, Cramer-von Mises and Related Statistics Without Extensive Tables. Journal of Royal Statistical Society, B 32, 115-122. Sutton, C.D. 1993. Computer-Intensive Methods for Tests About the Mean of an Asymmetrical Distribution. Journal of American Statistical Society, Vol. 88, No. 423, 802-810. Tarone, R. and Ware, J. 1978. On Distribution-free Tests for Equality of Survival Distributions. Biometrika, 64, 156-160. Thorn, H.C.S. 1968. Direct and Inverse Tables of the Gamma Distribution. Silver Spring, MD; Environmental Data Service. U.S. Environmental Protection Agency (EPA). 1989a. Methods for Evaluating the Attainment of Cleanup Standards, Vol. I, Soils and Solid Media. Publication EPA 230/2-89/042. U.S. Environmental Protection Agency (EPA). 1989b. Statistical Analysis of Ground-water Monitoring Data at RCRA Facilities. Interim Final Guidance. Washington, DC: Office of Solid Waste. April 1989. U.S. Environmental Protection Agency (EPA). 1991. A Guide: Methods for Evaluating the Attainment of Cleanup Standards for Soils and Solid Media. Publication EPA/540/R95/128. U.S. Environmental Protection Agency (EPA). 1992a. Supplemental Guidance to RAGS: Calculating the Concentration Term. Publication EPA 9285.7-081, May 1992. U.S. Environmental Protection Agency (EPA). 1992b. Statistical Analysis of Ground-water Monitoring Data at RCRA Facilities. Addendum to Interim Final Guidance. Washington DC: Office of Solid Waste. July 1992. U.S. Environmental Protection Agency (EPA). 1994. Statistical Methods for Evaluating the Attainment of Cleanup Standards, EPA 230-R-94-004, Washington, DC. U.S. Environmental Protection Agency (EPA). 1996. A Guide: Soil Screening Guidance: Technical Background Document. Second Edition, Publication 9355.4-04FS. 263 ------- U.S. Environmental Protection Agency (EPA). MARSSIM. 2000. U.S. Nuclear Regulatory Commission, et al. Multi-Agency Radiation Survey and Site Investigation Manual (MARSSIM). Revision 1. EPA 402- R-97-016. Available at http://www.epa.gov/radiation/marssim/ or from http://bookstore.gpo.gov/index.html (GPO Stock Number for Revision 1 is 052-020-00814-1). U.S. Environmental Protection Agency (EPA). 2002a. Calculating Upper Confidence Limits for Exposure Point Concentrations at Hazardous Waste Sites. OSWER 9285.6-10. December 2002. U.S. Environmental Protection Agency (EPA). 2002b. Guidance for Comparing Background and Chemical Concentrations in Soil for CERCLA Sites. EPA 540-R-01-003-OSWER 9285.7-41. September 2002. U.S. Environmental Protection Agency (EPA). 2002c. RCRA Waste Sampling, Draft Technical Guidance -Planning, Implementation and Assessment. EPA 530-D-02-002, 2002. U.S. Environmental Protection Agency (EPA). 2004. ProUCL Version 3.1, Statistical Software. National Exposure Research Lab, EPA, Las Vegas Nevada, October 2004. http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm U.S. Environmental Protection Agency (EPA). 2006a, Guidance on Systematic Planning Using the Data Quality Objective Process, EPA QA/G-4, EPA/240/B-06/001. Office of Environmental Information, Washington, DC. Download from: http://www.epa.gov/quality/qs-docs/g4-fmal.pdf U.S. Environmental Protection Agency (EPA). 2006b. Data Quality Assessment: Statistical Methods for Practitioners, EPA QA/G-9S. EPA/240/B-06/003. Office of Environmental Information, Washington, DC. Download from: http://www.epa.gov/quality/qs-docs/g9s-final.pdf U.S. Environmental Protection Agency (EPA). 2007. ProUCL Version 4.0 Technical Guide. EPA 600-R- 07-041, January 2007. U.S. Environmental Protection Agency (EPA). 2009. Statistical Analysis of Groundwater Monitoring Data at RCRA Facilities - Unified Guidance. EPA 530-R-09-007, 2009. U.S. Environmental Protection Agency (EPA). 2009a. ProUCL Version 4.00.05 User Guide (Draft). Statistical Software for Environmental Applications for Data Sets with and without nondetect observations. National Exposure Research Lab, EPA, Las Vegas. EPA/600/R-07/038, February 2009. Down load from: http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm U.S. Environmental Protection Agency (EPA). 2009b. ProUCL Version 4.00.05 Technical Guide (Draft). Statistical Software for Environmental Applications for Data Sets with and without nondetect observations. National Exposure Research Lab, EPA, Las Vegas. EPA/600/R-07/038, February 2009. Down load from: http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm U.S. Environmental Protection Agency (EPA). 2009c. ProUCL4.00.05 Facts Sheet. Statistical Software for Environmental Applications for Data Sets with and without nondetect observations. National Exposure Research Lab, EPA, Las Vegas, Nevada, 2009. http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm 264 ------- U.S. Environmental Protection Agency (EPA). 2009d. Scout 2008 - A Robust Statistical Package, Office of Research and Development, February 2009. http: //archive. epa.gov/e sd/archive -scout/web/html/ U.S. Environmental Protection Agency (EPA). 2010. Scout 2008 User Guide (Draft) EPA/600/R-08/038, Office of Research and Development, April 2010. http: //archive. epa. gov/e sd/archive -scout/web/html/ U.S. Environmental Protection Agency (EPA). 2010a. A Quick Guide to the Procedures in Scout (Draft), Office of Research and Development, April 2010. http: //archive. epa. gov/e sd/archive -scout/web/html/ U.S. Environmental Protection Agency (EPA). 2010b. ProUCL Version 4.00.05 User Guide (Draft). EPA/600/R-07/041, May 2010. http://www.epa.gov/osp/hstl/tsc/software.htm U.S. Environmental Protection Agency (EPA). 2010c. ProUCL Version 4.00.05 Technical Guide (Draft). EPA/600/R-07/041, May, 2010. http://www.epa.gov/osp/hstl/tsc/software.htm U.S. Environmental Protection Agency (EPA). 2010d. ProUCL 4.00.05, Statistical Software for Environmental Applications for Data Sets with and without nondetect observations. National Exposure Research Lab, EPA, Las Vegas Nevada, May 2010. http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm U.S. Environmental Protection Agency (EPA). 2011. ProUCL 4.1.00, Statistical Software for Environmental Applications for Data Sets with and without nondetect observations. National Exposure Research Lab, EPA, Las Vegas Nevada, June 2011. http://www.epa.gov/osp/hstl/tsc/softwaredocs.htm U.S. Environmental Protection Agency (EPA). 2013a. ProUCL 5.0.00 Technical Guide (Draft) EPA/600/R-07/041. September 2013. Office of Research and Development. http: //www .epa. gov/esd/tsc/TS C form .htm U.S. Environmental Protection Agency (EPA). 2013b. ProUCL 5.0.00 User Guide (Draft) EPA/600/R- 07/041. September 2013. Office of Research and Development. http: //www .epa. gov/esd/tsc/TS C form .htm U.S. Environmental Protection Agency (EPA). 2014. ProUCL 5.0.00 Statistical Software for Environmental Applications for Datasets with and without Nondetect Observations, Office of Research and Development, August 2014. http://www.epa.gov/esd/tsc/TSC form.htm Wald, A. 1943. An Extension of Wilks' Method for Setting Tolerance Intervals. Annals of Mathematical Statistics. Vol. 14,44-55. Whittaker, J. 1974. Generating Gamma and Beta Random Variables with Non-integral Shape Parameters. Applied Statistics, 23, No. 2, 210-214. Wilks, S.S. 1941. Determination of Sample Sizes for Setting Tolerance Limits. Annals of Mathematical Statistics, Vol. 12,91-96. Wilks, S.S. \961.Multivariate statistical outliers. Sankhya A, 25: 407-426. 265 ------- Wilson, E.B., and Hilferty, M.M. 1931, "The Distribution ofChi-Squares" Proceedings of the National Academy of Sciences, 17, 684-688. Wong, A. 1993. A Note on Inference for the Mean Parameter of the Gamma Distribution. Statistics Probability Letters, Vol. 17, 61-66. 266 ------- ------- &EPA United States Environmental Protection Agency Office of Research and Development (8101R) Washington, DC 20460 Official Business Penalty for Private Use \$300 EPA/600/R-07/038 Februaryl 2009 www.epa.gov Please make all necessary changes on the below label, detach or copy, and return to the address in the upper left-hand corner. If you do not wish to receive these reports CHECK HERE D ; detach, or copy this cover, and return to the address in the upper left-hand corner. PRESORTED STANDARD POSTAGE & FEES PAID EPA PERMIT No. G-35 V Recycled/Recyclable Printed with vegetable-based ink on paper that contains a minimum of 50% post-consumer fiber content processed chlorine free -------