United States Environmental Protection Agency Office of Research and Development National Exposure Research Laboratory Research Abstract Government Performance Results Act (GPRA) Goal 3 Annual Performance Measure 197 Significant Research Findings: An Example Uncertainty and Sensitivity Analysis for Risk Assessment of Land-Based Disposal of 7 Chemicals Using EPA's FRAMES 3MRA Version 1.0 Modeling System Scientific The Framework for Risk Analysis in Multimedia Environmental Systems - Problem and Multimedia, Multipathway, and Multireceptor Risk Assessment Policy Issues (FRAMES-3MRA) - is an important Windows-based modeling system developed by EPA for use in risk assessment of hazardous waste management facilities. The system was created through a joint effort between EPA's Office of Solid Waste (OSW) and the Office of Research and Development (ORD). 3MRA features a set of 17 science modules that collectively simulate release, fate and transport, exposure, and risk associated with hazardous contaminants treated or disposed of in various land-based waste management units (WMU). The modeling system embodies one of the first truly integrated multimedia modeling technologies available for risk assessment of both human and ecological populations. Before using the modeling system to support regulatory-based decision-making, OSW was directed by Congress to conduct a panel peer-review of FRAMES 3MRA, to be administered by EPA's Science Advisory Board (SAB). A major element of the review, uncertainty analysis (UA) and sensitivity analysis (SA) of 3MRA was of paramount importance to OSW, ORD, various stakeholders, and panel reviewers. The review was facilitated and successfully concluded, in part through development, demonstration, and communication of the example UA/SA performed for four organic compounds and three heavy metals, reported here. Performed by the Ecosystems Research Division (ERD), the analysis was based on a typical national-scale application for five WMU types handled by 3MRA. This 3MRA UA/SA work provides an initial basis for OSW in meeting guidance on the use of influential information (USEPA, 2002) and on evaluation of models supporting regulatory decision-making (USEPA, 2003). Increasingly critical to many of ORD's clients is the demonstrated ability to perform thorough UA/SA on complex, high order modeling systems such as 3MRA. UA/SA methods and technologies created through this work will ultimately be key in EPA's ability to use 3MRA to support a variety of national, regional, and site-specific exposure and risk assessments constructed to address many types of problem statements. Research Approach ERD has undertaken a broad range of programmatic research goals and tasks to improve the "sound science" of UA/SA. Demonstrating the efficacy of ------- Windows-based computational UA/SA technologies for high order modeling systems like 3MRA is an essential element of this overall research program. The example uncertainty analysis, summarized here, looks at risks from the disposal of seven chemicals and also captures the relative sensitivity of various exposure pathways, media, and receptor types in driving risk levels for ecological receptors and human health. Incorporating landfills, waste piles, aerated tanks, surface impoundments, and land application units; the site-based data used in the example analysis included 201 facilities across the United States representing 419 site-WMU combinations. The example chemicals analyzed here included benzene, benzo(a)pyrene, PCE, TCDD, arsenic, mercury, and nickel and were selected to represent a broad range of chemical properties important to fate and transport in the environment. The predictive uncertainty analysis for 3MRA was constructed using a "pseudo" 2nd order analysis approach, which produces outputs that essentially mimic a two- stage Monte Carlo technique (Babendreier, USEPA, 2003a). The probabilistic risk analysis approach evaluated uncertainty in describing population and sub- population receptor risk due to land-based disposal of contaminated solid wastes, on a national scale. The approach quantified and separated uncertainty and natural variability based upon best available information embodied in the 3MRA data sets. The uncertainty analysis also dealt with output sampling error which arises in Monte Carlo Simulation (MCS) used to solve the 3MRA modeling system equations. The latter error is an aspect of computational precision, where the treatment and separation of variability and empirical uncertainty deals with the accuracy of 3MRA modeling system predictions. In carrying out this work, EPA's PC-based supercomputing cluster, SuperMUSE, and its supporting software were utilized to facilitate verification testing and to carry out the UA/SA (see FY2004 APM 269). This supporting software underwent similar rigorous quality assurance testing employed by EPA in the development of the 3MRA modeling system, where the UA/SA was subjected to extensive peer-review by a 16-member SAB panel (USEPA, Babendreier, 2003a; SAB, 2004). The supporting SuperMUSE software tool set and 3MRA have been extensively tested through the execution of over 60 million model simulations conducted to date. The UA/SA tool set used in this approach was also subjected to additional internal and external peer review through submission and acceptance of a conference paper, journal paper, and poster (Babendreier and Castleton, 2002, 2004, Babendreier etal., 2003b). Results and Impact This initial demonstration of UA/SA for 3MRA was successfully executed, providing a proof-of concept in the ability to enhance quality assurance in complex problem solving that directly supports regulatory decision-making. The approach allowed EPA to develop an example uncertainty and sensitivity analysis for a national-scale risk assessment of seven chemicals which involved over 5,400,000 3MRA model system runs. With an average "stand-alone" PC model runtime of 160 seconds, and with the use of SuperMUSE and its supporting software, ORD was able to complete the needed modeling system simulations in a period of two months. An additional month was needed to develop the analysis of modeling inputs and outputs captured in this interactive research product. The ------- electronic form of the research product was formulated to allow a variety of users to easily query many facets of the UA/SA. The methodology, technology, and application developed through this work served OSW and ORD in successfully concluding the high-profile SAB peer review of the 3MRA Modeling System (SAB, 2004). The hands-on example of the 3MRA UA/SA was, in the end, pivotal in facilitating the SAB's informed review of 3MRA, and their general acceptance of the approach it embodies. The outcome of this peer review acknowledges that it is appropriate for OSW to use 3MRA for national-scale regulatory decision-making in the management of toxic hazardous wastes throughout the U.S. The analysis provided will also assist stakeholders in better understanding the strengths and weaknesses of 3MRA model predictions. Research Collaboration and Research Products ERD's UA/SA parallel computing research program has been carried out through a combination of in-house efforts and key collaborations with two external partners, including the Office of Solid Waste and the Department of Energy's Pacific Northwest National Laboratory operated by the Battelle Corporation. In addition to the Agency's core 3MRA Modeling Team, other contributors to the design approach and software development work included: Kurt Wolfe and Rajbir Parmar, NERL/ERD, in software system development. Examples of recent publications relevant to this study include: Babendreier J.E., Castleton, K. J.. (2004; Accepted). Investigating Uncertainty and Sensitivity in Integrated, Multimedia Environmental Models: Tools for FRAMES-3MRA. Invited paper to appear in Special Edition of International Journal of Environmental Modeling and Software. Babendreier, J.E.., USEPA (2003a). The Multimedia, Multipathway, Multireceptor Risk Assessment Modeling System (FRAMES-3MRA Version 1.0) Documentation. Volume IV: Evaluating Uncertainty and Sensitivity. Draft SAB Review Report: EPA530/D/03/001d. Office of Solid Waste and Office of Research and Development, Washington D.C. (see also EPA530/D/03/001a, b, c, e). http://www.epa.gov/ceampubl/mmedia/3mra/index.htm. Babendreier, J.E., Parmar, R.S., Wolfe, K., Uter, S., and McKendrick, M. (2003b). PC-based supercomputing for uncertainty and sensitivity analysis of models. EPA Science Forum 2003, Washington, DC, May 5-7, 2003. http://www.epa.gov/athens/forum2003/babendrier i poster.pdf Babendreier J.E., Castleton, K. J.. (2002). Investigating Uncertainty and Sensitivity in Integrated, Multimedia Environmental Models: Tools for FRAMES-3MRA. In Proc. Of 1st Biennial Meeting of International Environmental Modeling and Software Society, (2) 90-95, Lugano, Switzerland. SAB (2004). Multimedia, Multipathway, and Multireceptor Risk Assessment (3MRA) Modeling System Panel, http://www.epa.gov/sab/panels/3mramspanel.html USEPA (2002). Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. Office of Environ. Information. EPA/260R-02-008, http://www.epa.gov/qualitv/infonnationguidelines/index.html USEPA (2003). Draft Guidance on the Development, Evaluation, and Application of Regulatory Environmental Models. Office of Research and Development, Office of Science Policy, Council for Regulatory Environmental Modeling (CREM), http://cfpub.epa.gov/crem/cremlib.cfm. ------- Future Research While some uncertainties in the national scale assessment cannot be explicitly dealt with at this time, their impact can be further evaluated through use of various sensitivity analysis techniques that will eventually be investigated for 3MRA Current work is underway to develop an initial public release of the supporting UA/SA software tool set and documentation, to be captured as 3MRA Version 1.x, for distribution on EPA's Center for Exposure Assessment Modeling (CEAM). Over the next 3 to 5 years, ERD will also continue work to create additional customer-based applications of 3MRA UA/SA, along with external collaborations to further transfer and refine this model evaluation technology. Near-future experimentation for UA/SA of 3MRA applications include: • Evaluating two promising global-based sensitivity analysis techniques (Regional Sensitivity Analysis and Tree Structured Density Estimation). • Quantifying uncertainty in risk reduction resulting from a national Agency initiative to reduce persistent, bioaccumulative, and toxic (PBT) chemical disposal by 50% by 2005. Questions and inquiries can be directed to: Justin Babendreier, Ph.D., P.E. U.S. EPA, Office of Research and Development National Exposure Research Laboratory Ecosystems Research Division 960 College Station Road Athens, GA 30605-2700 Phone: 706/355-8344 E-mail: babendreier.justin@epa.gov Federal funding for this research was administered under EPA contract number GS-35F-5338H, Order No. 3D-5386-NBLX. Support for the in-house portion of this project was provided through the U.S. EPA's Office of Research and Development, National Exposure Research Laboratory, and the work was conducted by the Ecosystems Research Division. Contacts for Additional Information ------- |