P/EPA
            United States
            Environmental Protection
            Agency
              Solid Waste And
              Emergency Response
              (5103)
OSWER Directive #9029.00
EPA 500-B-94-003
July 1994
Assessment Framework For
Ground-Water Model Applications
            OSWER Information Management

-------

-------
      \
   UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
              WASHINGTON, D.C. 20460
                                 7 ioo/?
                                   tv.-v^^.
                                                            OFFICE OF
                                                       SOLID WASTE AND EMERGENCY
                                                            RESPONSE
MEMORANDUM
SUBJECT:
FROM:
TO:
Assessment Framework for Ground-Water- Model
Applications  -  Directive No/ 9029.00
Elliott P. Laws
Assistant Administ
                               tor
OSWER Office  Directors
Regional Waste  Management Division Directors
    • This memorandum establishes the guidance,  "Assessment
Framework for Ground-Water- Model.Applications",  as an OSWER
Directive 9029.00.

     The Framework  provides guidance for planning and evaluating
ground-water  'flow and advective transport model  applications.
The set of .criteria helps guide current or  future modeling by
assessing modeling  activities, thought processes,  and
documentation needs. It is intended for EPA technical support
staff and remedial  project managers, as well as  program managers
and contractors who support EPA's waste management program.

Purpose of  the Guidance                '                 '

     The purpose of this guidance is to promote  the appropriate
use of .ground-water models in EPA's waste management programs.
More specifically,  the objectives of the  "Assessment Framework
for Ground-Water Model Applications" are  to:        .   ' •

     Support  the use of ground-water models as  tools for aiding
     decision-making under conditions of  uncertainty;

     Guide  current  or future modeling;

     Assess., modeling activities and thought processes; and

     Identify 'model application documentation needs.
                                                      Recycled/Recyclable
                                                      Printed with Soy/Canola Ink on paper that
                                                      contains at least 50% recycled fiber

-------
Background

     OSWER management has been addressing the issue of the
increasing use of environmental regulatory models first by
reviewing modeling use and support in the waste management
program.  In 1992, OSWER produced the: Ground-Water Modeling-
Compendium to increase the awareness of ground-water models which
are available and supported by EPA.  The Compendium contained the
first version of the "Assessment Framework for Ground-Water Model
Applications", which was developed by a group of experts from
within and outside of EPA and reviewed by a cross-Agency ad hoc
group of scientists.

     The Science Advisory Board  (SAB) then reviewed the Framework
and suggested that it be distributed apart from the Compendium.
The SAB's comments are reflected in the attached version.

     The 1994 edition of the Ground-Water Modeling Compendium,
c6ntaining additional model information and cost guidelines, will
soon be available for distribution from the National Center for
Environmental Publications and Information (NCEPI), 500-B-94-003.

     In addition, in 1992, -OSWER and the Office of Research and
Development, requested the Deputy Administrator to establish a
temporary Agency Task Force on Environmental Regulatory Modeling
.(ATFERM) to address modeling issues across the Agency.  ATFERM
'has produced a report which it presented to the Science Policy
Council in July.  The Deputy Administrator has distributed the
"Guidance for External Peer Review of Environmental Regulatory
Modeling" which was developed by ATFERM and is^in their report.
In addition, the Science Policy Council agreed to the
establishment of a permanent, Agencywide group to provide a focus
for modeling issues and information.
        *
     In summary, OSWER and Regional managers should encourage the
use of the Framework to ensure that sound and defensible models
are being chosen, that these models are being applied in a
reasonable manner, and that management's decision objectives are
incorporated into the modeling objectives for each application.

Attachment - Assessment Framework for Ground-Water Model
             Applications - OSWER Directive 9029.00
                (EPA500-G-94-004)

-------
Assessment Framework
OSWER Directive #9029.00
    Introduction
Assessment Framework for  Ground-Water Model  Applications

Introduction

      The Assessment Framework addresses the use and review of primarily
ground-water flow and advective transport model applications. The criteria in this
Assessment Framework focus upon the activities and thought processes that should
be part of a model application and the subsequent documentation of that activity or
process.  The Assessment Framework is a "living document" which may be
expanded as additional information is collected, analyzed, and organized.

      The intended primary users of this framework are U.S.  Environmental
Protection Agency (EPA) technical support staff and remedial project managers. The
secondary users of the framework are Office of Solid Waste and Emergency
Response (OSWER) and Regional management, EPA contractors, and other
consultants.  However, this Assessment Framework is not a substitute for modeling
education and experience.  The framework should not be used to promote modeling
by inexperienced people, nor should it be relied upon to supplant experienced
professional judgment or measurement.

      The objectives of the Assessment Framework are to:

     O   Support the use of ground-water models as tools for  aiding decision-making
         under conditions of uncertainty;
     O   Guide current or future modeling;
     n   Assess modeling activities and thought processes; and
     D   Identify model application documentation needs.

      Modeling is often used for prediction and/or evaluation of alternative
remedial schemes. While prediction is often the endpoint of the modeling process,
the value of modeling is not limited to this goal. The modeling process can
enhance one's understanding of the natural system, help in the refinement of a
conceptual model, facilitate hypothesis  testing, help check consistency of data sets,
help identify critical processes,  and aid in the planning of site characterization.

      Modeling is not a linear  process but instead is an iterative evolutionary
approach to the refinement of our understanding of a natural system.

      The framework contains a series of assessment criteria, grouped into eight
categories:
     O   Modeling Application Objectives

     O   Project  Management
                                                                     Page 1

-------
Assessment Framework
OSWER Directive #9029.00
    Introduction
     O   Conceptual Model Development

     O   Model (code) Selection

     O   Model Setup and Input Estimation

     O   Simulation of Scenarios

     O   Post Simulation Analysis

     O   Overall Effectiveness.

      Figure 1 clarifies how the criteria in the Assessment Framework apply to the
ground-water modeling process.  The figure also indicates key points in the
modeling process where prior decisions or assumptions should be reviewed and
adjusted if necessary.  These review steps are emphasized because modeling is
naturally an iterative process and is analogous to the scientific method of
formulating a hypothesis and testing it. If the hypothesis, or for example,
conceptual model, is shown at a later stage in the modeling process to be incorrect or
incomplete,  another hypothesis needs to be formulated and the modeling process
started again. To optimize or streamline this process, a model should be prepared at
the beginning of a project (for example, during site characterization) so that it can be
refined and used during the entire length of a project (for example, for feasibility
studies, remedial design, remedial action, and eventually for site closure). Because
Figure 1  is generic and may not apply to all sites, professional advice and experience
should be utilized in the application of models.

      A user with the appropriate training and experience may apply these criteria
at various stages in the modeling process. For example, when modeling is initially
proposed the user may apply the "Modeling Application Objectives" and "Project
Management" criteria to help determine the applicability of the  modeling to the
specific situation. During the early application of the model a user may apply the
"Conceptual Model Development," "Model (code) Selection," and "Model Setup
and  Input Estimation" criteria to help guide the modeling process. Upon
completion of the model application the user may apply the "Simulation of
Scenario," "Post Simulation Analysis," and "Overall Effectiveness" criteria to help
assess the results of the modeling and to guide future efforts.

      OSWER and Regional managers may encourage the use of the criteria to
ensure that sound and defensible models are being chosen and that these models are
being applied in a reasonable manner.  The manager may also encourage the use of
the criteria to ensure that management decision objectives are incorporated into the
modeling objectives.
Page 2

-------
Assessment Framework
                                               Introduction
                       ESTABLISH MODELING OBJECTIVES
                       - Establish decision objectives
                       - Determine the necessity of ground-water modeling
                       - Determine the level of model complexity
                            ESTABLISH PROJECT MANAGEMENT PLAN
                        COLLECT, ORGANIZE, AND INTERPRET AVAILABLE
                                            DATA
                                PREPARE A CONCEPTUAL MODEL
                           SELECT A SUITABLE MATHEMATICAL CODE
                            SET UP THE MODEL AND PERFORM INPUT
                                         ESTIMATION
COMPARE WITH FIELD
DATA
Ifc.

CALIBRATE THE MODEL*
^
r
                                                            NO
   HISTORY MATCHING
  NOTES:

 * Includes calibration
  sensitivity analysis
 ** Includes predictive
   sensitivity analysis
                            EVALUATE THE OVERALL EFFECTIVENESS
                                      NO
             ARE THE MODELING
              OBJECTIVES MET?
                YES
                                      ARE CALIBRATION
                                        TARGETS MET?
                                  SIMULATE THE SCENARIOS
                             PERFORM POST-SIMULATION ANALYSIS
            (    COMPLETE    )

FIGURE 1 - Diagram of the groundrwater modeling process.
                                                                              Page 3

-------
Assessment Framework
OSWER Directive #9029.00
    Introduction
   •   These criteria will generally apply to most modeling applications, however,
in certain, instances some of the criteria may not be applicable or some of the criteria
may be applicable at different stages in the modeling process. In other instances,
some criteria may have to be modified or expanded.

      Documentation of the modeling process is crucial for assuring the
defensibility of the modeling application.  Consequently, some of the following
criteria are preceded with an asterisk (*) indicating that the analysis, process, or data
referred to in the question should be documented.  Some users may find it useful to
reference additional information when applying the criteria.  Therefore, some of the
criteria are followed by endnotes which provide further explanation and reference
additional sources of information.  The criteria also contain numerous technical
terms that may require additional explanation.  These terms are italicized
throughout the document. A glossary that follows the criteria contains the
definitions for these terms in the context of the criteria.
Page 4

-------
Assessment Framework
                                                              OSWER Directive #9029,00
                                                          Assessment Criteria
Assessment Criteria

Modeling Application Objectives

1       ^Management's  decision objectives  should be clearly specified up front,
        considering applicable regulatory and .policy issues.

2       The role and need for a modeling study in the pursuit of management's
        decision  objectives should be established.

3       * Management's  decision objectives  should be translated into modeling
        objectives up front.

4       Modeling  objectives should be based upon existing information about the
        physical characteristics of the site (e.g., hydrogeologic system) and the source,
        location> and nature of the contamination.

5       *A11 assumptions incorporated within the modeling objectives should be
        reviewed with respect to reality and their potential impacts on
        management's decision  objectives.

6       The purpose of the model application (e.g., data organization,
        understanding the system, planning. additional field characterization,
        prediction, or evaluation of remediation alternatives) should be defined
        during the development of the modeling  objectives.

7       The purpose of the model application should be reviewed during the course
        of the project and, if necessary, modified.

 8      Potential solutions to be evaluated (e.g., containment and/or remediation
        solutions) should be identified prior to the initiation of the modeling.

9       The level of model complexity and, in turn, the type of model required
        (e.g., numerical  model, analytical model, or graphical techniques) should be
        determined during the definition of the modeling  objectives.

        See Endnote 1.

10      This level of model complexity should be reviewed as  a better
        understanding of the site/problem/data is developed.

11      *Management, in consultation with a professional ground-water scientist,
        should specify the time period (e.g., 1 year, 10 years, or hundreds of years) for
        which model predictions are intended.
                                                                       Page 5

-------
Assessment Framework
    OSWER Directive #9029.00
Assessment Criteria
12      Calibration  targets (e.g., multiple criteria such as heads and ground-water
        discharge) for the model application should be specified up front.

13      If, during the modeling process, it is determined that the original calibration
        targets cannot be met, the modeling objectives should be reviewed.

14      *An analysis should be performed of the incremental costs associated with
        expanding these study objectives (e.g., expanding the size of the study area,
        the number of remedial technologies modeled, or the calibration targets of
        the model) and the  consequent incremental improvement in supporting
        management's  decision  objectives.

15      Management's  decision  objectives should be reaffirmed throughout the
        modeling process.

16      The modeling  objectives should be reviewed, after the development of the
        conceptual  model and prior to the initiation of the modeling, to ensure that
        they support management's  decision  objectives.
Page 6

-------
Assessment Framework
   OSWER Directive #9029.00
Assessment Criteria
Project Management

17     *A financial budget appropriate to the modeling objectives, level of analysis,
       available data, and available resources should be established at the inception
       of the project.

18     The individuals who are actually managing or performing the modeling
       should participate in the development of the financial budget.

19     The project organization should .be designed to facilitate the interative
       nature of the modeling process.

20     The individuals who are actually performing  the modeling, managing the
       modeling effort, or performing the peer  review should have the ground-
       water modeling experience required for the project. Specifically, for their
       role on the project, each should have the appropriate level of:
           '  •   Formal training in mathematics,  physics, chemistry, soil science,
                 fluid -mechanics, geology, hydrogeology, and modeling
             •   Work experience in modeling physical  systems, preferably with
                 the type of model being used on  the project
             •   Field experience characterizing site hydrogeology
             •   Modeling  project management experience.

        See Endnote 2.

21      These individuals should be organized as  a cohesive modeling team with
        well defined roles, responsibilities, and level of participation.

22      The organization of the team should be appropriate for the application.

23      A documentation procedure should be established up front to assure that
        an independent reviewer can duplicate the modeling results or perform a
        postapplication assessment using the documentation.

24      The documentation should include a discussion of the following:
                 General setting of the site
                 Physical systems of interest
                 Management's  decision  objectives
                 Role of modeling study
                 Potential solutions to be  evaluated
                 Modeling   objectives and timeframe for model predictions
                 Level of model complexity
                 Calibration targets
                 Quality assurance and peer review process
                 Composition of the modeling  team       ;               ,
                                                                       Page 7

-------
Assessment Framework
    OSWER Directive #9029.00
Assessment Criteria
                 Data -sources, quality, and completeness
                 Conceptual  model •
                        Hydro stratigraphy
                        Ground-water  flow system
                        Hydrologic  boundaries
                        Hydraulic  properties
                        Water  sources and sinks
                        Contaminant identity, source, loading, and areal extent
                    —   Contaminant transport  and transformation processes
                    —   Background chemical quality
                    —   Boundary  conditions
                 Selection of the computer code
                    —   Description of the code and documentation
                    —   Reliability
                    —   Usability
                    —   Transportability
                        Performance
                        Access to source code
                        Limitation
                    —   Related Applications
                 Ground-water model construction'
                    —   Code modifications
                        Geologic representation
                        Flow representation
                    —   Data averaging procedures
                        Input estimation procedures
                        Model grid  . .
                    —   Hydraulic parameters
                        Chemical parameters
                    —   Boundary  conditions
                        Water budget
                    —   Simplifying assumptions
                 Uncertainty  analysis              ,  ,   '
                 Calibration and calibration sensitivity analysis
                 Predictive simulations
                        Scenarios
                        Implementation of the  scenarios
                        Discussion of the results of each run
                    —   Predictive  sensitivity analysis
                    —   Postprocessing
                 Modeling study scope, conceptual  model, and model code
                 assumptions with respect to  reality and their impacts on the
                 modeling results
                 Results related to  management's information needs as formulated
                 in the decision objectives
Page 8

-------
Assessment Framework
                                                              OSWER Directive #9029.00
                                                           Assessment Criteria
            •    Executive summary (in terms of the decision objectives)
            •    References
            •    Input and output files.

        See Endnote 3.
                                \

25      An independent quality assurance  (QA) process not involving staff
        assigned to any aspect of the project should be established at the beginning
        of the project,

        See Endnote 4.

26      *This QA process should include ongoing peer review of the:
            •    Modeling objectives development
            •    Conceptual  model development
            •    Model code selection
            •    Model setup and calibration
            •    Simulation of scenarios
            •    Postsimulation analysis.
                                                                        Page 9

-------
Assessment Framework
   OSWER Directive #9029.00
Assessment Criteria
Conceptual Model Development

27      An initial conceptual  model of both the local and regional hydrogeological
        system should be developed prior to any computer modeling.

28      The conceptual  model should be based upon a quantification of field data
        as well as other qualitative data that includes information on the nature
        and variability of the:
            •   Aquifer system (distribution and configuration of aquifer and
                confining formations)
            •   Hydrologic  boundaries
            •   Hydraulic and chemical properties of formations
            •   Piezometric head and hydraulic gradient (i.e., magnitude and
                direction of flow within each model layer)
            •   Fluid properties
            •   Contaminant sources and properties
            •   Fluid sources and  sinks.

        See Endnote 5.

29      The quantity, quality, and completeness of the data should be analyzed
        with regard to their impact on the overall success of the model application.

30      *I£ there are data gaps, the tradeoff should be analyzed between the cost of
        acquiring additional data and the consequent improvement in meeting
        management's  decision objectives.

31      If there are data gaps (e.g., missing water level or hydraulic  conductivity
        information), any additional field work and other attempts to fill in these
        gaps should be documented.

32      The data sources should be documented.

33      *Any and all potential interactions with other physical systems (e.g., surface
        water systems or agricultural systems) should be evaluated  prior to the
        beginning of the modeling by means of a water  budget, a chemical mass
        balance, or other analytical techniques.

34      The manner in which existing and future engineering (e.g., wells or slurry
        walls) must be represented in a numerical  or  analytical model should be
        explicitly incorporated into the conceptual  model.

35      A mass balance of the contaminant should be developed as part of the
        conceptual model.                                       .
Page 10

-------
Assessment Framework
   OSWER Directive #9029.00
Assessment Criteria
36      *The conceptual  model should include a dear statement of the location,
        type, and state of boundary conditions; justification of their formulation;
        and source(s) of information used to develop the boundary conditions.

37      *A11 assumptions incorporated within the conceptual  model should be
        reviewed with respect to reality and their potential impacts on the modeling
        objectives.
                                                                      Page 11

-------
Assessment Framework
                                                             OSWER Directive #9029.00
                                                          Assessment Criteria
Model (code) Selection

38      "The selected model (code), as distinguished from the model application,
        should be described with regard to its flow, contaminant transport and
        transformation processes, mathematics, hydrogeologic system
        representation, boundary  conditions, and input parameters.

39      The reliability of the model (code) should be assessed including a review
        of:
            •    Peer  reviews of the model's theory (e.g., a formal review process
                 by an individual or organization acknowledged for their expertise
                 in ground-water modeling or the publication of the theory in a
                 peer-reviewed journal)
            •    Peer  reviews of the model's code (e.g., a formal review process by
                 an individual or organization acknowledged for their expertise in
                 assessing ground-water computer models)
            •    Verification  studies (e.g., evaluation of the model results against
                 laboratory tests, analytical solutions, or other well accepted
                 models)
            •    Relevant field tests (i.e., the application and evaluation of the'
                 model to site-specific conditions for which extensive data sets are
                 available)
            •    The model's (code) acceptability in the user community as
                 evidenced by the quantity and type of use.

        See Endnote 6.

40      The usability of the model (code) should be assessed including the
        availability of:
                 The model binary code
                 The model source  code
                 Pre- and postprocessors
                 Existing data resources
                 Standardized data formats
                 Complete user instruction manuals
                 Sample problems
                 Necessary hardware
                 Transportability across platforms
                 User support
                 Key  assumptions.

41      The tradeoff should be analyzed between model (code) performance (e.g.,
        accuracy and processing speed) and the human and computer resources
        required to perform the modeling.
 Page 12

-------
Assessment Framework
    OSWER Directive #9029.00
Assessment Criteria
42      If the ground-water model results will be used to form expert opinions, all
        parties should have access to the foundation of those opinions, including
        the source code and an executable image of the version used, if present. In
        addition, the ground-water model code should meet the following criteria:
            •    Publication and peer  review of the model's conceptual and
                 mathematical framework, including the model's underlying
                 assumptions
            •    Full model documentation
            •    Publication and peer  review of model code testing.

        See Endnote 7.

43      The assumptions in the model (code) should be analyzed with regard to
        their impact upon the modeling  objectives and site-specific conditions.

44      *Any and all discrepancies between the modeling requirements (i.e., as
        indicated by management's decision  objectives, conceptual  model, and
        available data) and the capabilities of the selected model should be identified
        and justified. For example, the implications of the selected code supporting
        1-, 2- or 3-dimensional modeling; providing steady versus unsteady state
        modeling; or requiring simplifications of the conceptual model should be
        discussed.

45      *If the modeling objectives are modified due to such discrepancies", those
        modifications should be documented.

46      *If the model source code is modified, the following tests should be
        performed and the testing methodology and results should be justified:
            •    Reliability testing (See Criterion #38)
            •    Usability evaluation (See Criterion #39)
            •    Performance testing.

        See Endnote 8.                                                      ^
                                                                      Page 13

-------
Assessment Framework
                                                             OSWER Directive #9029.00
                                                          Assessment Criteria
Model Setup and Input Estimation

47     *When the dimensional aspects of the geology at the site are simplified in
       the model  representation (e.g., representing a multilayer aquifer with a
       single layer), the impact of this simplification on the modeling results
       should be evaluated.

48     *When data averaging procedures are used to represent the site conditions
       in the model, the impact of the averaging upon the modeling results should
       be evaluated.

        See Endnote 9.

49      *When flow representations in the model are assumptions or
        simplifications of site conditions (e.g., only horizontal flow is considered,
        thus ignoring the impact of vertical flow components), the impact of these
        assumptions and simplifications on the modeling results should be
        evaluated.

50      For numerical  models, generally acceptable rules of grid design and time
        step selection should be applied to meet the modeling objectives.

        See Endnotes 10 and 19.

51      *When a numerical model  is used, the mapping of the  location of the
        boundary conditions and other geometric details (e.g., wells, slurry walls,
        and contaminant sources) on the grid should be evaluated. For example:
             •   The manner in which the boundaries are represented in the grid
                 should ensure the fineness of the grid, the accuracy of the
                 geometry, and the accuracy of the boundary conditions.
             •   For finite element grids, internal and external boundaries should
                 coincide with element boundaries.

        See Endnotes 11 and 12.

52      *If arbitrary or artificial boundaries are used, justification for their use
        should be given and evidence presented to demonstrate that their use does
        not adversely impact the model results within the area  of interest.

53      *When  an. analytical  model Is used, the following boundary  conditions
        should be evaluated:
             •   Where infinite boundaries are used, the engineering feature being
                 modeled should not impact actual physical boundaries  at the site
                 during the timeframe of interest.
 Page 14

-------
Assessment Framework
    OSWER Directive #9029.00
Assessment Criteria
             •    Where image wells are used, general rules of imaging should be
                 followed.

        See Endnote 13.

54      The data sources, the data collection procedures, and the data uncertainty
        for the model input data should be evaluated and documented in the
        project report or file.

55      *A11 model inputs should be defined as to whether they are measurements,
        interpretations, or assumptions including:
             •    The constitutive coefficients and  parameters (i.e., parameters that
                 are not generally observable but must be inferred from
                 observations of other variables; for example, the distribution of
                 transmissivity and specific storage)
             •    The forcing  terms (e.g., sources and  sinks of water and dissolved
                 contaminants)
             •    The Boundary conditions
             •    The initial conditions.

56      The input estimation process  whereby data are converted into model
        inputs  (e.g., spatial and temporal interpolation, extrapolation or Kriging, or
        averaging)  should be described. This description should include a map or
        table containing the spatial location and the associated values of data used to
        perform the interpolation.

        See Endnote 12.

57      The uncertainty associated with the input estimation process should be
        specified, explained, and documented.

58      The model should be calibrated.

59      *If the model is not calibrated, the  rationale for not calibrating the model
        should be explained.

        See Endnote 14.

60      The criteria (i.e., calibration targets) used in the termination of the
        calibration process should be justified with regard to the modeling
        objectives.

        See Endnote 12.
                                                                      Page 15

-------
Assessment Framework
    OSWER Directive #9029.00
Assessment Criteria
61  •    *The calibration should be performed in a generally acceptable manner.
        Specifically:
            •    A calibration sensitivity  analysis should be performed to
                 determine the key parameters and boundary  conditions to be
                 investigated during calibration.
           • •    The calibration should include an evaluation of spatial residuals
                 between simulated and measured values.  If transient data are
                 available, an evaluation of spatial  residuals at selected time steps
                 should also be performed.
            •    The calibration should be performed in the context of the physical
                 features (e.g., residuals should be analyzed with respect to the
                 pattern of ground-water contours  including mounds, depressions,
                 or indications of surface water discharge or recharge).

        See Endnote 15.

62      A water budget for the natural aquifer system based upon measurements
        and/or estimates should be developed and used to create a mass  balance for
        the model.

63      *If a water budget is not developed, the reasoning for not developing a
        water budget should be explained.

64      *If actual measurements of components of the water budget  are available,
        they  should'be used to calibrate the model.

65      *A11 changes in initial model parameter values due to calibration should be
        justified as to their reasonableness.

66      *Any discrepancies between the calibrated model parameters and the
        parameter ranges estimated in the conceptual model should be justified.

67      *If the conceptual   model is modified as a result of the model calibration, all
        changes in the conceptual  model should be justified.
Page IB

-------
Assessment Framework
                                                             OSWER Directive #9029.00
                                                          Assessment Criteria
Simulation of Scenarios

68      *For each modeling scenario, the model inputs and the location of features
        in the model grid should be justified. For example:
            •    For finite element  models, if a pumping well was not located at a
                 node, the allocation of well discharges among neighboring nodes
                 should be justified.
            •    If a slurry wall is a remedial alternative, the representation in the
                 model of the wall's geometric and hydraulic properties  should be
                 justified.
            •    If cleanup times are calculated, all assumptions about the location,
                 quantity, and state of the contaminants should be justified.
            •    When a remedial action, such as extraction wells, affects the flow,
                 such effects should be determined, including the extent of the
                 capture zone.
                                                                      Page 17

-------
Assessment Framework
                                                              OSWER Directive #9029.00
                                                          Assessment Criteria
Postsimulation Analysis

69      The success of the model application in simulating the site scenarios
        should be assessed.

70      This assessment should include an analysis of:
             •    Whether the  modeling simulations were realistic
             •    Whether the  simulations accurately reflected the scenarios
             •    Whether the hydrogeologic system was accurately simulated
             •    Which aspects of the conceptual model were successfully
                 modeled.

71      The sensitivity of the  model results to uncertainties in site-specific
        parameters (predictive  sensitivity analysis}  and the level of error in the
        model calibration (calibration sensitivity analysis) should be examined and
        quantified. For example, the modeling scenarios should be simulated for
        the range of possible values of the more sensitive hydrogeologic parameters.
        Moreover, the range of error in the model calibration should be considered
        when drawing conclusions about the model results.

        See Endnote 16.

72      The modeling results should be consistent with available data.

73      *The postprocessing, including the use of interpolation and smoothing,
        should be analyzed and documented to ensure that it accurately represents
        the modeling  results.

74      The postprocessing results should be analyzed to ensure that they support
        the modeling  objectives.

75      The final presentation  should effectively and accurately communicate the
        modeling results.

76      When feasible, a postaudit of the model should be carried out or planned
        for the future.

        See Endnote 17.  .  .
 Page 18

-------
Assessment Framework
                                                            OSWER Directive #9029.00
                                                         Assessment Criteria
Overall Effectiveness

77
*Any difficulties encountered in the model application should be   ,,0
documented.
78      The model application should provide the information being sought by
        management for decision making.

        See Endnote 18.

79      The model application results should be acceptable to-all relevant parties.

80      The model application should support a timely and effective regulatory
        decision process.

81      Those aspects of the modeling effort that in hindsight might have been
        done differently should be documented.                          ;'c:
                                                                     Page 19

-------
Assessment Framework
                                                            OSWER Directive #9029.00
                                                                   Endnotes
Endnotes                  ,

A complete list of references follows the Glossary of Technical Terms.

1.  Some of the factors which might influence the level of model complexity and,
    in turn, the selection of a particular type (e.g., numerical  model,, analytical
    model, or graphical techniques) of model include:
        •      The importance of the decisions which will be influenced by the
              model results
        •      The sensitivity of these decisions to the range of possible or likely
              outcomes of the modeling
        •      The availability of time and resources for the modeling application
        •      The complexity of site characteristics.

    See Anderson and Woessner 1992a, and other standard modeling references.

2.'  gee USOTA 1982 and van der Heijde and Park 1986.

3.  For more information on documentation see ASTM and CADHS.

4.  QA should assure that:
        •      The project is staffed with qualified people
        •      There is appropriate documentation of the modeling process
        •      Peer  review of the modeling process and project deliverables is
              performed.

    It should be noted that there is no way to assure that a specific model of a •
    physical system can ever be completely verified. Thus, the QA process helps to
    build confidence in a model application, but following the QA process does not
    guarantee accurate predictions. See Konikow and Bredehoeft 1992.

5.  For more information on data requirements for conceptual  model
    development, see ASTM; CADHS, page 4; NCR 1990, pages 221-230; and
    Anderson and  Woessner 1992a.

6.  Sources of information on model codes include USEPA 1992; Bond and Hwang
    1989; the  International Ground Water Modeling Center (IGWMC)  data base; the
    Integrated Model Evaluation  System (IMES);  and numerous ground-water
    modeling texts.

7.  The Special Master in the case of United States of America, et al.vs. Hooker
    Chemicals & Plastics Corporation, et al.(Love Canal) ruled on November 30,
     1989 that if models are to be relied upon to form expert opinions  in litigation,
     all relevant parties will be permitted access to the foundation of those opinions,
     including the source code.  At the same time, the Special Master granted a

Page 20

-------
Assessment Framework
OSWER Directive #9029.00
      Endnotes
     Protective Order so that the code could not be used by the opposing side for any
     purpose other than the trial.  (See United States of America, et al. vs. Hooker
     Chemicals & Plastics Corporation, et al. (Love Canal) on November 30,1989.)
     For further information on the use of ground-water models in litigation, see
     Kezsbom and Goldman 1991.  For more information on model code
     documentation, see van der Heijde and Elnawawy 1992.

8.    For more information on the  testing of model codes, see van der Heijde and
     Elnawawy 1992, Section 3.

9.    Contaminant  transport is affected by "nonaverage" conditions, with
     contaminant plumes following preferential flow paths.  See Anderson and
     Woessner 1992a, page 326; and Fetter 1988, page 395.

10.   For example, the grid should be  fine enough in the area of interest to produce
     accurate results and nodes should coincide with physical features, remediation
     wells, and contamination sources as much as possible.  Grid orientation, grid
     expansion factors, and aspect ratios should also meet general modeling
     standards. For more information, refer to Anderson and Woessner 1992a; and
     van der Heijde, El-Kadi, and Williams 1988, pages 45-48.

11.   For information on properly locating and representing boundary conditions,
     see Franke, Reilly, and Bennett 1984; and Anderson and Woessner 1992a.

12.   For information on model setup, input estimation, and criteria for the
     termination of the calibration  process, see Anderson and Woessner 1992a.

13.   For more information on imaging, see Freeze and Cherry 1979, page 330.

14.   The model should be calibrated,  especially if it is used for predictive purposes.
     For interpretive or generic models, calibration is encouraged but not required
     (Anderson and Woessner  1992a). Analytical models should also be calibrated,
     if possible (e.g., in the case of flow, to transient data; or in the case of transport,
     to plume data).

15.   For more information on the calibration of ground-water models, see
     Anderson and Woessner 1992a; van der Heijde, El-Kadi, and Williams 1988;
     ASTM, Subpart 6.5; and CADHS, Section 3.3.2.4.

16.   When a physical system is subject to new stresses (as during the application of a
     remedial strategy), errors in the conceptual  model which had little impact
     during the calibration phase  may become dominant sources of error for the
     prediction phase.  Because a specific model of a physical system can never be
     completely "verified," it becomes important to identify uncertainties in model
     input parameters and conceptual assumptions and to explore the implications
                                                                     Page 21

-------
Assessment Framework
                                                            OSWER Directive #9029.00
                                                                   Endnotes
     of these uncertainties on model predictions. For a more complete discussion,
     see Konikow and Bredehoeft 1992, pages 75-83.

17.   For more information on post audits, see Anderson and Woessner 1992b.

18.   For more information on decision making under conditions of uncertainty, see
     Freeze, Massmann, Smith, Sperling, and James 1990, pages 738-766.

19.   For information on how time-step size can affect the numerical accuracy of a
     model, see Huyakorn and Finder 1983, pages 206 and 392; van der Heijde, El-
     Kadi, and Williams 1988, pages 45-48; and Anderson and Woessner 1992a.
 Page 22

-------
Assessment Framework
           OSWER Directive #9029.00
Glossary of Technical Terms
Glossary of Technical Terms

This glossary provides definitions for some of the technical terms used in Section
2.0, Assessment Framework, of the Compendium  of Modeling  Information. Words
appearing in italics are defined elsewhere in the glossary.  Numbers in parentheses
following each definition correspond to the reference source for the definition.  A
complete list of references follows the glossary.

       Analytical model - mathematical expression used to study the behavior of
       physical processes such as ground-water ,flow and contaminant  transport.
       This type of model is generally more economical and simpler than a
       numerical model, but it requires many simplifying assumptions regarding
       the geologic setting and hydrologic conditions. In comparison with a
       numerical model, however, an analytical model provides a solution of the
       governing partial differential equation  at any location and/or time instead
       of approximate solutions at discreet locations and moments in time.  (15, 22)

       Boundary conditions - mathematical expressions specifying the dependent
     .  variable (head) or the derivative of the dependent variable (flux) along the
       boundaries of the problem domain. To solve the ground-water flow
       equation specification of boundary conditions, along with the initial
       conditions, is required. .Ideally, the boundary of the model should
       correspond with a physical boundary of the ground-water flow system, such
       as an impermeable body of rock or a large body of surface water.  Many
       model applications, however, require the use of nonphysical boundaries,
       such as ground-water divides and areas of aquifer underflow.  The effect of
     .  nonphysical boundaries on the modeling results must be tested.  (4)    -

:       Calibration - a procedure for finding a set of parameters, boundary
       conditions,, and stresses that produces simulated heads  and fluxes that
       match field-measured values within an acceptable range of error. (4)

       Calibration sensitivity analysis - a procedure to establish the effect of
       uncertainty .on the calibrated model. The calibrated model is influenced by
       uncertainty owing to the inability to define the exact spacial (and temporal)
       distribution of parameter  values in the problem domain.  There is also
       uncertainty over the definition of boundary conditions and stresses.  (4)

       Calibration target - a preestablished range of allowable error between heads
       and fluxes and field measured values.  (4)

       Capture zone  - steady state: the region surrounding the well that contributes
       flow to the well and which extends up gradient to the ground-water divide
       of the drainage basin; .travel time related: the region surrounding a well that
       contributes flow to the well within a specified period of time. (22)
                                                                     Page 23

-------
r
               Assessment Framework
           OSWER Directive #9029.00
Glossary of Technical Terms
                       Conceptual model - an interpretation or working description of the
                       characteristics and dynamics of a physical  system. The purpose of building a
                       conceptual  model is to simplify the field problem and organize the field data
                       so the system can be analyzed more readily.  (4,15)

                       Confining bed - a geologic unit with low values of hydraulic conductivity
                       which allows some movement of water through it, but at rates of flow
                       lower than those of adjacent aquifers.  A confining bed can transmit
                       significant quantities of water when viewed over a large area and long time
                       periods, but its permeability is not sufficient to justify production wells
                       being placed in it. It may serve as a storage unit, but it does not yield water
                       readily.  (1,19,20,23)

                       Constitutive coefficients and parameters - type of model input that is not
                       directly observable but must be inferred from observations of other  model
                       variables; for example, the distribution of transmissivity, specific storage,
                       porosity, recharge, and evapotraspiration. These are difficult to estimate
                       because they vary spatially and may vary temporally as well. (21)

                       Containment - action(s) undertaken, such as' constructing slurry trenches,
                       installing diversionary booms,  earth moving, plugging damaged tank cars,
                       and using chemical retardants.  These actions focus on controlling the
                       source of a discharge" or release and minimizing the spread of the hazardous
                       substance or its effects. (28)

                       Contaminant source, loading, and areal extent - the physical location of the
                       source contaminating the aquifer, the rate at which the contaminant is
                       entering the ground-water system, and the  surface area of the contaminant
                       source, respectively.  In order to model fate and transport of a contaminant,
                       the characteristics of the contaminant source must be known or assumed.
                       (3)

                       Contaminant transformation - chemical changes, reactions, and biological
                       transformations  that change the chemical properties of the contaminant. (3)

                       Contaminant transport - flow  and dispersion of contaminants dissolved in
                       ground-water in the subsurface environment.  (21)

                       Evapotranspiration - a combined term for water lost as vapor from a soil or
                       open water surfaces, such as lakes and streams (evaporation) and water lost
                       through the intervention of plants, mainly via  the stomata (transpiration).
                       Term is used because, in practice, it is difficult to distinguish water vapor
                       from these two sources in water balance and atmospheric studies.  Also
                Page 24

-------
Assessment Framework
           OSWER Directive S90E9.00
Glossary of Technical Terms
        known as fly-off, total evaporation, and water loss.  Losses from
        evapotranspiration can occur at the water table. (1,3)

        Field characterization - a review of historical, on- and off site, as well as
        surface and subsurface data, and the, collection of new,data to meet project
        objectives. When possible, aerial photographs/ contaminant  source
        investigations, soil and aquifer sampling, and the delineation of aquifer
        head and contaminant concentrations should be reviewed.  Field
        characterization is a necessary prerequisite to the development of a
        conceptual  model. (3)

        Finite difference model -  a type of numerical model that uses a
        mathematical technique called finite difference to obtain an approximate
        solution to the partial differential ground-water flow and transport
        equations. Aquifer heterogeneity is handled by dividing the aquifer into
        homogeneous rectangular blocks.  An algebraic equation is written for each
        block, leading to a set of equations which can be input into a matrix and
        solved numerically. This type of model has difficulty incorporating
        irregular and uneven boundaries.  (3,11,15,16)

        Finite element model - a type of numerical  model that uses the finite
        element technique to obtain an approximate solution to the partial
        differential ground-water flow and transport equations.  To handle aquifer
        heterogeneity the aquifer can be divided into irregular, homogeneous
        elements, usually triangles. This type of model can incorporate irregular
        and curved boundaries, sloping soil, and rock layers more easily than a
        finite difference model for some problem types.  This technique, like finite
        difference, leads to a set of simultaneous algebraic equations which is input
        into a matrix and solved numerically.  (3,11,15,16)

        Fluid potential - mechanical energy per unit weight of fluid at any given
        point in space and time with regard to an arbitrary state and datum.  (11, 24)
             *                 •               f
        Forcing terms - type of model input included in most ground-water models
        to account for sources and  sinks of water or dissolved contaminants.  They
        may be measured directly (e.g:, where and when contaminants are
        introduced into  the subsurface environment), inferred from measurements
        of more accessible variables, or they may be postulated (e.g., effect of
        ^proposed cleanup strategy). (21)               ,    .

        Ground-water divides •- ridges in the water table or potentiometric  surface
        from which ground-water moves away in both directions (14); a hydraulic
        boundary at the  crest or valley bottom of a ground-water flow system across
        which there is no flow.  (11)   •      :   .
                                                                      Page 25

-------
Assessment Framework
           OSWER Directive #9029.00
Glossary of Technical Terms
        Ground-water flow system - a rather vague designation pertaining to a
        ground-water reservoir that is more or less separate from neighboring
        ground-water reservoirs.  Ground-water reservoirs can be separated from
        one another by geologic or hydrologic boundaries. In some ground-water
        modeling studies, artificial or arbitrary boundaries may be applied. Water
        moves or "flows" through the ground-water reservoir through openings in
        sediment and rock. (10)

        Hydraulic conductivity - the ability of a rock, sediment, or soil to permit
        water to flow through it.  The scientific definition is the volume of water
        that will move in unit time under a unit hydraulic gradient through a unit
        area measured at right angles to the direction of flow.  (22)

        Hydraulic properties - those properties of a rock, sediment, or soil that
        govern the entrance of and the capacity  to yield and transmit water (e.g.,
        porosity, effective porosity,  specific yield,  and hydraulic conductivity). (3,33)

        Hydrologic boundaries - boundary  conditions which relate to the flow of
        water in an aquifer system.  (3)

        Hydrostratigraphy - a sequence of geologic units delimited on the basis of
        hydraulic properties. (10)

        Kriging - an interpolation procedure for estimating regional distributions of
        ground-water model inputs from scattered observations. (21)

        Management's decision objectives - the information needs required to
        identify courses  of action necessary for reaching environmental and
        regulatory goals.

        Model grid - a system of connected nodal points superimposed over the
        aquifer to spatially discretize the aquifer into cells  (finite  difference  method)
        or elements (finite element  method) for the purpose of mathematically
        modeling the aquifer.  (31, 33)

        Model representation - a conceptual, mathematical, or physical depiction of
        a field or laboratory system. A conceptual  model describes the present
        condition of the  system. To make predictions of future behavior: it is
        necessary to develop a dynamic model, such as physical scale models, analog
        models, or mathematical models.  Laboratory sand tanks simulate ground-
        water flow directly. The flow of ground-water can be implied by using an
        electrical analog model. Mathematical  models, including analytical, finite
        difference, and finite  element  models are more widely used because they are
        easier to develop and manipulate.  (4,10)
 Page 26

-------
Assessment Framework
           OSWER Directive #9029.00
Glossary of Technical Terms
        Modeling objectives - the information that the modeling application is
        expected to provide, so that management can evaluate potential courses of
        action.  (4)                           ,

        Numerical model -  a mathematical model that allows the user to let the
        controlling parameters vary in space and time, enabling detailed
        replications of the complex geologic and hydrologic conditions existing in
        the field. Numerical models require fewer restrictive assumptions and are
        potentially more realistic and adaptable than analytical models, but provide
        only approximate solutions at discreet locations and moments in time for
        the governing differential equations. (3, 16, 21)

        Peer review - a process by which a panel or individual is charged to review
        and compare the results of modeling efforts and to assess the importance
        and nature of any differences which are present.  The review may examine,
        for example, the scientific validity of the model, the mathematical code,
        hydrogeological/chemical/biological conceptualization, adequacy of data,
        and the application of the model to a specific site. (3, 21)

        Performance testing - determining for the  range of expected uses, the
        efficiency of the model in terms of the accuracy obtained versus the human
        and computer resources required by comparing model results with
        predetermined benchmarks.  (30)

        Porosity - total volume of void space divided by the total volume of porous
        material. The term;"effective porosity" is related. It is the total volume of
        interconnected void space divided by the  total volume of porous material.
        Effective porosity is used to compute average linear ground-water velocity.
        (3,10)

        Postaudit - comparison of  model predictions to the actual outcome
        measured in the field.  Used to determine the success of a model application
        as well as the acceptability of the model itself.  (21)

        Potentiometric surface - a surface that represents the level to which water
        will rise in tightly cased wells.  The water table is a particular potentiometric
        surface for an unconfined aquifer. (10)

        Predictive sensitivity analysis - a procedure to quantify the effect of
        uncertainty in parameter values on the prediction.  Ranges and estimated
        future stresses are simulated to examine the impact on the model's
        prediction.  (4)

        Reliability - the probability that a model will satisfactorily perform its
        intended function under given circumstances. It is the amount of credence
                                                                      Page 27

-------
Assessment Framework
           OSWER Directive #9029.00
Glossary of Technical Terms
        placed in a result.  Measures of reliability include peer  review of model
        theory and code; evaluation of the model results against laboratory tests,
        analytic solutions, or other well accepted modes; field testing; and user
        acceptability.  (23,30)

        Remediation - long-term action that stops or substantially reduces and
        prevents future migration of a release or threat of hazardous substances that
        are a serious but not an immediate threat to public health, welfare, or the
        environment.  (25)

        Residuals - the differences between field measurements at calibration points
        and simulated values.  (13)

        Sources and sinks - gain or loss of water or contaminants from the system.
        In a ground-water flow  system typical examples are pumping or injection
        wells. (3,21)

        Specific yield - quantity of water that a unit volume of aquifer, after being
        saturated, will yield by gravity (expressed as a ratio or percentage of the
        volume of the aquifer).  (23)

        Surface water bodies - all bodies of water on the surface of the earth.  (23)

        Transmissivity - the rate at which ground-water of a prevailing density and
        viscosity is transmitted through a unit width of an aquifer or confining bed
        under a unit hydraulic gradient/ It is a function of the properties of the
        liquid, porous media, and the thickness of the porous media. Often   •
        expressed as the product of the hydraulic conductivity and the full saturated
        thickness of the aquifer.  (1, 22)

        Uncertainty analysis -  process to identify uncertainties in model input
        parameters and conceptual assumptions, and the implications of these on
        the uncertainty in model predictions, including potential impacts on the
        decisions which will be made based on these predictions. (26)

        Verification study - consists of the verification of governing equations
        through laboratory or field tests, the verification of model code through
        comparison with other models or analytical solutions, and the verification
        of the model through tests independent of the model calibration data. (4, 7,
        30)

        Water budget - the sources and outflow of water to the system, which may
        include ground-water recharge from precipitation, overland flow, recharge
        from and discharge to surface water bodies, springflow, evaportranspiration,
        or pumping.  (4)
 Page 28

-------
Assessment Framework
                                                             OSWER Directive #9029.00
                                                                    Sources
Sources

Definitions and endnotes are directly drawn from and based upon the following
sources:

1.   Allaby, Ailsa and Michael. 1990. The Concise Oxford Dictionary of Earth
    Sciences.  Oxford: Oxford University Press.

2.  ' American Society of Testing and Materials (ASTM).  (No date.)  "Guide for
    Application of a Ground-Water Flow Model to a Site-Specific Problem." Draft
    ASTM Standard Section D-18.21.10.

3.   Anderson, Mary P. 1992.  Based wholly or in part on written comments
    provided on the initial draft version of the glossary.

4.   Anderson, Mary P., and William W. Woessner.  1992a.  Applied Groundwater
    Modeling - Simulation of Flow and Advective Transport. New York:
    Academic Press, Inc.

5.   Anderson, Mary P., and William W. Woessner.  1992b.  "The Role of the
    Postaudit in Model Validation." Advances In Water Resources.

6.   California Department of Health Services (CADHS). (No date.) "Standards for
    Mathematical Modeling of Ground Water Flow and Contaminant Transport at
    Hazardous Waste Sites." Chapter 4, Volume 2 of Scientific and Technical
    Standards for Hazardous Waste Sites-Draft.

7.   Belgin, Milovan S.  1987.  Testing, Verification, and Validation of Two-
    Dimensional Solute Transport Models.   Golden, Colorado: International
    Ground Water  Modeling Center, Colorado School of Mines.

8.   Bond, P., and S. Hwang.  1989. "Selection Criteria for Mathematical Models
    Used in Exposure Assessments: Ground Water Models." Office of Health and
    Environmental Assessment (OHEA1 Manual. EPA/600/2-89/028. U.S.
    Environmental Protection Agency.

9.   Franke, O.L., T.E. Reilly, and G.D. Bennett. 1984.  Definition of Boundary and
    Initial Conditions in the Analysis of Saturated Ground-Water Flow Systems -
    An Introduction. Open-File Report 84-458. Reston, Virginia: U.S. Geological
    Survey.

10. Fetter, C.W. 1988. Applied Hydrogeology - 2nd Edition.  Columbus:  Merrill
    Publishing Company.
                                                                     Page 29

-------
Assessment Framework
                                                            OSWER Directive #9029.00
                                                                   Sources
11.  Freeze, R. Allen, and John A. Cherry. 1979.  Groundwater. Englewood, New
    Jersey: Prentice-Hall.

12.  Freeze, Allen, Joel Massman, Leslie Smith, Tony Sperling, and Bruce James.
    1990. "Hydrogeological Decision Analysis."   Ground Water 28(5).

13.  Golden Software, Inc.  September 1990. Surfer Version 4 Reference Manual.
    Golden, Colorado.

14.  Huyakorn, P.S., and G.F. Finder. 1983.  Computation Methods in Subsurface
    Flow. New York: Academic Press.

15.  Istok, Jonathan. 1989.   Groundwater Modeling by the Finite Element Method.
    Washington, D.C.:  American Geophysical Union.

16  Javandel, Iraj, Christine Doughty, and Chin-Fu Tsang.  1984. Ground Water
    Transport: Handbook of Mathematical Models. Washington, D.C.:  American
    Geophysical  Union.

17.  Kezsbom, A., and A.V. Goldman. 1991. "The boundaries of groundwater
    modeling under the law; Standards for excluding speculative expert
    testimony." Tort and Insurance Law Tournal. Vol. XXVII, No.l.

18.  Konikow, Leonard F., and John D. Bredehoeft.  1992. "Ground-water Models
    Cannot Be Validated." Advances in Water Resources 15.

19.  Kruseman, G.P., and N.A. de Ridder.  1990.  Analysis and Evaluation of
    Pumping Test Data. The Netherlands:  International Institute for Land
    Reclamation and Improvement.

20.  Lohman, S.W.  1972.  Definitions of Selected Ground-Water Terms - Revisions
     and Conceptual Refinements. Geological Survey Water-Supply Paper 1988.
    Washington, D.C.:  U.S. Government Printing Office.

21.  National Research  Council (NRC)  Committee on Ground-Water Modeling
     Assessment.  1990.  Ground Water Models. Scientific and Regulatory
     Applications. Washington, D.C.: National Academy Press.

22.  The Ohio State University, Department of Geological Sciences.  1991.  Capzone
     Users Manual.  Columbus, Ohio.

23.  Parker, Sybil P.  1989.  McGraw-Hill Dictionary  of Scientific and Technical
     Terms.   Fourth Edition. New York:  McGraw-Hill Book Company.
 Page 30

-------
Assessment Framework
OSWER Directive #9029.00
       Sources
24.  Subsurface-Water Glossary Working Group, Ground Water Subcommittee,
     Interagency Advisory Committee on Water Data.  1989.  Subsurface-Water and
     Solute Transport Federal Glossary of Selected Terms.

25.  U.S. Environmental Protection Agency (USEPA) Office of Public Affairs.  1988.
     Glossary of Environmental Terms and Acronym List. Washington, D.C.

26.  U.S. Environmental Protection Agency (USEPA) Science Advisory Board. 1993.
     Review of the Assessment Framework for Ground-water Model Applications.
     Washington, D.C.

27.  U.S. Environmental Protection Agency (USEPA).  1992. Ground-Water
     Modeling Compendium - Draft.  Washington, D.C.

28.  U.S. Environmental Protection Agency (USEPA).  1981.  Technical Assistance
     Team (TAT) Contract Users Manual. Washington, D.C.

29.  U.S. Office of Technology Assessment (USOTA). 1982. Use of Models for
     Water Resources of the United States.  Washington, D.C.:  U.S. Government
    ' Printing Office.

30.  van der Heijde, Paul, KM., and O.A. Elnawawy. 1992. Quality Assurance and
     Quality Control in the Development and Application of Groundwater Models.
     EPA/600/R-93/011.  U.S. Environmental Protection Agency Office of Research
     and Development.

31.  van der Heijde, Paul, K.M., Aly I. El-Kadi, and Stan A. Williams,  1988.    :
     Groundwater Modeling: An Overview and Status Report.  EPA/600/2-89/028.
     U.S. Environmental Protection Agency.

32.  van der Heijde, Paul K.M., and Richard A. Park. 1986. Ground-water Modeling
     Policy Study Group Report. Report for the U.S. EPA Office of Research and
     Development.  Golden, Colorado: International Ground Water Modeling
     Center, Colorado School of Mines.

33.  Wang, Herbert P., and Mary P. Anderson.  1982. Introduction to Groundwater
     Modeling. Finite Difference and Element Methods. San Francisco, California:
     W.H. Freeman and Company.
                                                                    Page 31

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
To Ground-Water Modeling
EPA Publications Related to Ground-Water Modeling

                     CONTAMINANT FATE AND TRANSPORT

USEPA. Grove, D.B., and J. Rubin. 1976.  Transport and Reaction of Contaminants
    in Ground Water Systems, Proceedings of the National Conference on Disposal
    of Residues on Land. Office of Research and Development, pages 174-178.

USEPA. 1989. Determining Soil Response Action Levels Based on Potential
    Contaminant Migration to Ground Water:  A Compendium of Examples.
    EPA/540/2-89/057.

USEPA. 1989. Laboratory Investigations of Residual Liquid Organics from Spills,
    Leaks, and Disposal of Hazardous Wastes in Groundwater. EPA/600/6-90/004.

USEPA. Schmelling, Stephen, et al.  1989.  Contaminant Transport in Fractured
  •  Media: Models for Decision Makers: Superfund Ground Water Issue. Ada,
    Oklahoma.

USEPA. 1990. Subsurface Contamination Reference Guide. EPA/540/2-90/011.

USEPA. 1990. Groundwater, Volume I: Ground Water and Contamination.
    EPA/625/6-90/016a.

                                  DNAPLs

USEPA. 1992. OSWER Directive No. 9355.4-07.  Estimating the Potential for
    Occurrence of DNAPL at Superfund Sites.

USEPA. 1992. OSWER Directive No. 9283.1-06. Considerations in Ground Water
    Expediation at Superfund Sites and RCRA Facilities — Update.

USEPA. (No date.)  Dense Nonaqueous Phase Liquids - A Workshop Summary.

                      GROUND-WATER ISSUE PAPERS

USEPA. Puls, R.W., and M.J.  Barcelona.  1989. Ground Water Sampling for Metals
     Analysis. EPA/540/4-89/001.

USEPA. Lewis, T.Y., R.L. Crockett, R. L. Siegrist, and K. Zarrab. 1991. Soil Sampling
     and Analysis for Volatile Organic Compounds.  EPA/450/4-91/001.
 Page 32

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
to Ground-Water Modeling
USEPA. Breckenridge, R.P., J.R. Williams, and J.F. Keck.  1991. Characterizing Soils
    for Hazardous Waste Site Assessments. EPA/540/4-91/003.

USEPA. Schmelling,  S.G., and R.R. Ross. 1989. Contaminant Transport in
    Fractured Media: Models for Decision Makers. EPA/540/4-89/004.

USEPA. Ruling, S.G.  1989. Facilitated Transport. EPA/540/4-89/003.

USEPA. Piwoni, M.D., and  J.W. Keeley. 1990.  Basic Concepts of  Contaminant
    Sorption at Hazardous Waste Sites. EPA/540/4-90/053.

USEPA. Huling, S.G., and J.W. Weaver.  1991. Dense Nonaqueous Phase Liquids.
    EPA/540/4-91/002.

USEPA. Keely, J.F. 1989. Performance Evaluations of Pump-and Treat
    Remediations. EPA/540/4-89/005.

USEPA. Sims, J.L., J.M. Suflita,  and H.H. Russell. 1991. Reductive Dehalogenation
    of Organic Contaminants in Soils and Ground Water. EPA/540/4-90/054.

USEPA. Russell, H.H., J.E. Matthews, and G.W. Sewell.  1992.  TCE Removal from
    Contaminated Soil and Ground Water. EPA/540/S-92/002.

USEPA. Palmer, C.D., and W. Fish. 1992. Chemical Enhancements to Pump-and-
    Treat Remediation.  EPA/540/S-92/001.

USEPA. Sims, J.L., J.M. Suflita, and H.H. Russell. 1992.  In-Situ Bioremediation of
    Contaminated Ground Water.  EPA/540/S-92/003.

USEPA. 1990.  Colloidal-Facilitated Transport of Inorganic Contaminants in Ground
    Water: Part 1, Sampling. EPA/600/M-90/023,

USEPA.  Puls, R.W., R.M. Powell, D.A. Clark, and C.J. Paul. 1991.  Facilitated
    Transport of Inorganic Contaminants in Ground Water:  Part 2, Colloidal
    Transport.  EPA/600/M-91/040.

            GROUND-WATER MONITORING AND WELL DESIGN

USEPA. Denit, Jeffrey.  1990. Appropriate Materials  for Well  Casing and Screens in
    RCRA Ground Water Monitoring Networks.

USEPA. Aller; L., T.W.  Bennett, G.  Hackett, J.E. Denne, and R.J. Petty.  1990.
    Handbook of Suggested Practices for the Design and Installation of Ground
    Water Monitoring Wells. EPA/600/4-89/034.
                                                                   Page 33

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
To Ground-Water Modeling
USEPA. 1992. Chapter Eleven of SW-846, Ground Water Monitoring. Office of
    Solid Wastes, Washington, DC.  Pre-Publication.

                             HYDROGEOLOGY

USEPA. 1985. Issue Papers in Support of Groundwater Classification Guidelines.
    EPA/440/6-85/001.

USEPA. 1986. Criteria for Identifying Areas of Vulnerable Hydrogeology Under
    RCRA.  Appendix D:  Development  of Vulnerability Criteria Based on Risk
    Assessments.  Office of Solid Waste  and Emergency Response, Washington,
    DC.

USEPA. 1990. A New Approach and Methodologies for Characterizing the
    Hydrogeologic Properties of Aquifers. EPA/600/2-90/002.

USEPA. Molz, F.J., G. Oktay, and J.G. Melville. 1990. Measurement of Hydraulic
    Conductivity Distributions:  A Manual of Practice. Auburn University.

USEPA. 1990. Ground Water, Volume II: Methodology. EPA/625/6-90/016b.

                                MODELING

USEPA. Pettyjohn, W.A., D.C. Kent, T.A. Prickett, and H.E. LeGrand.  (No date.)
    Methods for the Prediction of Leachate Plume Migration and Mixing. Office of
    Research Laboratory, Cincinnati.

USEPA.  Bond, F., and S. Hwang.  1988. Selection Criteria for Mathematical Models
    Used in Exposure Assessments:  Ground Water Models. EPA/600/2-89/028.

USEPA.  1988. OSWER Directive No. 9355.0-08.  Modeling Remedial Actions at
    Uncontrolled Hazardous Waste Sites.

USEPA.  Bear, J., M. Geljin, and R. Ross.  1992. Fundamentals of Ground Water
     Modeling for Decision Makers. EPA/540/S-92/005.

USEPA.  van der Heijde, Paul K.M., Aly I. El-Kadi, and Stan A. Williams.  1988.
     Groundwater Modeling:  An Overview and Status Report. EPA/600/2-89/028.

USEPA.  van der Heijde, Paul K.M., and O.A. Elnawawy. 1992.  Quality Assurance
     and  Quality Control in the  Development and Application of Groundwater
     Models. EPA/600/R-93/011. Office of Research and Development.
 Page 34

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
to Ground-Water Modeling
                                   RCRA

USEPA. 1984. OSWER Directive No. 9504.01-84.  Enforcing Groundwater
    Monitoring Requirements in RCRA Part B Permit Applications.

USEPA. 1984. OSWER Directive No. 9481.06-84.  Clarification of the Definition of
    Aquifer in 40 CFR 260.10.

USEPA. 1984. OSWER Directive No. 9481.02-84.  ACL Demonstrations, Risk Levels,
    and Subsurface Environment.

USEPA. 1985. OSWER Directive No. 9481.02-85.  Ground Water Monitoring Above
    the Uppermost Aquifer.

USEPA. 1985. OSWER Directive No. 9481.05-85.  Indicator Parameters at Sanitary
   • Landfills.

USEPA. 1985. OSWER Directive No. 9931.1. RCRA Ground "Water Monitoring   -
    Compliance Order Guidance.

USEPA. 1985. OSWER Directive No. 9476.02-85.  RCRA Policies on Ground Water
    Quality at Closure.      '                                          .

USEPA. 1985. OSWER Directive No. 99050.0.  Transmittal of the RCRA Ground ,
    Water Enforcement Strategy.

USEPA. 1986. OSWER Directive No. 9950.2. Final RCRA Comprehensive Ground
    Water Monitoring Evaluation (CME) and Guidance.

USEPA. 1986. Leachate Plume Management. EPA/540/2-85/004.

USEPA. 1987. OSWER Directive No. 9950.1. RCRA Ground Water Monitoring
    Technical Enforcement Document.

USEPA. 1987. OSWER Directive No. 9481.00-10.  Implementation Strategy for
    Alternate Concentration Levels.

USEPA. 1988. OSWER Directive No. 9476.00-10.  Ground Water Monitoring at
    Clean Closing Surface Impoundment and  Waste Pile Units.
                                                                   Page 35

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
To Ground-Water Modeling
USEPA.  1988. ,O§WER Directive No. 9950.3.  Operation and Maintenance
     Inspection Guide (RCRA) Ground Water Monitoring Systems.
                            RISK ASSESSMENT

USEPA. Bond, F., and S. Hwang.  1988.  Selection Criteria for Mathematical
     Models Used in Exposure Assessments: Ground Water Models. EPA/600/8-
     88/075.

USEPA. 1988. Superfund Exposure Assessment Manual.  EPA/540/1-88/001.
USEPA. 1989. Risk Assessment Guidance for Superfund, Vol. I: Human Health
     Evaluation Manual.  EPA/540/1-89/002.

USEPA. 1989. Risk Assessment Guidance for Superfund, Vol. II: Environmental
     Evaluation Manual.  EPA/540/1-89/001.

                     SAMPLING AND DATA ANALYSIS

Bauer, K.M., W.D. Glauz, and J.D. Flora. 1984. Methodologies in Determining
     Trends in Water Quality Data.

•USEPA. 1986. NTIS No. PG-86-137-304.  Practical Guide for Ground Water
     Sampling.                     .

USEPA. 1987. OSWER Directive No. 9355.0-14.  A Compendium of Superfund Field
     Operation Methods, Volumes  1 and 2.

USEPA. 1988. Field Screening Methods Catalog:  User's Guide. EPA/540/2-88/005.

USEPA. 1989. Data Quality Objectives for Remedial Response Activities:
     Development Process.  EPA/540/G-87/003.

USEPA. 1991. OSWER Directive No. 9355.4-04FS. A Guide:  Methods for
     Evaluating the Attainment of Clean-Up Standards for Soils and Solid Media.

USEPA. 1991. OSWER Directive No. 9360.4-06.  Compendium of ERT Ground
     Water Sampling Procedures.

              REMEDIAL INVESTIGATION/FEASIBILITY STUDY
 Page 36

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Belated
to Ground-Water Modeling
USEPA.  1988. Guidance on Conducting Remediation Investigation and Feasibility
     Studies (RI/FS) Under CERCLA. EPA/540/G-89/004.

USEPA.  1988. OSWER Directive No. 9283.1-02. Guidance on Remedial Action for
     Contaminated Ground Water at Superfund Sites.

USEPA.  1989. OSWER Directive No. 9355.4-03. Considerations in Ground Water
     Remediation at Superfund Sites.

USEPA.  1989. Example Scenario:  RI/FS Activities at a Site with Contaminated Soil
     and Ground Water.  EPA/540/G-87/004.

USEPA.  1989. OSWER Directive No. 9835.8. Model Statement of Work for a
     Remedial Investigation and Feasibility Study Conducted by a Potentially •
     Responsible Party.

USEPA.  1991. Conducting Remedial Investigation/Feasibility Studies for CERCLA
    . Municipal Landfill Sites.  Office of Emergency Remedial Response,
     Washington, DC.

USEPA.  1991. OSWER Directive No. 9835.3-2a. Model Administrative Order on
     Consent  for Remedial Investigation/Feasibility Study.

USEPA.  1992. Site Characterization for Subsurface Remediation.  EPA/625/4-
     91/026.

                  REMEDIAL DESIGN/REMEDIAL ACTION

USEPA.  1986. OSWER Directive 9355.0-4A.  Superfund Remedial Design and
     Remedial Action Guidance.

USEPA.  1988. OSWER Directive No. 9355.0-08. Modeling Remedial Actions at
     Uncontrolled Hazardous Waste Sites.

USEPA.  1988. OSWER Directive No. 9283.1-02. Guidance on Remedial Action for
     Contaminated Ground Water  at Superfund Sites.

USEPA.  1989. OSWER Directive No. 9355.4-03. Considerations in Ground Water
     Remediation at Superfund Sites.

USEPA.  1990. OSWER Directive No. 9355.0-27FS.  A Guide to Selecting Superfund
     Remedial Actions:
                                                                   Page 37

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
To Ground-Water Modeling
USEPA. 1990.  Guidance on Expediting Remedial Design and Remedial Action.
    EPA/540/G-90/006.

USEPA. 1990.  Guidance on EPA Oversight of Remedial Designs and Remedial
    Actions Performed by Potentially Responsible Parties.  Interim Final.
    EPA/540/G-90/001.

USEPA. Ross, R. (No date.) General Methods for Remedial Operations
    Performance Evaluations.

USEPA. 1990.  OSWER Directive No. 9355.4-01FS. Guide on Remedial Actions at
    Superfund Sites with PCB Contamination.

USEPA. 1990.  OSWER Directive No. 9833.0-2b. Model Unilateral Order for
  •  Remedial Design and Remedial Action.

USEPA. 1991.  OSWER Directive No. 9835.17. Model CERCLA RD/RA Consent
    Decree.
                           RECORD OF DECISION

USEPA. 1990.  OSWER Directive No. 9283.1-03.  Suggested ROD Language for
    Various Ground Water Remediation Options.

USEPA. 1991.  OSWER Directive No. 9355.3-02FS-3.  Guide to Developing
    Superfund No Action, Interim Action, and Contingency Remedy  RODs.

USEPA. 1991.  OSWER Directive No. 9355.7-02. Structure and Components of Five
    Year Reviews.

                          CLEANUP  STANDARDS

USEPA. 1990.  OSWER Directive No. 9234.2-11FS. ARARs Q's and A's: State
    Ground Water Antidegradation Issues.

USEPA. 1990.  OSWER Directive No. 9234.2-09FS. ARARs Q's and A's: Compliance
    with Federal Water  Quality Criteria.

USEPA. 1990.  OSWER Directive No. 9234.2-06FS. CERCLA Compliance with Other
    Laws Manual: CERCLA Compliance with the Clean Water Act (CWA) and the
    Safe Drinking Water Act (SDWA).

USEPA. 1991. OSWER Directive No. 9355.4-04FS. A Guide:  Methods for
     Evaluating the Attainment of Cleanup Standards for Soil and Solid Media.
 Page 38

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
to Ground-Water Modeling
                     PUMP AND TREAT REMEDIATION

USEPA. 1989. Evaluation of Ground Water Extraction Remedies, Volume I:
    Summary Report.  EPA/540/2-89/054a.

USEPA. 1989. Evaluation of Ground Water Extraction Remedies, Volume II: Case
    Studies 1-19. EPA/540/2-89/054b.
USEPA. 1989. Evaluation of Ground Water Extraction Remedies, Volume III:
    General Site Data, Data Base Reports. EPA/540/289/054c.

USEPA. 1989. OSWER Directive No. 9355.0-28.  Control of Air Emissions from
    Super fund Air Strippers at Superfund Ground Water Sites.

USEPA. Mercer, J.W., D.C. Skipp, and D. Giffin.  1990.  Basics of Pump and Treat
    Ground Water Remediation Technology. EPA/600/8-90/003.

USEPA. Saunders, G.L. 1990.  Comparisons of Air Stripper Simulations and Field
    ' Performance Data. EPA/450/1-90/002.

USEPA. 1989.  Forum on Innovative Hazardous Waste Treatment Technologies:
    Domestic and International. EPA/540/2-89/056.

USEPA. 1990.. OSWER Directive No. 9355.0-27FS.  A Guide to Selecting Superfund
    Remedial Actions.

                          IN SITU REMEDIATION

Middleton, A.C., and D.H. Killer. 1990. In Situ Aeration of Ground Water:  A
    Technology Overview.

USEPA. 1990. OSWER Directive No. 9355.0-27FS.  A Guide to Selecting Superfund
    Remedial Actions.

USEPA. 1990.  Emerging Technologies:  Bio-Recovery Systems Removal and
    Recovery of Metal Ions from Ground Water. EPA/540/5-90/005a.

USEPA. 1989.  Forum on Innovative Hazardous Waste Treatment Technologies:
    Domestic and International. EPA/540/2-89/056.

                     LANDFILLS AND LAND DISPOSAL

USEPA. Kirkham, R.R., et al. 1986. Estimating Leachate Production from Closed
    Hazardous Waste Landfills. Project Summary. • Cincinnati, Ohio.
                                                                   Page 39

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
To Ground-Water Modeling
USEPA. Mullen, H., and S.I. Taub. 1976.  Tracing Leachate from Landfills, A
    Conceptual Approach, Proceedings of the National Conference on Disposal of
    Residues on Land.  Environmental Quality Symposium, Inc.  pages 121-126.

USEPA. 1990. OSWER Directive No. 9481.00-11. Status of Contaminated Ground
    Water and Limitations on Disposal and Reuse.

Federal Register, Part H, 40 CFR Parts 257 and 258, Wednesday, October 9,1991. Solid
    Waste Disposal Facility Criteria:  Final Rule.

USEPA. 1989. NTIS No. PB91-921332/CCE. Applicability of Land Disposal
    Restrictions to RCRA and CERCLA Ground Water Treatment Reinjection,
    Superfund Management.

USEPA. 1991. Conducting Remedial Investigation/Feasibility Studies for CERCLA
  .  Municipal Landfill Sites. Office of Emergency Remedial Response,
    Washington, D.C.

             MISCELLANEOUS GROUND-WATER PUBLICATIONS

USEPA. 1986. Background Document Groundwater Screening Procedure.  Office of
    Solid Waste.

USEPA. 1990. Continuous Release - Emergency Response Notification System and
    Priority Assessment Model: Model Documentation, Office of Emergency and
    Remedial Response, Washington, D.C.

USEPA. 1991. Protecting the Nation's Ground Water: EPA's Strategy for the 1990s;
    The Final Report of the EPA Ground Water Quality Task Force.  Office of the
    Administrator, Washington, D.C.

USEPA. 1990. ORD Ground Water Research Plan:  Strategy for 1991 and Beyond.
    EPA/9-90/042.

                        DRINKING WATER SUPPLY

USEPA. 1988. Guidance on Providing Alternate Water Supplies.  EPA/540/G-
    87/006.

                                CATALOGS

USEPA. 1992. Catalog of Superfund Program Publications. EPA/540/8-91/014.
 Page 40

-------
Assessment Framework
         OSWER Directive #9029.00
 EPA Publications Related
to Ground-Water Modeling
To obtain these documents, contact the EPA Regional Libraries, the EPA Regional
Records Center, or the Center for Environmental Research Information (513-569-
7562).
                                                                       Page 41

-------

-------