Watershed Model Phase 5 Peer Review Group Model Review Questions Posed May, 2005 Introduction Evaluating a model used for environmental regulatory purposes must be done within the context in which the model will be applied and questions posed to a model review group must also be framed in this context. The Phase 5 Watershed Model is intended to be used for regional modeling at tributary or subtributary scales to determine the nutrient and sediment load reductions, or caps, needed to be achieved and maintained in order to satisfy tidal water quality standards now being prepared by the States of Virginia and Maryland. Maryland intends to use the Phase 5 Watershed Model for local TMDLs as well, to harmonize local TMDLs with nutrient and sediment reductions needed to achieve Chesapeake Bay water quality standards. The following questions are intended as guidance, and by no means intended to constrain the model review's range of topics or questions. Four general questions are posed with some specific sub-questions suggested on the following pages. Phase 5 Watershed Model Peer Review Overarching Questions: Question 1: Is the model structure sufficient for the management purposes with regard to segmentation, land uses, HSPF modifications, and ancillary software? Question 2: Are the data inputs sufficient to support management decisions with regard to meteorology, nutrient inputs, land use, BMPs, septic systems, and point sources. Question 3: Is the Phase 5 Model sufficiently calibrated for management purposes? Evaluate the calibration data, criteria, methodology, robustness, and reproducibility. Question 4: How can the Phase 5 Model best be used in management? What are appropriate questions and on what scale can the Phase 5 Model be used for TMDLs? ------- Phase 5 Watershed Model Peer Review Detailed Questions: Overarching Question 1: Is the model structure sufficient for the management purposes with regard to segmentation, land uses, HSPF modifications, and ancillary software? Sub-questions: • Are the simulated land uses in the Phase 5 watershed model sufficient to answer the management questions and are they well characterized? • Are the appropriate modules available in HSPF being used to address the management questions? • Are the modifications made to the standard HSPF code reasonable? Are there other modifications that should be pursued? • Is the ancillary software necessary and sufficient for the purposes of the watershed model? Are there improvements to the methodology? • In an HSPF simulation, river reaches are simulated as completely mixed reactors. Given constraints of monitoring data available for calibration and computational constraints, what is the optimal scale for river reach segments? Similarly, are the land segmentation scales appropriate to the level of data and the management questions? • Evaluate the efficacy of the edge-of-field (EoF) to edge-of-stream (EoS) transport of sediment methodology to estimate ultimate sediment load delivery from unit area field scale HSPF land segment simulations to simulated river reaches. • Evaluate the efficacy of using a variant of the EoF to EoS transport of sediment methodology to estimate ultimate nitrogen and phosphorus load delivery from unit area field scale HSPF land segment simulations to simulated river reaches. • Is there any available scientific evidence to suggest changes to the model design and/or key parameters and assumptions prior to its use for regulatory purposes? • What key research is necessary to refine or improve the model and/or the data bases upon which it relies? • Is the CBP documentation of the code modifications and the ancillary software clear and adequate? • To what extent does river order effect key mechanisms, and by association calibration rules, in a detailed watershed simulation, i.e., autotrophic high order river mechanisms versus heterotrophic low order river mechanisms. • To what extent would an explicit simulation of periphyton, including high flow periphyton ------- scour with subsequent biomass substrate colonialization and biomass recovery, explicit simulation of benthic light regimes with stream order, etc. improve simulation performance? • To what extent would an explicit simulation of riverine macrophytes improve simulation performance? Overarching Question 2: Are the data inputs sufficient to support management decisions with regard to meteorology, nutrient inputs, land use, BMPs, septic systems, and point sources. Sub-questions: • Provide an evaluation of the precipitation model performance. What are the best metrics to use when evaluating precipitation models against observed data? • What is the adequacy of the sediment and nutrient input data in terms of quality, quantity, and spatial and temporal applicability taking into account the regulatory objectives of the model? • Is the quantity of data sufficient to address the spatial, temporal, and hydrologic variability? What statistical analyses were performed and are they appropriate? • Do additional data need to be collected and for what purpose? • Is the CBP documentation of the data inputs clear and adequate? Overarching Question 3: Is the Phase 5 Model sufficiently calibrated for management purposes? Evaluate the calibration data, criteria, methodology, robustness, and reproducibility. Sub-questions: • What is the adequacy of the sediment and nutrient calibration data in terms of quality, quantity, and spatial and temporal applicability taking into account the regulatory objectives of the model? • The most basic test of a model's adequacy is to understand how well its results compare with real world measurements. Are the criteria that have been used to assess model performance appropriate? Is the model sufficiently calibrated to observed data? • How accurate can the model be expected to perform? Does the model exhibit any overall bias throughout the range of its predictions? Bias is an important test of the model's formulation since intrinsic system uncertainty is not present. • Was the overall calibration methodology appropriate? Did it provide a robust and reproducible calibration? Did it arrive at a reasonable calibration? • Did the data bases used in the performance evaluation provide an adequate test of the model in terms of applicability to the modeling niche? ------- • Evaluate the methodology of the large scale transfer calibration methodology from above fall line calibrated land segments predominately in the Piedmont to uncalibrated below fall line regions of the coastal plain. • How well does the model output quantify the overall uncertainty resulting from limitations/simplifications in its design; use of standard assumptions; availability of supporting data; etc.? • Is the Phase 5 documentation of the calibration clear and adequate? Overarching Question 4: How can the Phase 5 Model best be used in management? What are appropriate questions and on what scale can the Phase 5 Model be used for TMDLs? Sub-questions: • What kinds of nutrient and sediment input data are required to apply the model in scenario mode? To what extent are these data available and what are the key nutrient and sediment data gaps? • A retrospective analysis of the "big picture" may sometimes reveal insights that an analysis of individual components of a model may miss. Does the model satisfy its intended scientific and regulatory objectives as both a regional and local TMDL model for nutrients and sediment? • Is the CBP documentation of the scenario operations clear and adequate? • Is the Community Model documentation clear and comprehensive? In what ways can the Community Model documentation be improved? • How do we improve the utility of the Phase 5 Community Watershed Model for the following communities of potential users: regional Chesapeake Bay-wide decision makers? TMDL analysts and regulatory community? research/academic community? ------- |