HR: 13:40h
AN: H12G-01    [PDF]
TI: A Perspective on Simulations of Environmental Systems and Prediction Uncertainty
AU: * Hill, M C
EM: mchill@usgs.gov
AF: U.S. Geological Survey, 3215 Marine St., Boulder, CO 80303 United States
AU: Tiedeman, C R
EM: tiedeman@usgs.gov
AF: U.S. Geological Survey, 345 Middlefield Rd, MS 496, Menlo Park, CA 94025 United States
AB: Prediction uncertainty is the likely discrepancy between model predictions and the actual, unrealized system responses. Contributions to uncertainty include anything that causes inaccurate predictions. This can include numerical model solution error and capability limitation, data error and deficiency, and conceptual model error. For example, using conceptual models to build simulations forces ideas about system behavior that are often vague and(or) wrong to be tested against measurements. Closer correspondence between the simulation and measurements often indicates the model more accurately represents a system. However, when models are calibrated, predictive capability can be degraded by fitting measurements too closely. This can occur when the model is overparameterized and close model fit is achieved by fitting measurement and other errors. The connection between such errors and prediction accuracy is clear, and thorough evaluation of such errors and the possibility of overfitting are critical. This is especially true for stochastic and Bayesian methods applied to models with many parameters, for which overfitting is controlled using prior information and smoothness constraints that may not be well understood by the modeler. When a reasonably accurate simulation of a system has been achieved through careful model development, calibration, and error evaluation, the simulation itself becomes an invaluable tool for sensitivity analysis, data assessment, and uncertainty evaluation. Three categories of sensitivity and data assessment methods include techniques for identifying (1) the importance of observations important to parameter values (observations that dominate model calibration); (2) parameter values that dominate the predictions; and (3) observations that dominate the predictions. For instance, gradient-based methods such as composite scaled sensitivities, prediction scaled sensitivities and the value of improved information, and the observation-prediction statistic are used to address the three categories, respectively. These local-sensitivity methods assume model linearity, but have found to be useful for nonlinear ground-water models. More computationally intensive methods that do not assume model linearity include variance-based global sensitivity analysis methods for identifying parameters important to predictions, and jackknife and bootstrap methods for identifying observations that dominate predictions. Uncertainty evaluation methods can be categorized as gradient, selective sampling, and random sampling methods. Gradient methods include linear and nonlinear confidence intervals, and are limited to propagating uncertainties related to parameter values. Selective sampling often involves establishing a most probable and one or more worst-case scenarios. Random sampling includes Monte Carlo methods such as Latin-Hypercube sampling, and can produce results similar to nonlinear confidence intervals if only parameter values are sampled and if simulations with poor model fit are omitted. Model development and evaluation are obviously complex endeavors involving a number of steps. To make wise societal decisions based on environmental model predictions, it is important to establish solid methods for evaluating the importance of observations and parameters to predictions and for quantifying prediction uncertainty.
DE: 3210 Modeling
DE: 3220 Nonlinear dynamics
DE: 3230 Numerical solutions
DE: 3260 Inverse theory
SC: Hydrology [H]
MN: 2003 Fall Meeting