HR: 1340h
AN: H23C-1140    [Abstracts]
TI: Sampling Uncertainties for Ensemble Forecast Verification Measures
AU: * Bradley, A
EM: allen-bradley@uiowa.edu
AF: University of Iowa, IIHR--Hydroscience & Engineering, 107 C. Maxwell Stanley Hydraulics Laboratory, Iowa City, IA 52242 United States
AU: Schwartz, S
EM: s.schwartz1@csuohio.edu
AF: Cleveland State University, Center for Environmental Science, Technology, and Policy, 2121 Euclid Avenue, MC219 , Cleveland, OH 44115 United States
AB: Verification of forecasts is an essential first step for their operational use in decision-making. Forecast verification is carried out using a verification data set, which contains a record of forecasts and subsequent observations. A comparison of the forecasts with the observations is then made to assess forecast quality. For some forecasting systems, the archive of operational forecasts may be sufficient to make a comparison. For others, it may be necessary to reconstruct forecasts for the past, where a longer record of observations is available (a technique also known as hindcasting). Regardless of the approach, forecast quality measures evaluated during forecast verification are {\it sample estimates}, and are often based on relatively small sample sizes. For instance, with long-range ensemble streamflow forecasts, there is only one forecast made each year (for a given lead time and forecast period). Since most flow records have about 50 years or fewer of observations, even a reconstructed forecast sample is severely limited. In this presentation, we examine the sampling uncertainty of distributions-oriented (DO) forecast quality measures for probability forecasts from ensemble forecasting systems. With the DO verification approach, the correspondence between forecasts and observations is modeled explicitly by their joint probability distribution. Aspects of forecast quality of interest in verification are derived from the joint distribution model. Sampling theory is used to develop exact or approximate bias and standard error estimates for derived forecast quality measures. We also explore various statistical modeling approaches, which are needed to completely model the joint distribution, to estimate biases and standard errors for certain measures. The uncertainty estimators are evaluated for several prototype forecasts using Monte Carlo simulation. The estimators are then applied to long-range streamflow forecasts from the National Weather Service's (NWS) Advanced Hydrologic Prediction System (AHPS) for the Des Moines River basin. The results illustrate how sampling uncertainty affects inferences on forecast quality for probability distribution forecasts from ensemble systems.
UR: http://www.iihr.uiowa.edu/$\sim$verification/
DE: 1812 Drought
DE: 1821 Floods
DE: 1833 Hydroclimatology
DE: 1869 Stochastic processes
SC: Hydrology [H]
MN: 2004 AGU Fall Meeting