GC53A-01 INVITED
The use of the Multi-model Ensemble in Probabilistic Climate Projections
Recent coordinated efforts, in which numerous climate models have been run for a common set of experiments, have produced large datasets of projections of future climate for various scenarios. Those multi-model ensembles sample initial condition, parameter as well as structural uncertainties in the model design, and they have prompted a variety of approaches to quantify uncertainty in future climate in a probabilistic way. This overview presentation outlines the motivation for using multi-model ensembles, briefly discusses the methodologies published so far and compares their results for regional temperature projections. It discusses the challenges in interpreting multi-model results, caused by the lack of verification of climate projections, the problem of model dependence, bias and tuning as well as the difficulty in making sense of an "ensemble of opportunity".
GC53A-02 INVITED
Performance Metrics in the Evaluation and Application of Climate Models
It is difficult to assess the accuracy and uncertainty of model projections of climate change because opportunities for directly testing the models are limited. Climate changes of the past can be simulated by models, but during times of relatively abundant observations (recent decades), the changes are small, and for earlier times when changes were large (paleoclimates), the observations are sparse. Consequently, confidence in model veracity stems not primarily from their ability to simulate climate change, but from their ability to simulate a multitude of observable phenomena comprising present-day climate. Metrics can serve to summarize various aspects of model performance, but the relevance of proposed metrics to predictive capability remains largely unknown. We describe how metrics for climate models differ from those used in evaluating weather prediction models, and suggest that at present it may be better to retain an extensive suite of metrics to characterize model skill. Different models excel in simulating different aspects of climate, but best agreement with observations almost invariably occurs when we form a multi-model mean of simulated fields. Collapsing a suite of metrics to a single "performance index" is possible, but hides information that likely could leave the index vulnerable to misinterpretation. As our understanding of the relationship between skill in simulating present climate and predictive skill improves, however, we expect to work toward defining a reasonably small set of performance indices, each one designed to indicate how suitable a model is for a particular application (e.g., future climate projection, ENSO forecast, drought prediction).
GC53A-03 INVITED
Predictions of climate change utilizing perturbed physics and multi-model ensembles
We present a method to produce probabilistic climate predictions for the coming century, conditional upon different emissions scenarios. The method is built upon ensembles of the Hadley Centre HadCM3 model with perturbations to key parameters (perturbed physics ensembles) and uses a Bayesian statistical technique. The technique seeks to "emulate" the parameter space of HadCM3 based on some prior assumptions about parameter ranges, and then down-weights regions of parameter space based on a comparison of modelled historical mean climate and climate change with observations (accounting for observational uncertainties). The effect of structural uncertainties, not sampled by the perturbed physics approach, are further accounted for by incorporating information from the CMIP3 and CFMIP multi-model ensembles in a term which we call the discrepancy. The method seeks to account for the major uncertainties in feedbacks associated with the atmosphere, surface, ocean, sulphur cycle and terrestrial carbon cycle in a systematic way as well as tracking uncertainties from the statistical components of the method. The resulting probability distribution functions for future climate change provide a benchmark whereby sensitivities to methodological assumptions may be tested and the value of future progress in climate modelling and new observations may be measured. The method is currently being implemented, together with a combined dynamical-statistical downscaling approach, to produce probabilistic predictions for the UK at 25km resolution.
GC53A-04 INVITED
Analyzing Regional Climate Experiments via Multivariate Spatial Models
The North American Regional Climate Change Assessment Program (NARCCAP) seeks to examine the uncertainty in the output of regional climate models and projections of future climate and climate change. At the heart of the program is an ambitious experiment that seeks to use a number of regional climate models (RCMs) with boundary conditions supplied by different atmosphere-ocean general circulation models (GCMs) to produce a wide range of model output over North America. Our goal within this program is to develop statistical methodology to analyze this model output and assess the sources of uncertainty. To that end, we are developing a Bayesian hierarchical framework that is based upon a multivariate spatial model. This allows us to capture the complex distribution of the spatial fields produced by these regional climate models and make inferences about the effects resulting from the GCM/RCM model pairs. In this talk, the methodology will be discussed and examples of the implementation presented.
GC53A-05
Evaluating Significance of Uncertainties in Climate Model Development Against Observational Uncertainty
A statistical approach is presented to select members of an ensemble of climate models so that the ensemble is representative of observational uncertainty. Observational constraints on the choice of different models or versions of a single model can be imposed through quantification of a model's skill to reproduce observations. Whatever measure is used, one may calculate the effects of observational uncertainty on this skill score in order to define a range of acceptability. Estimates of this range will be presented based on the effect of differences in the NCEP and ECMWF reanalysis data products on a variety of measures of model skill. We then use these ranges to evaluate the significance of the improvements that have occurred through several generations of the NCAR atmospheric climate model and the spread in predictions from models that participated in the 2007 IPCC Fourth Assessment Report.
GC53A-06
Regional Climate Projections Utilizing Multimodel Ensembles: Some Stationarity Considerations
A variety of efforts have been made to generate regional climate change projections using ensembles of atmosphere-ocean general circulation models, information from the ensemble being utilized to both project regional changes and estimate uncertainties associated with those projections. A number of methodologies used for this purpose involve differential weighting of the constituent models, typically based on some assessment, or "metric" of model performance with respect to the observed climate. The underlying assumption in such methods is that the characteristics identified by such metrics will continue to differentiate the models as climate changes, and that the assigned weightings will thus remain valid, at least for the time horizon contemplated by the projections generated. While observations of the future are not available to directly test this assumption, it can at least be assessed for different periods in the observed record. In addition, the stability of intermodel relationships can be assessed over the entire time range of available simulations. Here, insights from such assessments are presented, and implications for the generation of multimodel climate projections discussed.
GC53A-07
Applications and Limitations of Using Large State-of-the-Art Climate Ensembles for Risk Management
Climate model output is increasingly being offered to both policy makers and industry in support of detailed risk management and decision making strategies. Ensemble techniques illuminate the impact of some sources of uncertainty in future climate scenarios, and may provide valuable information to risk managers in their attempts to make robust decisions. Ensemble climate experiments are designed to explore variability in model behaviour due to differences in initial conditions, model parameters, and to a more limited extent model structure and other uncertainties. After discussing the strengths and limitations of this approach, an unprecedented range of behaviour is shown to arise in an ensemble of 45,000 General Circulation Model (HaDSM3) climateprediction.net runs. Initial condition uncertainty is shown to play a significant role if this model were to be used for risk management, especially in terms of estimating extremes. In terms of global behaviour, both low (less than 1 degree Celsius) and high (over 16 degrees Celsius) climate sensitivity runs are observed in models versions with comparable performance in 1xCO2 simulations. The data set is produced using a perturbed physics grand ensemble generated by the climateprediction.net experiment, a publically distributed computing experiment. Over 10,000 different model versions (the same structural model with different parameter values) are run, allowing for an assessment of parametric uncertainty. For each of these model versions, an initial condition ensemble is run, providing an estimate of each model versions' climate distribution. A grand ensemble of runs gives the opportunity for a better understanding of the state of climate modeling science. The wide range of behaviour raises important questions of how to interpret climate predictions for policy makers and how state-of-the-art (2007) climate modeling experiments might be related to the Earth's climate. While ensembles contain useful information, the limits on the potential for extracting decision relevant probability distributions from models of limited realism is discussed, and the coherence of attempting to weight empirically inadequate models in the absence of a relevant forecast-verification archive is questioned. Challenges to the evaluation of model performance are shown and the consequences for constraining/weighting ensembles are discussed. http://www.lse.ac.uk/collections/cats/
GC53A-08
Questioning the Relevance of Model-Based Probability Statements on Extreme Weather and Future Climate
We question the relevance of climate-model based Bayesian (or other) probability statements for decision support and impact assessment on spatial scales less than continental and temporal averages less than seasonal. Scientific assessment of higher resolution space and time scale information is urgently needed, given the commercial availability of "products" at high spatiotemporal resolution, their provision by nationally funded agencies for use both in industry decision making and governmental policy support, and their presentation to the public as matters of fact. Specifically we seek to establish necessary conditions for probability forecasts (projections conditioned on a model structure and a forcing scenario) to be taken seriously as reflecting the probability of future real-world events. We illustrate how risk management can profitably employ imperfect models of complicated chaotic systems, following NASA's study of near-Earth PHOs (Potentially Hazardous Objects). Our climate models will never be perfect, nevertheless the space and time scales on which they provide decision- support relevant information is expected to improve with the models themselves. Our aim is to establish a set of baselines of internal consistency; these are merely necessary conditions (not sufficient conditions) that physics based state-of-the-art models are expected to pass if their output is to be judged decision support relevant. Probabilistic Similarity is proposed as one goal which can be obtained even when our models are not empirically adequate. In short, probabilistic similarity requires that, given inputs similar to today's empirical observations and observational uncertainties, we expect future models to produce similar forecast distributions. Expert opinion on the space and time scales on which we might reasonably expect probabilistic similarity may prove of much greater utility than expert elicitation of uncertainty in parameter values in a model that is not empirically adequate; this may help to explain the reluctance of experts to provide information on "parameter uncertainty." Probability statements about the real world are always conditioned on some information set; they may well be conditioned on "False" making them of little value to a rational decision maker. In other instances, they may be conditioned on physical assumptions not held by any of the modellers whose model output is being cast as a probability distribution. Our models will improve a great deal in the next decades, and our insight into the likely climate fifty years hence will improve: maintaining the credibility of the science and the coherence of science based decision support, as our models improve, require a clear statement of our current limitations. What evidence do we have that today's state-of-the-art models provide decision-relevant probability forecasts? What space and time scales do we currently have quantitative, decision-relevant information on for 2050? 2080? http://www.lse.ac.uk/collections/cats/