Introduction

To many people, probability forecasts are still much less familiar than traditional deterministic forecasts. Two issues are often raised as practical problems for the use of probabilities. First, there is a common perception that probability forecasts have no place in the real world, where users need to make hard yes/no decisions. Secondly there is the feeling that probability forecasts are difficult to assess – ‘probability forecasts are never wrong’, the scores are complicated, and different scores tend to show different ‘skill’. As an illustration of this last point, Figure 25.1 shows two examples of the evaluation of probabilistic skill for the ECMWF Ensemble Prediction System (EPS; Buizza, this volume; Palmer et al., 1993; Molteni et al., 1996; Buizza et al., 2003). The ROC skill score (based on the area under the relative operating characteristic (ROC) curve; Richardson, 2000, 2003) shows substantial skill, remaining above 40% throughout the 10-day forecast range. However, the Brier skill score (BSS; Wilks, 1995) decreases quickly so that there is no skill at all beyond day 8. Clearly, the two skill measures present contrasting perceptions of the performance of the EPS. This raises the obvious question of whether the forecasts are skilful or not and, perhaps more importantly, are the forecasts useful or not? It should be noted that these questions are not restricted to probability forecasts but are equally relevant to the more traditional deterministic forecasts.