Calibrated Probabilistic Mesoscale Weather Field Forecasting: The Geostatistical Output Perturbation Method; Comment
Tebaldi, Claudia, Nychka, Doug, Journal of the American Statistical Association
Forecasting the weather presents a unique context for statistics, blending physical modeling with complicated observational data to produce information that is used at many different levels of sophistication. We are pleased that Gel, Raftery, and Gneiting (GRG) have brought this area to the attention of a statistical audience. In this discussion we give the reader a broader view of the use of ensemble techniques in numerical weather prediction (NWP). We have some comments about the use of ensembles idea presented by GRG and also present some of our recent analysis of the value of ensemble forecasts.
1. THE VALUE OF A FORECAST AND QUANTIFYING FORECAST SKILL
Weather forecasts have many users and, of course, the value and form of a forecast may depend on its intended purpose. Perhaps the most common use of a forecast is the estimate, say maximum surface temperature for a point location and a companion measure of uncertainty (e.g., Nychka's daughter asks him each morning what the temperature will be in Boulder so she can choose her outfit for school; she then asks him if he is "sure" about the forecast). In contrast, the geostatistical output perturbation (GOP) method goes beyond point forecasts using representations of the spatial covariance of the forecast accuracy to yield an ensemble of meteorological fields. The variability about the mean surface quantifies the uncertainty. Although this gives a significantly richer inference concerning the forecast, we also contend that it targets a sophisticated consumer.
To illustrate the distinction between point forecasts versus ensembles of fields, consider the following example. The Colorado Department of Transportation must make a decision whether to salt a highway to prevent icing. This decision is based on whether at any point along the highway the temperatures will dip below freezing. Thus, in statistical language the inference is whether the minimum of the field over a particular domain (the highway) has a high probability of being below freezing. To our mind, GRG give an elegant solution to this problem. For each ensemble field, the minimum temperature along the route of the highway must be found. The result is an empirical distribution of minimum temperatures that attempts to incorporate the spatial dependence of errors in the field and so may be more accurate in assessing the potential for icing. We are not sure how a correct inference would be drawn from just point forecasts of temperature with accompanying standard errors, so GRG's approach seems particularly useful in this context.
It is not clear that the man on the street or the forecaster on the evening news can interpret ensembles of fields and draw straightforward conclusions on the confidence he or she has in the forecast. In this respect, we question the need for a cultural change in forecaster attitude toward realizations of the GOP method. Based on the preceding example, it may be that specific applications of the forecast will benefit from ensemble fields, but in many cases a pointwise assessment of a best guess plus or minus a range of uncertainty, a simple probability density function, or a number between 0 and 1 that characterizes the degree of confidence in the forecast will do. Accordingly, in the last part of this discussion we focus more on the problem of obtaining more accurate inferences for point forecasts.
1.1 Ensemble Forecasting
A statistician can think of an ensemble as a discrete sample whose empirical distribution approximates a continuous distribution of interest. An idealized ensemble is a random sample from the posterior distribution for the state of the atmosphere given all past data and incorporating all known physical models of the flow.
Let [x.sub.t] denote the vector of meteorological variables on a spatial grid that describe the state of the atmosphere at time t. The entire physical and geographic knowledge of the atmosphere's dynamical behavior can be subsumed by a function g, the NWP model, such that