Subjective Probability Forecasts for Recessions: Evaluation and Guidelines for Use
Lahiri, Kajal, Wang, J. George, Business Economics
Probabilistic forecasts are often more useful in business than point forecasts. In this paper, the joint subjective probabilities for negative GDP growth during the next two quarters obtained from the Survey of Professional Forecasters (SPF) are evaluated using various decompositions of the Quadratic Probability Score (QPS). Using the odds ratio and other forecasting accuracy scores appropriate for rare event forecasting, we find that the forecasts have statistically significant accuracy. However, compared to their discriminatory power, these forecasts have excess variability that is caused by relatively low assigned probabilities to forthcoming recessions. We suggest simple guidelines for the use of probability forecasts in practice.
Forecasting relatively rare business events like recessions or major stock market corrections is inherently risky, resulting in frequent misses and false signals. However, when uncertainty about future events is expressed in terms of probabilities (e.g., the probability of a recession next year is 30 percent), these forecasts are more informative and useful than purely categorical forecasts (e.g., recession or no recession) in that the probabilities can be used in the calculation of various measures of interest such as expected payoffs and downside risks. Also, because more information is imbedded in probability forecasts, there may be more scope to improve prediction.
The failure of point forecasts from large scale structural macro and VAR models or from professional surveys (e.g., Blue Chip, OECD, Survey of Professional Forecasters, National Association for Business Economics, etc.,) in predicting--or even timely recognition of--postwar recessions is well documented. (1) Admittedly, recessions that are caused by external shocks cannot, by definition, be predicted. However, the transmission of the exogenous shocks through the economy can take some time to generate a full-fledged recession. Moreover, anti-inflationary monetary policies, that have often caused recessions in the U.S. economy, take quarters to take effect. Thus, the basic challenge is whether one can identify, at least probabilistically, an impending recession by understanding the structure of the transmission mechanism. Not surprisingly, in recent years, economists have developed advanced macroeconomic models to generate probability forecasts for business cycle turning points. (2)
However, one such model--the dynamic single index model developed by Stock and Watson (1993)--could not identify its first two out-of-sample recessions, viz., those of 1990 and 2001. Since the Stock-Watson model is built on one of the strongest scientific foundations found in the literature and on extensive use of time series data, the failures of their recession indexes represent a significant challenge for today's business cycle researchers. In explaining forecast failures, Stock and Watson (2003) painfully fall back on Leo Tolstoy in Anna Karenina, "Happy families are all alike; every unhappy family is unhappy in its own way." That is, econometric models typically fail to predict recessions because each recession is special in its own novel way. For example, while the decline of the stock market gave some advance warning of the 2001 recession, it was not otherwise reliable during the 1980s and the 1990s. In short, the structure of the economy changes--sometimes abruptly--and no single model specification or a set of variables can do justice to all forthcoming recessions.
Yet, recessions inflict enormous costs to society, the exact extent of which we have just begun to explore. For instance, Bangia, et al. (2002) showed that the economic capital required to capitalize a bank during a recession year is about 25-30 percent higher than that during an expansion year. Carey (2002) found that losses of a typical bank portfolio during a recession are about the same as losses in the 0.5 percent tail during an expansion. …