Academic journal article Federal Reserve Bank of Atlanta, Working Paper Series

Forecasting Using Relative Entropy

Academic journal article Federal Reserve Bank of Atlanta, Working Paper Series

Forecasting Using Relative Entropy

Article excerpt

Abstract: The paper describes a relative entropy procedure for imposing moment restrictions on simulated forecast distributions from a variety of models. Starting from an empirical forecast distribution for some variables of interest, the technique generates a new empirical distribution that satisfies a set of moment restrictions. The new distribution is chosen to be as close as possible to the original in the sense of minimizing the associated Kullback-Leibler Information Criterion, or relative entropy. The authors illustrate the technique by using several examples that show how restrictions from other forecasts and from economic theory may be introduced into a model's forecasts.

JEL classification: E44, C53

Key words: approximate prior information, Kullback-Leibler Information Criterion, relative numerical efficiency


One of the frustrations of macroeconometric modeling and policy analysis is that empirical models that forecast well are typically nonstructural, yet making the kinds of theoretically coherent forecasts policymakers wish to see requires imposing structure that may be difficult to implement and that in turn often makes the model empirically irrelevant. In this paper, we describe the application of a procedure that can, in principle, be used to produce forecasts that are consistent with a set of moment restrictions without imposing them directly on the model. Even when it is desirable to impose the restrictions directly on the forecasting model, the technique in this paper can be used to examine the likely validity of a range of restrictions without the need to re-fit the model each time, and thereby provides the modeler with considerable flexibility to experiment with various types of restrictions.

Our procedure, inspired by Stutzer (1996) and Kitamura and Stutzer (1997), involves changing the initial predictive distribution to a new one that satisfies specified moment conditions, but that changes the other properties of the new distribution the least. That is, we minimize the relative entropy between the two distributions, subject to the restriction that the new distribution satisfies the specified moment conditions. Stutzer (1996) used this idea to modify a nonparametric predictive distribution for the price of an asset to satisfy the martingale condition associated with risk-neutral pricing. Foster and Whiteman (2002) build on this idea to price soybean options using a predictive model reflecting weather, market conditions, etc. Kitamura and Stutzer (1997) used the idea to provide an alternative to generalized method of moments estimation in which the moment conditions hold exactly relative to a new measure (but not necessarily in the data); likewise, our procedure imposes the moment conditions exactly on a new predictive distribution that is as close (in the information-theoretic sense) as possible to the original.

The need to incorporate conditioning information into a forecast arises routinely. This is particularly true in the context of handling data release lags. In circumstances when observations on some variables are released before others, a forecaster would like to make predictions for the unknown post-sample values conditional on all the available data. In these circumstances, the known post-sample data could be thought of as a mean restriction on the forecast.

Conditioning information has been incorporated into forecasting models in a variety of settings (see for example Theil, 1971). In the VAR literature, Doan, Litterman and Sims (1984) exploit the contemporaneous and inter-temporal variance-covariance matrix structure in a VAR to account for the impact of conditioning a forecast on post-sample values for some variables in the model. Waggoner and Zha (1999) extended the Doan, Litterman and Sims analysis to accommodate uncertainty in model parameters in a fully Bayesian setting. Our procedure can be viewed as an alternative to the Waggoner-Zha technique, where we incorporate the conditioning information directly into the prior. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.