© 1991 American Statistical Association
Journal of Business & Economic Statistics, July 1991, Vol. 9, No. 3
Calibration as Testing: Inference in Simulated Macroeconomic Models
Allan W. Gregory and Gregor W. Smith
Department of Economics, Queen’s University, Kingston, Ontario, K7L 3N6, Canada
A stochastic macroeconomic model with no free parameters can be tested by comparing its features, such as moments, with those of data. Repeated simulation allows exact tests and gives the distribution of the sample moment under the null hypothesis that the model is true. We calculate the size of tests of the model studied by Mehra and Prescott. The approximate size of their test (which seeks to match model-generated, mean, risk-free interest rates and equity premia with historical values) is 0 although alternate, empirical representations of this model economy or alternate moment-matching tests yield large probabilities of Type I error.
KEY WORDS: Equity premium; Monte Carlo; Simulation; Type I error.
Calibration in macroeconomics is concerned primarily with testing a model by comparing population moments (or perhaps some other population measure) to historical sample moments of actual data. If the correspondence between some aspect of the model and the historical record is deemed to be reasonably close, then the model is viewed as satisfactory. If the distance between population and historical moments is viewed as too great, then the model is rejected, as in the widely cited equity-premium puzzle of Mehra and Prescott (1985). A drawback to the procedure as implemented in the literature is that no metric is supplied by which closeness can be judged. This leads to tests with unknown acceptance and rejection regions.
This article provides a simple way to judge the degree of correspondence between the population moments of a simulated macroeconomic model and observed sample moments and develops a framework for readily calculating the size (probability of Type I error) of calibration tests. We apply this method to the well-known equity-premium case. This article is not concerned with a “solution” to the equity-premium puzzle. Rather it evaluates the probability of falsely rejecting a true macroeconomic model with calibration methods. One finding is that the size of the test considered by Mehra and Prescott (which seeks to match mean risk-free interest rates and equity premia) is 0, so the model with their parameter settings is unlikely to have generated the observed historical moments. Some alternate versions of the consumption-based asset-pricing model or alternate moment-matching tests yield large probabilities of Type I error.
Section 1 characterizes calibration as testing. A simple formalization of calibration as Monte Carlo testing allows exact inference. Section 2 contains an application to the test conducted by Mehra and Prescott (1985). Section 3 concludes.
Calibration in macroeconomics has focused on comparing observed historical moments with population moments from a fully parameterized simulation model—that is, one with no free parameters. One might elect to simulate a model because of an analytical intractability or because a forcing variable is unobservable. In macroeconomics, examples of unobservable forcing variables include productivity shocks in business-cycle models or consumption measurement errors in asset-pricing models.
Consider a population moment θ, which is restricted by theory, with corresponding historical sample momentfor a sample of size T. Call the moment estimator Assume that is consistent for θ. The population moment is a number, the sample moment is the realization of a random variable (an estimate), and the estimator is a random variable. The calibration tests applied in the literature compare θ and and reject the model if θ is not sufficiently close to In some calibration studies, attempts are made to exactly match the population moment to the sample moment (there must be some leeway in parameter choice to make this attempt nontrivial). Such matching imposes unusual test requirements because θ and can differ even when the model is true due to sampling variability in Moreover, judging closeness involves the sampling distribution of the estimator Standard hypothesis testing procedures may be unavailable because the exact or even asymptotic distribution of the estimator is unknown.
One prominent advantage in the calibration of macroeconomic models that has not been exploited fully is that the complete data-generating process is specified. Thus the sampling variability of the simulated moment can be used to evaluate the distance between θ and