Forecast Comparisons

By Barrell, Ray; Kirby, Simon et al. | National Institute Economic Review, July 2005 | Go to article overview
Save to active project

Forecast Comparisons


Barrell, Ray, Kirby, Simon, Metz, Robert, National Institute Economic Review


This paper compares the National Institute of Economic and Social Research (NIESR) forecasts for output, inflation and key public sector finance variables against the corresponding forecasts from HM Treasury (HMT), the Bank of England (Bank) and the Institute for Fiscal Studies (IFS). We find that NIESR outperforms, on average, other major bodies in its forecasts for output and in particular inflation where simple scores are used. It also performs well on the forecasting of the government current budget surplus but not public sector net borrowing. Statistical estimates of accuracy provide a less clear picture but their reliability is blighted by the small sample size.

Keywords: evaluating forecasts; output forecasting; inflation forecasting; public sector finance forecasting

JEL classification: C53, E17

Introduction

In this note we look at forecasts of output growth, inflation, the government current surplus and public sector borrowing undertaken over the past eight years by the National Institute (NIESR), the Treasury (HMT), the Bank of England (Bank) (for output and inflation) and the Institute for Fiscal Studies (IFS) (for the surplus and borrowing). We compare these forecasts to the first outcome for the data, as has been common in other such exercises, (1) as these are the data that will induce policymakers to change their actions. For each we calculate the Mean Absolute Deviation (MAD), the Root Mean Squared Deviation (RMSD) and also a score (which is nearer). Given the short data period, the score (always reported as THEM:US) is revealing.

The Treasury produces two types of forecasts, and it appears that those in the spring Budget get marginally higher scores than the NIESR spring forecast, as we can see from table 1, whilst NIESR normally performs better in the autumn as compared to the Pre-Budget Report. (2) In particular, NIESR scores as highly in the autumn on its forecast of the target public sector current budget surplus for the next fiscal year. This is probably the single most important indicator when setting plans for fiscal policy for the next year. For both output and especially inflation NIESR marginally outscores the Bank, as we can see from table 1, at least for the two forecasts considered. Compared with the IFS, overall scores on public sector forecasts are even, but NIESR scores noticeably better when evaluating the target public sector current surplus for the next fiscal year. Compared with the others on the table, NIESR on average has a higher score for output (12-13) and especially inflation (9-14), and it has a higher score on the target current surplus for next year (9-11). With respect to inflation, the evaluation of NIESR's inflation forecast on pp. 60-69 of this Review suggest that its point forecast is reliable.

Methods of comparing forecasts

Forecast comparisons are always fraught with difficulty, especially over such short periods. Single forecasts can distort comparisons over such periods, and we have to find a way to reduce the impact of outliers. The 'score' is probably the least sensitive to outliers, followed by the MAD. The RMSD is considered the industry standard, and can most easily be associated with a (quadratic) loss function for evaluating the costs of forecast errors. With eight observations it is not always easy to interpret comparisons for this indicator, although they are of value. Different forecasters use different degrees of rounding and comparisons can be spuriously accurate if one uses intervals and the other uses point forecasts, and hence any evaluation can be disputed. Where intervals are reported we use their midpoint. In addition, forecasts are produced for different purposes, and hence can use marginally different indicators. HMT (and the IFS) forecast seasonally unadjusted public sector indicators for the fiscal year, whilst NIESR forecasts seasonally adjusted National Accounts consistent public sector outcomes, and hence each has to be evaluated against the relevant first data.

The rest of this article is only available to active members of Questia

Sign up now for a free, 1-day trial and receive full access to:

  • Questia's entire collection
  • Automatic bibliography creation
  • More helpful research tools like notes, citations, and highlights
  • Ad-free environment

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
Loading One moment ...
Project items
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited article

Forecast Comparisons
Settings

Settings

Typeface
Text size Smaller Larger
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

While we understand printed pages are helpful to our users, this limitation is necessary to help protect our publishers' copyrighted material and prevent its unlawful distribution. We are sorry for any inconvenience.
Full screen

matching results for page

Cited passage

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited passage

Welcome to the new Questia Reader

The Questia Reader has been updated to provide you with an even better online reading experience.  It is now 100% Responsive, which means you can read our books and articles on any sized device you wish.  All of your favorite tools like notes, highlights, and citations are still here, but the way you select text has been updated to be easier to use, especially on touchscreen devices.  Here's how:

1. Click or tap the first word you want to select.
2. Click or tap the last word you want to select.

OK, got it!

Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.

For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

Already a member? Log in now.

Are you sure you want to delete this highlight?