Testing the Tea Leaves: Evaluating the Validity of Forecasts: A Variety of Formal Methods Exist but Decision-Makers Should Also Apply a Systematic Assessment Framework

By Vanston, John H.; Vanston, Lawrence K. | Research-Technology Management, September-October 2004 | Go to article overview

Testing the Tea Leaves: Evaluating the Validity of Forecasts: A Variety of Formal Methods Exist but Decision-Makers Should Also Apply a Systematic Assessment Framework


Vanston, John H., Vanston, Lawrence K., Research-Technology Management


Planning is, by definition, oriented to the future. No one makes dinner plans for last week. No successful manager is truly interested in the present, except with regard to how it can be changed for the future. Thus, all business plans, all financial plans, and all marketing plans are based on projections about how the future will unfold. These projections--forecasts--can be formal of informal, implicit of explicit, short term or long term. However, regardless of the type of forecast used in business planning, the success of the plan will, in large measure, depend on the validity of the forecast.

Because of the importance of valid forecasts and because the people charged with making key business decisions typically rely, to a great extent, on forecasts made by others, it is essential that planners, executives and other decision-makers be able to assess the validity of various forecasts. In making such assessments, these people typically rely on the reputation of the forecaster, the results of past forecasts, or their personal comfort with the forecast. However, in many cases, a more formal assessment of a forecast can be of significant value to people who must stake their reputations and careers on its validity. The purpose of this paper is to provide a set of tools that can assist in making such an assessment.

The authors' 30 years of forecasting experience involving more than 150 companies, government agencies, associations, and academic institutions, including more than half of the technology-oriented Fortune top 50 companies, has convinced us that there are two primary reasons for forecast failures: the use of inappropriate or outdated information and the use of improper models. It appears reasonable, therefore, to use procedures that will test each of these factors.

Two general types of data are typically used in forecasting: statistical data and expert opinion. Tests for the two types of data are similar, but do have significant differences.

Statistical Data

Statistical data should be examined for the following qualities:

Reliability of source

Obviously, data sources with long reputations for reliability and accuracy are more credible than those without such reputations. Data from official government agencies usually have strong credence, as do data from recognized authoritative sources such as professional associations, public service organizations, and media files. For example, a projection of school age population in future years by the U.S. Census Bureau could be accepted as quite reliable, both because the organization has a long history of accurate population projections and because most of the people reflected in the projection will have already been born. On the other hand, data presented in most companies' promotional material may well be subject to question.

A number of commercial organizations provide data for a fee. These vary from those that organize and publish data in a general subject area to those that provide data in narrow and specialized areas. The reliance that can be placed on this information depends on the reputation of the organization, its past record, and the credibility of the organization's own information sources.

E-mail and Internet websites offer a wealth of data to people familiar with their use. Although there are means of checking the accuracy of such data to some extent, in general, data obtained from such sources should be viewed with considerable trepidation. Of course, the most serious questions about reliability arise when the sources of data are not indicated at all.

Currency

Because the gathering of primary source data is both difficult and expensive, forecasters often extrapolate

from old information or, for convenience, continue to use information that has grown long-of-tooth. Moreover, many forecasters use data available from other agencies without giving due regard to its timeliness. …

The rest of this article is only available to active members of Questia

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

Items saved from this article

This article has been saved
Highlights (0)
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

Citations (0)
Some of your citations are legacy items.

Any citation created before July 30, 2012 will labeled as a “Cited page.” New citations will be saved as cited passages, pages or articles.

We also added the ability to view new citations from your projects or the book or article where you created them.

Notes (0)
Bookmarks (0)

You have no saved items from this article

Project items include:
  • Saved book/article
  • Highlights
  • Quotes/citations
  • Notes
  • Bookmarks
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited article

Testing the Tea Leaves: Evaluating the Validity of Forecasts: A Variety of Formal Methods Exist but Decision-Makers Should Also Apply a Systematic Assessment Framework
Settings

Settings

Typeface
Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Help
Full screen

matching results for page

    Questia reader help

    How to highlight and cite specific passages

    1. Click or tap the first word you want to select.
    2. Click or tap the last word you want to select, and you’ll see everything in between get selected.
    3. You’ll then get a menu of options like creating a highlight or a citation from that passage of text.

    OK, got it!

    Cited passage

    Style
    Citations are available only to our active members.
    Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

    1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

    Cited passage

    Thanks for trying Questia!

    Please continue trying out our research tools, but please note, full functionality is available only to our active members.

    Your work will be lost once you leave this Web page.

    Buy instant access to save your work.

    Already a member? Log in now.

    Oops!

    An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.