Forecasting software developers have made significant progress in the area of antomation, scalability, and the incorporation of judgment into a forecast ... we now have access to more and better data but the demand patterns are becoming worse ... as more people touch a forecast through collaboration there are more chances for the forecast to get worse as well as better.
Forecasting is a serious business, but not all parties to the history of forecasting can be taken seriously. In the past, it was the prophets, oracles, astrologers, and psychics who provided us with guidance about the future-from when to plant crops to when to go to war. Today there is no shortage of self-proclaimed "seers" and "futurists" that will gladly provide guidance for a fee, but we also have access to terabytes of data, powerful computers, statistical software, and elaborate acronymed processes. This article looks at the historical evolution of the forecasting profession, examines what the forecasting software industry has provided us over the past twenty-five years, and takes a glimpse at where the industry might take us in the future.
No reflection on the past twenty-five years of forecasting software would be complete without a quick romp through the historical and philosophical foundations of this much-maligned profession.
Methods of prediction date back centuries and millennia, long before the advent of computers and forecasting software. Ancient Greece had the Oracle of Delphi, where wisdom and prophesy were exchanged for gifts and gold. The sixteenth century brought poet/astrologer Nostradamus whose 942 Quatrains remain as cryptic and broadly decipherable as Alan Greenspan's testimony before congress. In the twentieth century, we were blessed by the coming of the celebrity psychic, with such obfuscators as Jeane Dixon and Sylvia Browne, whose ability to predict correctly is dwarfed only by their propensity to predict wrongly. (Why do all psychic powers suddenly disappear when subjected to scientific scrutiny?)
Perhaps the first serious treatment of the philosophical foundations of forecasting came in 1748, in the book An Enquiry Concerning Human Understanding, by the British Empiricist philosopher David Hume. Hume was concerned with the problem of induction-the reasoning from particular facts to general conclusions. Business forecasting of today is an example of this, where we use specific known facts about the past (i.e., historical demand) to infer general statements about the future (the pattern of future demand). Hume observed that when we eat a piece of bread, it nourishes us. So we extrapolate our finite number of particular experiences to a general belief that "bread nourishes." When we do this, we are, in fact, creating a forecast-that the next piece of bread we eat will nourish us.
But Hume was a philosopher, so it was his role to be a critic and question everything we believed. He was compelled to ask the question: What justification is there for this belief that bread nourishes? Just because bread has exhibited this "nourishment" characteristic in the past, is that any proof that bread will continue to nourish in the future? Or to put the problem in business forecasting terms, just because demand has followed a particular pattern in the past-our model fits the history-is this any proof that this demand pattern will continue?
The answer is, of course, no. Hume reached the skeptical conclusion that there is no proof that the future will behave like the past. We ultimately have no justification for our forecasts! It has been over 250 years and there is still no refutation of Hume's argument. Hume is correct.
But those of us in the forecasting profession have not let Hume's skepticism ruin our day (or our livelihoods). Just because we have no logical proof that our forecasts will be any good (and we have a lot of empirical evidence that our forecasts are frequently bad! …