Casing the NASDAQ for Profits: After All That Work Understanding Case-Based Analysis and Preprocessing Our Market Data, Is This Approach Really Going to Be Worth It as a Trading Application? We Find out, Using the NASDAQ Market
Ruggiero, Murray A., Jr., Modern Trader
Discussing a nice trading theory is one thing. Executing that theory in the markets is quite another. In this third of three parts, we'll attempt to make this leap in the context of case-based reasoning. First, we introduced the concept. Second, we demonstrated the dry but vital step of pre-processing market data for use in cased-based forecasting applications. In this article, we will finish our Excel-based application for case-based analysis and use it to show how this methodology can be used in market forecasting and trading applications.
Our model uses brute force Euclidian distance measures to find similar cases and uses the average of the similar cases to create a forecast for the case in question. This process is very slow and will not work effectively in a real world application because it takes about two to three minutes per year, per market to iterate for a single bar on a high-end PC. It is also too slow to use more advanced case-based methodologies such as adjusting the weighting on each feature.
That doesn't mean we have to abandon this approach. We will show how our simple tool works using real market data. Finally, we will discuss several advanced topics that will be necessary to incorporate to take our technology to the next level.
WEATHER: A CASE STUDY
Weather forecasting is an everyday problem that has a lot of similarity to forecasting financial data. It is fractal--a chaotic type problem that is hard to predict because of its irregularity at all levels of examination. in weather forecasting, most of the equations are known, which is one major advantage over financial forecasting where the equations are not known. Even with this major difference, we can learn from research in weather forecasting applications and apply what we learn to financial forecasting because it is a problem where small model errors limit forecasting capabilities to about 10 days with reasonable accuracy.
Researchers have tried to use case-based reasoning and fuzzy set theory to forecast six-hour cloud ceilings and visibility at an airport. They used 300,000 consecutive hourly airport weather observations over a 36-year period as the database to generate the templates and cases. These forecasts were compared to what is called climatology, which uses averages and seasonal trends to create a baseline in which any forecast that is not better is said to have no predictive value.
An example of this concept would be a system that produces random buy and sell signals in the 10-year Treasury note. If we averaged 1,000 different random sets of signals, we would create a model that has no predictive value but might be profitable because bonds have been in an uptrend for the past 20 years. A cross example of a climatological forecast is to average the high temperature for a given date and use that average as the forecast temperature for that date this year.
There are two general approaches to weather forecasting. The first approach is empirical and the second is dynamical. The empirical approach is based upon the occurrence of analogs or similar weather situations and maps that can be used in forecasting. The dynamical approach is based upon a computer model using current conditions in observation grids with few miles in each grid pattern. Equations are applied to the observations to produce a forecast for a short time period. Then, the forecast for the grid point for this new time is used to produce the next updated forecast. This method causes small errors that are in a 12-hour forecast to become huge errors within 36 to 48 hours.
In this airport case study, a large database was used with a case-based concept called "fuzzy-nn algorithm" to measure the similarity between cases. This concept uses fuzzy logic, which is an approach to computing based on "degrees of truth" rather than the usual "true or false" logic.
Lotfi Zadeh of the University of California at Berkeley first advanced the concept of Fuzzy logic in the 1960s. …