variables being predicted from experimental factors. Unfortunately, the hypothesis-testing mind set of most researchers leads them to believe that the most important generalization taken from a study is whether a factor is significant or not. We think that attending to the size of the effect (e.g., by examining the regression equation parameters, or the mean differences in ANOVA) rather than solely to its significance may be quite important for deciding what factors to pursue in future studies. We don't mean to disparage normal science in cognitive aging. Hunting for critical parameters parallels the state that many sciences faced early in their existence, for instance, astronomy when compiling reliable measurements about the positions and movements of the planets and stars. We do want to encourage people to use these parameters to build predictive models.
The second path to progress is to derive strong theories about performance differences using mathematical and computer simulation models (e.g., Kieras & Polson, 1985; Richman, Staszewski, & Simon, 1995). Needless to say, reliable measurements are a necessary condition for this path, but obviously not a sufficient one. It will take a great deal of effort to formalize the learning of a commercial word processor by translating this process into production system models. Perhaps large-scale simulation architectures such as SOAR (e.g., Newell, 1990) can be tried. One challenge is that modern commercial word processors are very complex; for example, the Word for Windows 6.0 word processor includes over 500 commands. Modeling performance would require very detailed task analyses, which might take so much time that the word processor would be obsolete by the time that the research was completed.
We also believe that getting feedback on our theories from occasional forays into applied research is a necessary step on either road to progress. We can all benefit from attempting to scale up our favorite laboratory effects, and discovering the inevitable mismatches between lab-generated theory and applied data. Such failures can spur us on to identify missing factors and to propose more inclusive theories about cognitive aging.
This work was supported by grants from the Natural Sciences and Engineering Research Council of Canada, NSERC A0790, and by the Canadian Aging Research Network (CARNET), one of 15 Networks of Centres of Excellence supported by the Government of Canada.