How Each Reinforcer Contributes to Value: "Noise" Must Reduce Reinforcer Value Hyperbolically
Michael L. Commons
Harvard Medical School
University of Chicago Business School
Edward J. Trudeau
Empirical evidence has shown that the effectiveness of a reinforcer is related to the delay between reinforcer and response (e.g., for pigeons, see Chung & Herrnstein, 1967; Mazur, 1987). For people the finding of a similar function shows that the effectiveness of a stimulus is related to the delay between its presentation and the time at which it is to be recalled (for people, see Ebbinghaus, 1885). This decrease in effectiveness of an event over time has come to be known as the recency effect (e.g., Glanzer & Cunitz, 1966; Madigan, 1971; Murdock, 1962; Sternberg, 1966; Woodword , 1970). That reinforcer effectiveness over time may be represented as an exponential decay was not an unreasonable hypothesis, and has been suggested previously (e.g., Commons, 1981, and White & McKenzie, 1982, in animal psychology; Muth, 1960, in forecasting theory; Wickelgren, 1974, in human psychology). Yet previous data suggest that this relation is hyperbolic ( Commons, Woodford, & Ducheny, 1982; McCarthy's data as analyzed in McCarthy & White, 1987). Theoretical considerations from a number of sources suggest that the decay of value is best represented by a hyperbolic model ( Ainslie, 1975; Davison & Tustin, 1978; Fantino, 1981; Fantino, Abarca, & Dunn, 1987; Mazur, 1987).
The reduction in reinforcer effectiveness implicit in the hyperbolic model may be due to the imperfections of memory and the confusion caused by intervening events. From a signal-detection perspective, intervening events function as noise under this supposition, obscuring previous events. If reinforcer effectiveness decreases hyperbolically, then the decrease will be rapid at first and fall more slowly as time progresses, though perhaps never reaching zero. It will be shown that the relationship between reinforcer