Academic journal article Psychonomic Bulletin & Review

Unexpected Downshifts in Reward Magnitude Induce Variation in Human Behavior

Academic journal article Psychonomic Bulletin & Review

Unexpected Downshifts in Reward Magnitude Induce Variation in Human Behavior

Article excerpt

Published online: 25 July 2013

© Psychonomic Society, Inc. 2013

Abstract We investigated how changes in outcome magnitude affect behavioral variation in human volunteers. Our participants entered strings of characters using a computer keyboard, receiving feedback (gaining a number of points) for any string at least ten characters long. During a "surprise" phase in which the number of points awarded was changed, participants only increased their behavioral variability when the reward value was downshifted to a lower amount, and only when such a shift was novel. Upshifts in reward did not have a systematic effect on variability.

Keywords Human learning . Variability

When presented with novel activities, we must identify which behaviors or strategies will lead to successful outcomes. In most cases, this is not a matter of identifying a single "perfect" action that will always result in success; instead, many activities demand that behavior display an ongoing degree of variability. In this respect, the mastery of a task consists not only of minimizing "errors," but also of sustaining appropriate levels of variability in our actions (Stokes, 2001).

In behavioral paradigms, variability has often been characterized in terms of "operant response classes": Rather than reinforcing a single discrete behavior, schedules instead reinforce those behaviors that belong to broad classes that vary along multiple dimensions, such as timing and response topography. The degree of variability of these various dimensions can also be shaped by feedback (Shahan & Chase, 2002).

Organisms can readily increase or decrease their behavioral variability, whether responses are constrained to a narrow class (Davison & Baum, 2000) or widened to broad conceptual categories (Neuringer & Jensen, 2010). Indeed, behavioral variability seems not only inescapable, but also often manifests at the precise levels appropriate to a given context; response variability not only adapts, but is also adaptive (Neuringer, 2002).

The experimental evidence suggests that strategic increases in variability precede the discovery of new problem-solving strategies. Greater variability during skill acquisition is associated with greater learning (Stokes, Lai, Holtz, Rigsbee, & Cherrick, 2008). This pattern of "variability- as-path-to-discovery" is observed in children mastering grammatical rules (Bowerman, 1982), learning to solve arithmetic problems (Siegler & Jenkins, 1989), or acquiring novel concepts (Goldin-Meadows, Alibani, & Church, 1993). Furthermore, children who use more strategies when first learning a task acquire the correct strategy more often than those who use fewer initial strategies (Siegler, 1995). This has also been seen in adults making novice-to-expert transitions in radiology (Lesgold, Rubinson, Feltovich, Klopfer, & Wang, 1988) or cardiology (Johnson et al., 1981), in whom greater variability precedes acquisition of advanced diagnostic expertise. This makes modulating levels of variability central to exploration/exploitation strategies (March, 1991).

The increase in variability under extinction protocols is well-established (see Balsam, Deich, Ohyama, & Stokes, 1997, for a review), and this may simply be a basic principle of behavior. Natural selection plausibly favors mechanisms for generating variability in the face of failure, and such a mechanism would have relevance to a wide range of problem domains (Neuringer, 2002). In studies of extinction, reported changes in behavior are often more quantitative than qualitative. Neuringer, Kornell, and Olufs (2001) reported, for example, that although extinction increased the frequency of rare response sequences, relatively common sequences were still exhibited more often than their uncommon counterparts. In addition to extinction effects, intermediary levels of variability are observed when reinforcement is reduced without being entirely extinguished. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.