Academic journal article Memory & Cognition

Optimal Classifier Feedback Improves Cost-Benefit but Not Base-Rate Decision Criterion Learning in Perceptual Categorization

Academic journal article Memory & Cognition

Optimal Classifier Feedback Improves Cost-Benefit but Not Base-Rate Decision Criterion Learning in Perceptual Categorization

Article excerpt

Unequal payoffs engender separate reward- and accuracy-maximizing decision criteria; unequal base rates do not. When payoffs are unequal, observers place greater emphasis on accuracy than is optimal. This study compares objective classifier (the objectively correct response) with optimal classifier feedback (the optimal classifier's response) when payoffs or base rates are unequal. It provides a critical test of Maddox and Bohil's (1998) competition between reward and accuracy maximization (COBRA) hypothesis, comparing it with a competition between reward and probability matching (COBRM) and a competition between reward and equal response frequencies (COBRE) hypothesis. The COBRA prediction that optimal classifier feedback leads to better decision criterion learning relative to objective classifier feedback when payoffs are unequal, but not when base rates are unequal, was supported. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback. In addition, delayed feedback affected learning of the reward-maximizing decision criterion.

The need to categorize on the basis of uncertain information is ubiquitous in both personal and professional life. Each time we decide to follow or exceed the speed limit, to bring or not bring a jacket, to hire or not hire an individual, or to diagnose a patient, we are categorizing. All organisms categorize and must perform this task with some measure of success or they will die. In light of this fact, it is reasonable to suppose that, in many domains, human categorization performance is nearly optimal (Ashby & Maddox, 1998). Although optimality can be defined in a number of different ways, a common definition is performance that maximizes long-run reward (Green & Swets, 1966).

The optimal classifier is sensitive to information about category base rates (e.g., the prevalence of different dis eases in the population) and the costs and benefits associated with correct and incorrect categorization (e.g., the benefit of correctly diagnosing a heart attack, and the cost of misdiagnosing) and uses this information to set a decision criterion that maximizes reward; values below the criterion yield one categorization response, and values above the criterion yield another response. The optimal decision criterion is affected similarly by base-rate and cost-benefit manipulations (see Equation 2, below). For example, if the probability of observing an exemplar from Category A is three times that of observing an exemplar from Category B (a 3:1 base-rate condition), or if the benefit of a correct Category A response is three times the benefit of a correct Category B response (assuming no cost for an incorrect response; referred to as a 3:1 payoff condition), then the optimal decision criterion, β^sub o^ = 3, maximizes long-run reward.

Despite the survival value of optimal categorization, direct comparisons of human behavior with that of the optimal classifier suggest that humans rarely behave optimally. A robust finding in the categorization decision criterion learning literature is that observers use a suboptimal decision criterion in both conditions, referred to as conservative cutoff placement, but that the magnitude of conservative cutoff placement is much larger when payoffs are manipulated than when base rates are manipulated, even when the optimal decision criterion is equated across conditions (Busemeyer & Myung, 1992; Erev, 1998; Healy & Kubovy, 1981; Maddox, 2002; for related work from the recognition memory literature, see Estes & Maddox, 1995; Heit, Brockdorff, & Lamberts, 2003). A thorough understanding of the mechanisms underlying these performance differences is of fundamental importance for at least two reasons. First, a better understanding of the cognitive processes involved in decision criterion learning under unequal base-rate and payoff conditions will help cognitive scientists develop and test computational models of performance in this ubiquitous, real-world task. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.