Academic journal article Psychonomic Bulletin & Review

Decision Criteria Do Not Shift: Commentary on Mueller and Weidemann (2008)/Decision Noise May Mask Criterion Shifts: Reply to Balakrishnan and MacDonald (2008)

Academic journal article Psychonomic Bulletin & Review

Decision Criteria Do Not Shift: Commentary on Mueller and Weidemann (2008)/Decision Noise May Mask Criterion Shifts: Reply to Balakrishnan and MacDonald (2008)

Article excerpt

(ProQuest: ... denotes formulae omitted.)

J. D. Balakrishnan

California Polytechnic State University,

San Luis Obispo, California

and

Justin A. MacDonald

New Mexico State University, Las Cruces, New Mexico

The effects of base rates and payoffs on the shapes of rating receiver operating characteristic curves are inconsistent with the basic assumptions of signal detection theory (SDT), in particular the notion of a shifting decision criterion. Mueller and Weidemann (2008) propose that these unexpected phenomena are not due to problems with the decision-criterion construct but are instead due to two compounded effects: instability of the decision criterion across trials, and even greater instability in the flanking criteria that determine which confidence rating will be reported. There are several problems with the authors' decision-noise hypothesis. First, even if their hypothesis about decision noise were taken for granted, the key feature of the ratings data that rejects the SDT model would remain a mystery. Second, the same violations of SDT that are exhibited in the ratings paradigm are also exhibited in the yes-no detection task when response time is substituted for confidence as a basis for analysis. Finally, the decision-noise hypothesis predicts that sensitivity will increase when one source of this variation-the response on a previous trial-is controlled for. This prediction was consistently violated in both the yes-no and ratings conditions of Mueller and Weidemann's experiment. In an Addendum, we respond to Weidemann and Mueller's (2008) reply to this Comment.

In many areas of perception and memory research, experimental phenomena that appear to have significant implications about perception or memory processes per se could actually be due to the effects of response biases. For the past 50 years or so, the most widely accepted method of distinguishing these two possibilities has been to apply a signal detection analysis to the data (e.g., Green & Swets, 1966). Signal detection theory (SDT) is, at its core, the formal expression of an intuitively compelling idea.that is, that decisions about stimuli are based on decision criteria. A biased decision criterion demands relatively stronger sensory or memory evidence before an unpreferred response will be prescribed. The signal detection model is thought to be well supported by scores of classical studies, followed up by many years of apparently successful applications.

Recently, we reported two empirical results that appear to be as robust as any of the classical results in the SDT literature and yet are inconsistent with the classical SDT framework (Balakrishnan, 1998a, 1998b, 1999; see also Van Zandt, 2000). The first problem is the fact that the receiver operating characteristic (ROC) curves obtained from yes. no detection experiments with confidence-rating responses change shape under different biasing conditions. The second problem is that the likelihood-ratio function (a measure that is closely related to the ROC curve, as we explain in this article) is always very close to 1 at the point of the function corresponding to the lowest confidence responses. Both properties are anathema from the SDT point of view. The first result contradicts the assumption that the sensory or memory effects of the stimuli do not depend on the amount of response bias in the decision process; the second result contradicts the assumption that a change in response bias is a change in how sensory or memory states are mapped to responses.that is, a shift in the decision criterion.

Three attempts to account for these empirical phenomena, without giving up on the central concepts of SDT, have been published: Treisman (2002), Kornbrot (2006), and Mueller and Weidemann (2008). Treisman?fs and Kornbrot?fs arguments were inadequate, in our view, for various reasons, the most obvious being that they did not show that any kind of signal detection model could actually fit the data we reported. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.