"I Was Well-Calibrated All Along": Assessing Accuracy in Retrospect
The fact that beliefs tend to come in degrees is clear from the ambiguity, ambivalence, and uncertainty we continually experience in our lives. We may have absolutely no doubt in the proposition that the river Thames flows through London, be quite convinced that Holland is more densely populated than Finland but only have a faint hunch that "Absinthe" is not a precious stone. It should come as no surprise that degrees of belief, confidence, or subjective probabilities play a central role in many normative and descriptive analyses of human behavior. The confidence we have in our convictions is one of the critical components of normative models of rational decision making (e.g., Savage, 1954), but is also receiving increased attention in research on memory performance ( Koriat & Goldsmith, 1996), psychophysical discrimination ( Baranski & Petrusic, 1994; Juslin & Olsson, 1997; Olsson & Winman, 1996; see also Chapter 5 of the present volume), and applied matters such as eyewitness confidence ( Juslin, Olsson, & Winman , 1998; Wells & Murray, 1984). However, two pathologies--or in more appropriate present-day jargon, cognitive biases--haunt much of modern research on confidence assessment: the overconfidence phenomenon and the hindsight or "I-knew-it all-along" phenomenon, both of which may be illustrated by reference to a simple, two-alternative general knowledge question.
In calibration studies that purport to demonstrate the overconfidence phenomenon, the participants are presented with large sets of general knowledge items of the following form--