Academic journal article Memory & Cognition

Statistical Judgments Are Influenced by the Implied Likelihood That Samples Represent the Same Population

Academic journal article Memory & Cognition

Statistical Judgments Are Influenced by the Implied Likelihood That Samples Represent the Same Population

Article excerpt

Published online: 29 October 2011

# Psychonomic Society, Inc. 2011

Abstract When sample information is combined, it is generally considered normative to weight information based on larger samples more heavily than information based on smaller samples. However, if samples appear likely to have been drawn from different subpopulations, it is reasonable to combine estimates of these subpopulation means (typically, the sample means) without weighting these estimates by sample size. This study investigated whether laypeople are influenced by the likelihood of samples coming from the same population when determining how to combine information. In two experiments we show that (1) implied binomial variability affected participants' judgments of the likelihood that a sample was drawn from a given population, (2) participants' judgments were more affected by sample size when samples were implied to be drawn randomly from a general population, compared to when they were implied to be drawn from different subpopulations, and (3) people higher in numeracy gave more normative responses. We conclude that when determining how to weight and combine samples, laypeople use not only the provided data, but also information about likelihood and sampling processes that these data imply.

Keywords Judgment . Reasoning . Inductive reasoning . Mathematical cognition . Individual differences

(ProQuest: ... denotes formulae omitted.)

When an inference is made from a sample to a population, that sample's representativeness is of foremost concern. The more representative the sample, the better one can judge the true nature of the population. Humans are more trusting of data they believe is more representative (Kahneman & Tversky, 1972). Numerous studies highlight the importance of the law of large numbers in establishing representativeness: Humans are intuitively aware that larger samples typically provide more reliable information (e.g., Evans & Dusior, 1977; Evans & Pollard, 1985; Irwin, Smith & Mayfield, 1956; Jacobs & Narloch, 2001; Masnick & Morris, 2008; Nisbett, Krantz, Jepson & Kunda, 1983; Obrecht, Chapman & Gelman, 2007; Peterson & Beach, 1967; Sedlmeier & Gigerenzer, 1997; Sedlmeier & Gigerenzer, 2000). However, laypeople do not use sample size in what researchers consider to be a normative fashion. People's apparent confidence in samples' representativeness increases at a shallower rate than a normative use of sample size would predict (Obrecht, 2010). Also, mean difference has a stronger effect on people's judgments than does sample size (Obrecht et al., 2007). Furthermore, competing factors such as anecdotal descriptions and encounter frequency may lead to sample size being overlooked (Nisbett et al., 1983; Obrecht, Chapman & Gelman, 2009; Ubel, Jepson & Baron, 2001). Additionally, as illustrated by Kahneman and Tversky's (1972) classic "hospital problem," many people fail to recognize that larger sample sizes will reduce the chance that sample means will differ from a population mean (but see Evans & Dusior, 1977; Sedlmeier, 1998).

However, sample size is not the only factor that determines representativeness. The normative standard of weighting means by sample size assumes that samples are drawn randomly from a general population. When samples instead come from different subpopulations, ignoring sample size when combining estimates of subpopulation means is statistically correct. For example, consider a case in which you wish to determine what proportion of the general population would like a movie: 90% of a sample of 500 men and 10% of a sample of 100 women liked the movie. Weighting means by sample size, one would infer that 77% of the general population would like the movie. This would incorrectly overweight men's opinions and underweight women's opinions, relative to their prevalence in the general population. Rather, the opinions of both subpopulations (as estimated from the samples) should be weighted by their frequency within the general population, or lacking this information, be given equal weight. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.