Academic journal article Attention, Perception and Psychophysics

"Plateau"-Related Summary Statistics Are Uninformative for Comparing Working Memory Models

Academic journal article Attention, Perception and Psychophysics

"Plateau"-Related Summary Statistics Are Uninformative for Comparing Working Memory Models

Article excerpt

Published online: 11 April 2014

© Psychonomic Society, Inc. 2014

Abstract Performance on visual working memory tasks decreases as more items need to be remembered. Over the past decade, a debate has unfolded between proponents of slot models and slotless models of this phenomenon (Ma, Husain, Bays (Nature Neuroscience 17, 347-356, 2014). Zhang and Luck (Nature 453, (7192), 233-235, 2008) and Anderson, Vogel, and Awh (Attention, Perception, Psychophys 74, (5), 891-910, 2011) noticed that as more items need to be remembered, "memory noise" seems to first increase and then reach a "stable plateau." They argued that three summary statistics characterizing this plateau are consistent with slot models, but not with slotless models. Here, we assess the validity of their methods. We generated synthetic data both from a leading slotmodel and from a recent slotless model and quantified model evidence using log Bayes factors. We found that the summary statistics provided at most 0.15%of the expected model evidence in the raw data. In a model recovery analysis, a total ofmore than a million trials were required to achieve 99 % correct recovery when models were compared on the basis of summary statistics, whereas fewer than 1,000 trials were sufficient when raw data were used. Therefore, at realistic numbers of trials, plateau-related summary statistics are highly unreliable for model comparison. Applying the same analyses to subject data from Anderson et al. (Attention, Perception, Psychophys 74, (5), 891-910, 2011), we found that the evidence in the summary statistics was at most 0.12 % of the evidence in the raw data and far too weak to warrant any conclusions. The evidence in the raw data, in fact, strongly favored the slotless model. These findings call into question claims about working memory that are based on summary statistics.

Keywords Visual workingmemory . Workingmemory . Model selection . Statisticalmethods

(ProQuest: ... denotes formulae omitted.)

The English novelist Samuel Butler stated that "life is the art of drawing sufficient conclusions from insufficient premises" (Jones, 1912). Nothing is truer in the empirical sciences, where data are generally noisy and time to collect them is limited. In such a setting, progress critically depends on the application of proper statistical techniques to assess the evidence that data provide for different candidate models. Virtually all reputable model comparison techniques are directly or indirectly based on the probability of the raw data (in psychophysics: individual-trial subject responses) given a hypothesized model and a hypothesized set of model parameters. This probability is called the likelihood of the model and its parameters, and common methods, like Bayes factors (Kass & Raftery, 1995), the Akaike information criterion (Akaike, 1974), the Bayesian information criterion (Schwartz, 1978), and the deviance information criterion (Spiegelhalter, Best, Carlin, & Van der Linde, 2002), are all derived from it.

In recent years, many papers have debated the nature of working memory limitations (including Alvarez & Cavanagh, 2004; Anderson & Awh, 2012; Anderson, Vogel, & Awh, 2011; Bays, Catalao, & Husain, 2009; Bays, Gorgoraptis, Wee, Marshall, & Husain, 2011; Bays & Husain, 2008; Buschman, Siegel, Roy, & Miller, 2011; Donkin, Nosofsky, Gold, &Shiffrin, 2013; Elmore et al., 2011; Fougnie, Suchow, & Alvarez, 2012; Fukuda, Awh, & Vogel, 2010; Heyselaar, Johnston,&Pare, 2011; Keshvari, Van den Berg,&Ma, 2013; Lara & Wallis, 2012; Luck & Vogel, 2013; Rouder, Morey, Cowan, Morey, & Pratte, 2008; Sims, Jacobs, & Knill, 2012; Van den Berg, Awh, & Ma, 2014; Van den Berg, Shin, Chou, George, & Ma, 2012; Wilken & Ma, 2004; Zhang & Luck, 2008), (For a review of the debate, see (Ma, Husain, Bays, 2014).). A key aspect of this debate has been whether or not there exists an upper limit to the number of items that can be held in visual workingmemory. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.