Statistical Logic and Choosing a Statistic: An Overview of the Role of Statistics in Psychological Research
The need for statistical tests arises when the data are less than perfectly reliable and are highly variable. As a possibly familar example, suppose that you had a bathroom scale that was unreliable. If you got on and off it a few times in succession, you might find that sometimes you were a pound above or a pound below the last reading. If you only got on it once, read the weight, changed your diet for a week, then got on the scale again and found that you had gained or lost a pound, you would not know whether the random variations in the scale's readings were responsible or whether you had actually changed in weight.
As mentioned in the preceding chapter, most psychological tests have reliability problems. Assume a person is seen in a psychological clinic, is given a test, and is diagnosed as depressed. The individual is treated with a new form of therapy, retested, and is found to be less depressed. Has the new treatment been effective, or is this a case of random variation in the scores?
The numbers produced by psychological tests give us a general idea of people's relative standing, but they do not identify exact true scores. Random variability in the measurement process adds to or subtracts from the true score. Random variability is unpredictable variability caused by chance factors, or caused by such a multiplicity of varied and changing influences that the precise contribution of random variability to the scores cannot be determined. This is why