Charles Lewis Educational Testing Service
Suppose you were reading a journal article describing the results of a repeated measures experiment and you cam across statements such as "the main effect of Alternation Rate was marginally significant, F(5, 115) = 2.06, p < .08, and the Playing Speed × Alternation Rate interaction was reliable, F(5, 115) = 3.48, p < .01. . . . The main effect of Playing Speed [was not] significant . . . F(1, 23) = 2.56, ns" ( Samuel, 1991, p. 396).1 Leaving aside any substantive interpretation of these effects, what would your reaction be to the statistical information that has been given? (For completeness, it should be noted that the 2 × 6 cell means that are being compared in the analysis are also plotted in an accompanying figure.)
To focus attention on the object of concern, suppose the reported p values were changed as follows: for Alternation Rate, from <.08 (actually .075) to .165; for the interaction, from <.01 (actually .006) to .075; and for Playing Speed, no change (current value .123). Would your reaction to the results change at all? The author would apparently now identify both main effects as ns, and the interaction as marginally significant. The same sort of changes could be made for the p values associated with any standard F test of a main effect or interaction involving a repeated measures factor with three or more levels. But wait a moment! Where did these new p values come from and what do they represent?
Almost 40 years ago, Box ( 1954) pointed out that correlated observations in the analysis of variance can have an impact on the probability of a Type I error for any F test used to test hypotheses about means. One place where correlated____________________