Three conceptual issues are considered in this chapter. The first is how probability should be conceptualized in statistical theory. Should probability be defined in terms of objective considerations that can command agreement for different individuals? Or should it include subjective belief, which must differ for different individuals?
The main line of statistics in this century has an objectivist foundation. In this framework, probability is typically defined in terms of observable frequencies of chance events. This objectivist framework underlies the content and presentation of standard texts.
Recently, however, subjectivist schools have been vigorously developed, the most popular being the Bayesian school. In these frameworks, probability is defined in terms of personal belief. This subjectivist view has challenged the foundation and the very way of thinking of the objectivist school.
It is becoming clear that the subjectivist approach is viable and fills a real need. In applied settings, it has notable advantages for incorporating auxiliary information in making judgments and decisions, some of which seem intractable with the objectivist approach. These applications, however, generally require very considerable expertise, illustrated with the three case examples of Bayesian analysis given on pages 613–618.
For experimental psychology, these theoretical controversies are of minor practical concern. With the common fixed factor designs, subjective Bayesian analysis with a noninformative prior leads to the same numerical results as traditional Anova. Also, Neyman–Pearson theory of confidence intervals and power has been smoothly assimilated into the original Fisherian framework, as Neyman and Pearson intended, and is now taken for granted.
The second conceptual issue concerns the significance test. It has been viewed very differently in the three theoretical schools of statistics considered in the first main section. Also, it has been harshly criticized from many empirical directions. Section 19.2 presents a pragmatic view, showing how the significance test performs a useful function in empirical science (pages 624 ff).
The third conceptual issue concerns psychological measurement theory, especially the concept of linear (equal interval) scale. Some writers have argued that a linear scale is an absolute prerequisite for Anova. Their argument is mistaken (Section 19.3, pages 630 ff). With randomization, valid tests require only a monotone scale. Randomization thus justifies most uses of Anova in experimental analysis.