If the data analysis shows evidence for a real effect, natural next questions are: How big is the effect? How important is it? A significance test can provide evidence that the observed effect is real, not merely response variability, but this is a minor aspect of understanding the data.
Statistics texts, accordingly, have felt a need to go beyond tests of significance to consider size and importance. Numerical statistical indexes have been developed in attempts to answer these two questions.
Unfortunately, most indexes proposed to measure size and importance have little value. Size and importance are primarily substantive issues; to evaluate size and importance generally requires some substantive standard of comparison. Statistical indexes find it awkward or impossible to incorporate substantive standards. Instead of clarifying, they often obscure the data. b
Three kinds of size–importance indexes have been considered. The obvious kind looks at mean differences, and this kind is naturally useful. The second looks at percentages of variance, and the third refers to a mathematical model. These three are considered in Sections 18.1.2 to 18.1.4, following a preliminary overview in terms of the process–outcome distinction.
The process–outcome distinction of Section 1.2.1 gives a useful perspective on the size–importance issue: A small effect can be important, both for process and for outcome. But importance must be assessed in substantive, extrastatistical terms, although for different reasons in the two cases.