Three topics are considered in this chapter that extend the Anova techniques of the previous chapter: confidence intervals, focused comparisons, and power.
Confidence intervals present the sample mean together with an estimate of its variability. A confidence interval is thus an ideal statistic; the interval is a better measure of central tendency than its middle point by itself. The interval summarizes the sample evidence in an efficient, useful way by including a visible index of uncertainty. This uncertainty is submerged in the F or t ratio, but brought out in the clear in the confidence interval.
Focused comparisons look for specific patterns in the data. Focused comparisons have two potential advantages: They can be more informative than the overall F; and they typically have greater power.
The simplest focused comparison is a comparison of two means. With more than two experimental conditions, a statsig F implies real differences but does not localize them. When grounded two-mean comparisons have been planned beforehand, therefore, the overall Anova should ordinarily be bypassed in favor of two-mean confidence intervals.
Another useful focused comparison is the linear trend, which looks for straight-line patterns in the data. In the example of Section 4.2, the linear trend reveals a real difference missed by the overall F. No less important, this trend analysis is informative about the pattern of treatment effects. Forethought in planning linear trend tests may increase the effectiveness of your experiment at little cost.
Power is critical. Without adequate power, an experiment lacks value as evidence; it is a waste, at best. Some experiments are doomed to failure because of inadequate power. Such failures can often be avoided by making a preliminary guesstimate of power. If this guesstimate is too low, power can be increased in various ways. Alternatively, a better experiment may be found.
Everyone considers power, in effect, when they guess how many subjects to run. These intuitive guesses often do well. Still, a simple formula is available that can often improve these intuitive guesses. This power formula is the ounce of prevention that avoids the pound of woe.
The most important aspect of power is not statistical, but extrastatistical—how to increase power. Power depends on empirical specifics, especially on procedures that reduce response variability. This aspect of power is discussed in the final section, Nine Ways to Increase Power.