Design and Analysis of Single-Case Research

3.00; in other words, at .0027. The study is stopped when the null hypothesis of no difference between conditions can be rejected or when the maximum duration is reached, whichever occurs first. Via Monte Carlo simulation, we computed the total Type-I error probability for this procedure. Using a nominal α of .05, the total a was only .0599. Table 10.5 shows some representative results from simulations we conducted under various alternative hypotheses. Effect size is expressed as the standardized mean difference (d); total N is the total number of observations required to achieve the level of power labeled "analytic power," calculated with standard power analysis software ( Gorman, Primavera, & Allison, 1995); early N is the average number of observations that actually needed to be collected over 1,000 simulated experiments; and actual power is the proportion of times a significant result occurred over the 1,000 simulations.

What is apparent is that the power is slightly greater with the early stopping rule and, more important, there is an average saving of approximately 12% on trial costs. Berntsen et al. ( 1991) discussed the possibility of further reducing average trial costs by incorporating "futility tests" in which one can abandon a trial midway through when interim analyses suggest that there is virtually no chance of obtaining a significant result, even if the trial was extended to its maximum.

Of course, it should be noted that the exact results reported here apply only to the particular early stopping rule described. But the point is that judicious use of an appropriate stopping rule often will reduce the total number of required observations.

In conclusion, it can be seen that there is no substitute for a well-designed, well-powered study. The prudent investigator will devote as much time to statistically planning his or her study as to analyzing the study. We hope that this chapter makes the task a bit more tractable.

-367-