Empirical Direction in Design and Analysis

By Norman H. Anderson | Go to book overview
Save to active project

The advantages of confidence intervals over standard t and F tests have been noted by many (e.g., Cochran & Cox, 1957, p. 5). In psychology, Grant (1962), Cohen (1988), Loftus (1993), and Reichardt and Gollob (1997), among many others, have been strong advocates of confidence intervals.
With unequal n, the confidence interval for the difference between two independent means is

Note that the square root term is the standard deviation of the mean difference. You can check that this expression reduces to Expression 1a when n1 = n2.

A seeming peculiarity of the significance test is that an observed difference between two means may be statsig but opposite to the true difference. Rejecting H0 only implies that the true means are not equal, not which is greater. The obvious solution is to decide direction by visual inspection. When the true difference is very close to zero, visual inspection will lead to an incorrect directional conclusion with probability very close to ½ α. This has been reified as “Type III” error to distinguish it from the Type I error (false alarm) of claiming a real effect when the true effect is exactly zero. There is no way to avoid this possibility when the real effect is small.

This issue has been discussed at length by Leventhal and Huynh (1996). They argue for a directional version of the null hypothesis, but this differs little from standard practice based on visual inspection. Their main finding is that standard power calculations yield slight overestimates in selected extreme cases.

In the range view of the null hypothesis, Type I error includes Type III error as a special case. If the true effect is substantial, on the other hand, a wrong directional conclusion is improbable.

A coin toss helps clarify the difference between confidence and probability. Before we toss the coin, the probability of heads is ½. Now we toss the coin but do not look at the outcome. The outcome, however, is either heads or tails; there is no longer a chance element, which is essential to probability. In the frequentist view, our ignorance about the outcome is not probability.

But we may have 50% confidence in heads; even odds on the outcome is a fair bet. Confidence is thus a valid guide to action (see Confidence, Probability, and Belief, Section 19.1.2, page 607).

Usually, the linear trend will be superior to the overall F. However, the numerical example in the text unduly favors the trend test because these data are perfectly linear. If the data depart substantially from linearity, the overall F might have more power, although it would be less informative (see further Sections 18.2 and 18.3).
In a contrast, the weights must sum to zero. This can always be accomplished by subtracting the mean weight from each weight; the pattern remains unchanged. The weights for the linear trend example would thus be 1.5, −.5, +.5, + 1.5, obtained by subtracting the mean of 2.5 from the listed values, 1, 2, 3, 4 (Section 18.2).


Notes for this page

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
Loading One moment ...
Project items
Cite this page

Cited page

Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited page

Bookmark this page
Empirical Direction in Design and Analysis
Table of contents

Table of contents



Text size Smaller Larger
Search within

Search within this book

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

While we understand printed pages are helpful to our users, this limitation is necessary to help protect our publishers' copyrighted material and prevent its unlawful distribution. We are sorry for any inconvenience.
Full screen
/ 864

matching results for page

Cited passage

Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited passage

Welcome to the new Questia Reader

The Questia Reader has been updated to provide you with an even better online reading experience.  It is now 100% Responsive, which means you can read our books and articles on any sized device you wish.  All of your favorite tools like notes, highlights, and citations are still here, but the way you select text has been updated to be easier to use, especially on touchscreen devices.  Here's how:

1. Click or tap the first word you want to select.
2. Click or tap the last word you want to select.

OK, got it!

Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.

For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

Already a member? Log in now.

Are you sure you want to delete this highlight?