Empirical Direction in Design and Analysis

By Norman H. Anderson | Go to book overview
Save to active project


A useful way to look at data is in terms of variance: If the differences between our treatment means are large—compared to individual differences within each treatment condition—this is a sign of real treatment effects.

This intuitive decision process is made precise in this chapter through analysis of variance (Anova). Formulas are presented for the variance between treatment means (MSbetween) and for the variance of the individual responses within a single treatment (MSwithin). The ratio, F = MSbetween/MSwithin, becomes our decision guide. A larger F is stronger evidence for real differences between our treatments. If F is “large enough, we have a statistically significant result, provisional evidence for real treatment effects.

Anova makes “large enough” precise. In terms of Chapter 1, your F ratio is an index of reliability, that is, the reliability of the observed differences between the treatment means.

In practice, a significance test is easy. Just give your data to the computer. It will calculate your F ratio and tell you whether it is “large enough.”

The F test applies to two or more conditions; it includes the t test as a special case. Further, MSwithin = s2, which may be used to construct confidence intervals using the expressions in Chapter 2. Confidence intervals can be very helpful to your reader when you describe your data.

Anova depends on certain assumptions. Two of these, normal distribution and equal variance, are not usually problematic in experimental studies. Independence is critical, but can usually be ensured through careful procedure and random assignment. Practical aspects of How to Randomize are discussed in the appendix beginning on page 77.

It cannot be emphasized too much that the statistical significance test says nothing whatever about substantive significance of your results. The significance test merely tells whether your result has some minimum degree of reliability. This is a minimum first step; unless the result is reliable, there is little point in worrying what it might mean.

Questions of meaning, however, are primarily extrastatistical. Questions of meaning involve considerations at lower levels of the Experimental Pyramid of Chapter 1. Statistics can help with some of these questions, as will be seen in later chapters, but this help requires going beyond the significance test to issues of experimental design.


Notes for this page

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
Loading One moment ...
Project items
Cite this page

Cited page

Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited page

Bookmark this page
Empirical Direction in Design and Analysis
Table of contents

Table of contents



Text size Smaller Larger
Search within

Search within this book

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

While we understand printed pages are helpful to our users, this limitation is necessary to help protect our publishers' copyrighted material and prevent its unlawful distribution. We are sorry for any inconvenience.
Full screen
/ 864

matching results for page

Cited passage

Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited passage

Welcome to the new Questia Reader

The Questia Reader has been updated to provide you with an even better online reading experience.  It is now 100% Responsive, which means you can read our books and articles on any sized device you wish.  All of your favorite tools like notes, highlights, and citations are still here, but the way you select text has been updated to be easier to use, especially on touchscreen devices.  Here's how:

1. Click or tap the first word you want to select.
2. Click or tap the last word you want to select.

OK, got it!

Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.

For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

Already a member? Log in now.

Are you sure you want to delete this highlight?