# Empirical Direction in Design and Analysis

By Norman H. Anderson | Go to book overview

NOTES
18.1a
This discussion of effect size and importance is based on Anderson (1982), Chapter 6).
18.1b
The need for empirical rather than statistical framework is striking with the issue of effect size. Numerous writers have extolled the virtue of going beyond the significance test to report some measure of effect size. This theme seems totally persuasive—until one looks at the measures of effect size that have been proposed. They smother the empirical effect under an additional blanket of statistics.

A total of 40 measures of effect size were collected from the literature by Kirk (1996) and are listed in his Table 1. Notably absent from this list are the most important indexes: the mean, the one-variable regression coefficient, and their confidence intervals. This state of affairs is a symptom of the pervasive orientation that fixates on statistics to the neglect of empirics.

Questions of size and importance have no easy answer because they are basically extrastatistical issues. They need to be addressed in empirical terms, within the framework of each experiment. Focus on statistical indexes obscures the real issue. See further Importance Indexes and Self-Estimation Methodology in Chapter 6 of Anderson (1982) as well as Anderson and Zalinski (1991). Also of interest are Kruskal and Majors (1989) and Wright (1988).

18.1.1a
That small effects can cumulate across many instances to be important for outcome analysis has been emphasized by numerous writers, including Gilbert, Light, and Mosteller (1975), Yeaton and Sechrest (1981), and Abelson (1985). Prentice and Miller (1992) point up importance of small effects in process analysis.
18.1.2a
The d index of Equation 3 is not a general measure of effect size because effect size is a substantive concept that must be understood in extrastatistical terms. The same d of.50 would mean different things in experiments on person cognition, verbal memory, animal learning, and even in two different experiments within any of these fields. Cohen (1988) covers over this substantive problem by classifying d values of.20,.50, and.80 as “small, ” “medium, ” or “large.” This arbitrary classification obscures the prime importance of interpreting the mean difference—in its unstandardized form—within its own empirical framework.
18.1.2b
A much-cited non sequitur that published empirical reports generally have inadequate power was initiated by Cohen (1962) on the basis of his small-medium-large classification of effect sizes cited in the previous note. This claim was a non sequitur because, as Mulaik, Raju, and Harshman (1997) point out, Cohen did not calculate power from the data of any empirical report.

Instead, Cohen tabulated sample size for each report. This sample size was used to estimate power for the three cited hypothetical effect sizes of.20,.50, and.80 using Equation 3 (or formulas for comparable measures of effect size such as r). Cohen found that power averaged just under.50 for a “medium” effect size. He arbitrarily assumed that a “medium” effect size was the norm for empirical studies and so concluded that published experiments generally lack power.

-589-

If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

#### Items saved from this book

This book has been saved
Highlights (0)
Some of your highlights are legacy items.
Citations (0)
Some of your citations are legacy items.
Notes (0)
Bookmarks (0)

You have no saved items from this book

Project items include:
• Saved book/article
• Highlights
• Quotes/citations
• Notes
• Bookmarks
Notes

#### Cited page

Style
Citations are available only to our active members.
Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

#### Cited page

Empirical Direction in Design and Analysis

• Title Page iii
• Dedication v
• Foreword vi
• Contents vii
• Preface xvi
• Chapter 1 - Scientific Inference 1
• Preface 30
• Chapter 2 - Statistical Inference 31
• How to Do Exercises 54
• Exercises for Chapter 2 54
• Preface 58
• Chapter 3 - Elements of Analysis of Variance I 59
• Notes 75
• Appendix: How to Randomize 77
• Exercises for Chapter 3 84
• Preface 90
• Chapter 4 - Elements of Analysis of Variance II 91
• Notes 111
• Exercises for Chapter 4 113
• Preface 118
• Chapter 5 - Factorial Design 119
• Notes 145
• Appendix: Hand Calculation for Factorial Design 148
• Exercises for Chapter 5 151
• Preface 158
• Chapter 6 - Repeated Measures Design 159
• Notes 177
• Exercises for Chapter 6 181
• Preface 188
• Chapter 7 - Understanding Interactions 189
• Notes 209
• Exercises for Chapter 7 214
• Preface 218
• Chapter 8 - Confounding 219
• Notes 250
• Preface 258
• Chapter 9 - Regression and Correlation 259
• Notes 280
• Exercises for Chapter 9 282
• Preface 286
• Chapter 10 - Frequency Data and Chi-Square 287
• Notes 300
• Exercises for Chapter 10 302
• Preface 306
• Chapter 11 - Single Subject Design 307
• Notes 338
• Exercises for Chapter 11 345
• Preface 350
• Chapter 12 - Nonnormal Data and Unequal Variance 351
• Notes 373
• Exercises for Chapter 12 378
• Preface 382
• Chapter 13 - Analysis of Covariance 383
• Notes 395
• Exercises for Chapter 13 397
• Preface 400
• Chapter 14 - Design Topics I 401
• Notes 431
• Exercises for Chapter 14 437
• Preface 442
• Chapter 15 - Design Topics II 443
• Notes 475
• Exercises for Chapter 15 481
• Preface 484
• Chapter 16 - Multiple Regression 485
• Notes 514
• Exercises for Chapter 16 520
• Preface 524
• Chapter 17 - Multiple Comparisons 525
• Notes 546
• Exercises for Chapter 17 548
• Preface 550
• Chapter 18 - Sundry Topics 551
• Notes 589
• Exercises for Chapter 18 596
• Preface 602
• Chapter 19 - Foundations of Statistics 603
• Notes 637
• Preface 646
• Chapter 20 - Mathematical Models for Process Analysis 647
• Notes 677
• Exercises for Chapter 20 681
• Preface 688
• Chapter 21 - Toward Unified Theory 689
• Notes 729
• Exercises for Chapter 21 742
• Preface 750
• Chapter 22 - Principles and Tactics of Writing Papers 751
• Notes 761
• Preface 764
• Chapter 23 - Lifelong Learning 765
• Notes 780
• Preface 782
• Chapter 0 - Basic Statistical Concepts 783
• Notes 803
• Exercises for Chapter 0 805
• Statistical Tables 808
• References 820
• Author Index 847
• Subject Index 854
Settings

#### Settings

Typeface
Text size Reset View mode
Search within

Look up

#### Look up a word

• Dictionary
• Thesaurus
Please submit a word or phrase above.

Why can't I print more than one page at a time?

Help
Full screen
/ 864

## Questia reader help

### How to highlight and cite specific passages

1. Click or tap the first word you want to select.
2. Click or tap the last word you want to select, and you’ll see everything in between get selected.
3. You’ll then get a menu of options like creating a highlight or a citation from that passage of text.

## Cited passage

Style
Citations are available only to our active members.
Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

## Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.