Confounding is a primary problem of experimental analysis. Most arguments in the literature revolve around issues of confounding, that is, alternative interpretations of an observed result. Much of the thought that goes into planning an experiment is concerned with how to avoid or control confounds.
Confounding is a basic connection between empirics and statistics portrayed in the Experimental Pyramid of Chapter 1. Confounding depends first on choice of task and response measure, second on experimental design. A statsig result is only a minimal first step; the important question is what it means. What it means hinges on questions of confounding at each lower level of the Pyramid.
Some important functions of statistics appear in dealing with confounding. These functions depend on empirical judgment melded with concepts and techniques of statistics. To make statistics an organic part of empirical investigation depends on empirical appreciation of confounding.
This chapter includes a broad collection of examples of confounding, chosen to illustrate common difficulties that need consideration when planning an experiment. One purpose of these examples is to help develop a bird's-eye perspective on confounding.
Among these examples are some in which psychological science went astray. Careless or even culpable neglect of confounds has sometimes led to fruitless controversy, to wasted effort by many workers, and to untruths in introductory texts. These lessons from the past can help us increase effectiveness of our future work.
These examples also illustrate that knowledge about confounds is a basic part of knowledge in any field. Some variables initially considered important turn out to be irrelevant; unsuspected variables are sometimes found to be important. The worth and quality of your work depend first on a good choice of problem and second on your knowhow about confounds and ways to deal with them. Such knowhow is part of the cumulative knowledge in each field.
The last part of this chapter discusses five ways to deal with confounding. Four of these concern experimental procedure and experimental design. The fifth is theory control, which can sometimes handle confounds not tractable with experimental manipulations.