Blind Research: Are the Hard Sciences Immune from Experimenter Effects?

Article excerpt

IN SCIENTIFIC RESEARCH, AS IN EVERYDAY life, our beliefs and biases often influence how we observe and interpret the world. In experimental psychology and clinical research, this problem is widely recognized, which is why experiments in these subjects are often carried out under blind or double-blind conditions. There is solid experimental evidence that experimenters' attitudes and expectations can influence the outcome of experiments.

In single-blind experiments, an investigator does not know which samples or treatments are which. But when human subjects are involved, as in medicine and experimental psychology, double-blind procedures can be used to guard against the expectancy of both subjects and investigators. In a double-blind clinical trial, for example, some patients are given tablets of a drug and others are given similar-looking placebo tablets, pharmacologically inert. Neither clinicians nor patients know who gets what.

In such experiments, the largest placebo effects usually occur in trials in which both patients and physicians believe a powerful new treatment is being tested. The inert tablets tend to work like the treatment being studied, and can even induce its characteristic side-effects. Likewise, experimenter expectancy effects are well known in experimental psychology, and also show up in experiments on animal behavior.

It is fascinating to learn that blind assessment first began in the late eighteenth century, in which the first blind experiments were carried out to evaluate mesmerism. They were literally conducted with blindfolds, and took place in France at the house of Benjamin Franklin, the American minister plenipotentiary, who was head of a commission of inquiry appointed by King Louis XVI. (The report is translated and reprinted in its entirety in SKEPTIC, Vol. 4, No. 3.)

The use of blind assessment was adopted in the mid-nineteenth century by homeopaths, and by the end of that century was taken up by psychologists and psychical researchers. But it was not until the 1930s that blind techniques combined with no-treatment control groups started to be used in clinical trials, and only after World War II did blind assessment in randomized controlled trials became a standard technique.

In medicine and psychology, blind experimentation began as a deterrent against the unconventional, but its general importance is now recognized for orthodox research. It has been internalized within the mainstream. Although researchers in medicine and psychology have been aware of the effects of experimenters' expectations for decades, how widely has this awareness spread throughout the scientific community? Can the expectations of experimenters affect their results in other branches of science? No one seems to know. Most people simply assume that scientists in orthodox fields of inquiry are immune from the problem.

My colleagues and I have attempted to quantify attention to possible experimenter effects in different branches of science by means of two surveys. The first survey was of experimental papers recently published in leading scientific journals, including Nature and the Proceedings of the National Academy of Sciences.

In the physical sciences, we found no blind experiments in any of the 237 papers we reviewed. In the biological sciences, there were 7 blind experiments out of 914 (<1%); in psychology and animal behavior, 7 out of 143 (5%); and in the medical sciences, 55 out of 227 (24%). By far the highest proportion (but the smallest sample) was in parapsychology, 23 out of 27 (85%).

In the medical journals, out of the 55 reports involving blind methods, only 25% (11 of the total surveyed) involved double-blind trials. Thirty employed single-blind methods, with one or more of the investigators carrying out blind evaluations or analyses. The majority of the papers involved no blind methods. (The journals surveyed were the American Journal of Medicine, Annals of Internal Medicine, British Journal of Clinical Pharmacology, British Medical Journal and New England Journal of Medicine. …