Designs, Participants, and Measurement Methods in Psychological Research
Bodner, Todd E., Canadian Psychology
Critical reviews of psychological scholarship suggest that self-report questionnaires, experimental designs, and college students dominate psychological research. Although researchers within specific subfields of psychology have the requisite knowledge to assess the generality of these concerns, novices of psychological research do not. To provide such knowledge, we surveyed a random sample of 200 journal articles in the PsycINFO database coding for psychological content area, research design, measurement method, and participant type. Results indicate self-report questionnaires, experimental designs, and college students each appeared in a minority of studies and these study characteristics often varied significantly by content area. Results also suggest no single dominant "typical" study in psychological research but rather that the characteristics of such research exhibit remarkable diversity.
What features characterize a "typical" empirical study in psychological research? How "typical" is that kind of study? Answers to these motivating questions will vary depending on one's schema of psychological research. Schemata are mental structures that organize prior knowledge, guide current perceptions, and provide future expectations (Fiske & Taylor, 1991). Research suggests that experts have more complex (and presumably more accurate) schemata in their areas of expertise compared to novices (Newell & Simon, 1972). Thus, researchers in psychology, students in college, and individuals outside of academia are likely to answer these questions about psychological research differently. Indeed, past research suggests that people hold many misconceptions about psychology and psychologists (Friedrich, 1996; Nauta, 2000; Rosenthal, McKnight, & Price, 2001; Rosenthal, Soper, Rachal, McKnight, & Price, 2004; Webb & Speer, 1985; Wood, Jones, & Benjamin, 1986), that correcting these misconceptions is difficult (Gutman, 1979; Lamal, 1979; Vaughn, 1977), but that these misconceptions decrease with greater levels of education and expertise (Gardner & Hund, 1983).
The purpose of this article is to answer the two motivating questions empirically to correct potentially inaccurate schematic representations of the characteristics of psychological research. We view this research as important because perceptions of psychology can affect the level of available funding by private and public agencies (Shaffer, 1977; Wood et al., 1986), the choice of undergraduate major and career by college students (Flowers & Schandler, 2006; Nauta, 2000), and the perceived usefulness of psychological research by the general public (Webb & Speer, 1985; Wood et al., 1986). We focus this investigation on the types of research designs, participants, and measurement methods used in psychological research because information sources from inside and outside of psychology may lead novices to inaccurate schematic representations about these particular features.
We are unaware of any formal research on portrayals of psychological research outside of psychology (e.g., in the entertainment industry, the news and print media, and everyday conversations). However, research suggests that portrayals of psychologists in film, radio, and television are often narrow and stereotypic in nature with particular emphasis given to the therapeutic and health services side of psychology (Fishman & Neigher, 1982; Flowers & Frizler, 2004; Flowers & Schandler, 2006). We conjecture that portrayals of psychological research (by novices to novices) would also be highly stereotypical in nature and not capture the range of features in actual psychological research. For example, stereotypical portrayals of psychological research in cartoons often involve a researcher in a white laboratory coat and rodents. Such stereotypic portrayals encourage a narrow view and potentially inaccurate schemata of psychological research. …