Use, Misuse, and Abuse of Content Analysis for Research on the Consumer Interest
Carlson, Les, The Journal of Consumer Affairs
While possessing a long and respected history as a research method, content analysis studies attempting to address issues of the consumers' interests often are published with invalid conclusions or implications. This essay offers that the source of these problems is rooted in inferring causal relations from what is nothing more than descriptive data.
Content analysis is a methodological technique that I have used extensively in my own research career and one that I often find in papers that I review. Because of these realities, I believe I have developed a background and expertise about content analysis that qualifies me to comment on how it is being used by other researchers. While content analysis can be and has been applied successfully in any number of research settings, I also believe that there is potential for its abuse, particularly regarding what content analysis results can and, even more importantly, cannot infer to researchers, public policy officials, and other individuals who are interested in findings that are based on content analysis applications.
Rather than reviewing the literature in an effort to identify exemplary examples of where content analysis has been used appropriately, it could be more instructive to more broadly expound on the uses, misuses, and abuses of the research method itself. Content analysis can be misused and even abused when applied to the questions and investigative work that is of interest to readers of this journal as well as other publications that feature consumer interest research and the related issues of public policy.
This concern about content analysis has become increasingly important to me because of a disconcerting sameness that I have begun to find in at least some of the research that I read that features this methodology. Consequently, certain of my reviews of this work have begun to exhibit similar thoughts, words, and phrases irrespective of the content analysis topic or focus because the problems that I identify occur again and again across papers I read. Because of this reoccurrence of errors in applying content analysis, I have started to crystallize in my own mind what I believe are mistakes in how content analysis results are interpreted. Hence, the purpose of this article is to convey those thoughts, and consequently, it is my hope that this note will alert researchers about certain pitfalls that are being manifested in content analysis-based investigations. These mistakes can be easily avoided and, for reasons to be explicated below, must be evaded because of the potential adverse consequences that arise when this technique is misused and misapplied.
I will begin by building a research exemplar that reflects the types of problems that I believe are being manifested in some content analysis research. The scenario that is repeated in content analysis research goes something like this. A public policy or consumer interest "problem" is cited that in almost every instance is well documented, factual in nature, current, and (most importantly) generally recognized as an issue that needs to be resolved. While I could cite actual problems that have characterized content analysis research I have read recently, for the sake of protecting the review process regarding those manuscripts, I will base my hypothetical scenario on a fictitious "problem" for which I have not reviewed a manuscript, at least not recently.
In the hypothetical scenario under consideration, the problem to be considered has to do with advertising heavily sugared foods to children. In the not-so-distant past, this issue was at the forefront of public policy research and debate. In my exemplar, the problem is described and documented; at one time, many of the foods being advertised to children did (and perhaps still do) contain ingredients, such as sugar, that are not necessarily conducive to healthy development in children.
The problem documentation is then followed by a discussion on a theoretical component that could be useful in explaining how children process ads for heavily sugared foods for which kids are the target. …