Selecting a Quantitative or Qualitative Research Methodology: An Experience
Sogunro, Olusegun A., Educational Research Quarterly
Olusegun A. Sogunro, Central Connecticut State University
The selection of an appropriate research method has always been a dilemma for many researchers and evaluators. While the quantitative-- qualitative research debate ravages, what is obvious is that there is no one best research method for all research and evaluations. Different research purposes require the use of different research methods, separately or in concert with each other. For all practical purposes, both quantitative and qualitative methods have different, but complementary roles to play in a research process and outcome. This paper explains the experience of the author in using a mixture of the two research approaches to evaluate a leadership training program.
The fray between champions of these two distinguishable research approaches is essentially ideological and political. Basically, the two approaches differ in their ways of conducting research, and each tends to claim superiority over the other. Ironically, each tradition overtly discredits the other as if it is infallible. The stage is always charged so that, given the chance, these champions would fight at any setting to defend their research philosophies. Fueling this charged situation is the subconscious luring of graduate students into these dichotomous camps of research methodologies and paradigms, especially from the standpoint of the research orientations of the professors - instructing or advising. This paper presents my experience as a researcher, using both quantitative and qualitative research methods.
Creswell (1994) defined a quantitative research as "an inquiry into a social or human problem, based on testing a theory composed of variables, measured with numbers, and analyzed with statistical procedures, in order to determine whether the predictive generalizations of the theory hold true" and a qualitative research as "an inquiry process of understanding a social or human problem, based on building a complex, holistic picture, formed with words, reporting detailed views of informants, and conducted in a natural setting" (pp. 1-2). In a very simplistic form, Punch (1998) defined quantitative research as an "empirical research where the data are in the form of numbers" and qualitative research as an "empirical research where the data are not in the form of numbers" (p. 4). Gay and Airasian (2000) defined quantitative research as "the collection of numerical data in order to explain, predict and/or control phenomena of interest" and qualitative research as "the collection of extensive data on many variables over an extended period of time, in a naturalistic setting, in order to gain insights not possible using other types of research (p. 627).
While both research approaches are equally recognized and used in conducting research, the major differences between them are in the areas of data collection and analyses. According to Gall, Gall & Borg (1999), quantitative research "rely heavily on numerical data and statistical analysis." In contrast, qualitative research "make little use of numbers or statistics but instead rely heavily on verbal data and subjective analysis" (p. 13).
My Experience in Using a Mixture of Quantitative and Qualitative Research Approaches
In the course of undertaking an evaluation study toward my dissertation, it became apparent that the suggestions given to me by my advisors were largely based on their professional preparations, interest or orientations. For instance, one professor suggested the use of questionnaire for data collection while the other suggested that the use of interviews alone would suffice. However, based on my curiosity to explore the two research approaches, I adopted the mixed methodology approach.
Between January 1994 and December 1995, I conducted an evaluation study of the impact of a leadership training program on the participants. The program was organized by the Rural Education Development Association (REDA) of Alberta, Canada. The evaluation study served as my doctoral dissertation. I observed and participated directly in the two-week training program in January 1994 for Levels I and II.
Although the evaluation study involved extensive data collection and processing, I was able to obtain a more comprehensive data and interpretation of the data by combining both quantitative and qualitative research strategies. Reichardt and Cook's (1979) belief that "researchers cannot benefit from the use of numbers if they do not know, in common sense terms, what the numbers mean" (p. 23), was valid in this study. For instance, meanings were drawn from both quantitative and qualitative data by "going back and forth, progressively clarifying the findings of one with those of the other" (Linn, cited in House, 1994, p. 19). Through content analysis (i.e., coding, frequency counts, and ranking), meanings were given to qualitative data, while through means, and standard deviations, quantitative data were made more meaningful.
Since the focus of the evaluation study was to determine the impact of the leadership training program on the participants, there was the need to go beyond the collection of specific data, such as goal identification and accomplishments through distant collection of hard data, to a closer and interactive collection of soft data which embraces getting testimonials of program impact from the program participants and stakeholders as well as taking cognizance of emerging concerns and issues with a view to coming to terms with what to do to enhance the program. Conceived in this way, the use of the twin approaches cannot be easily divorced from each other. Table I represents a summary of some of the differing characteristics of both research methods as garnered from the literature and experienced in the course of conducting my evaluation study.
These differing characteristics are similar to those identified by Borg and Gall (1989), Creswell, (1994); Creswell, (1994); Hedrick, (1994); Reichardt and Rallis, (1994), Bogdan and Biklen (1998); Punch, (1998, 2000); Tashakkori and Teddlie, (1998); Gall, Gall, and Borg, (1999). While acknowledging the fact that some major differences exist between the two approaches, especially in the nature of their data and methods for collecting and analyzing data, Punch (1998), cautioned that "these differences should not obscure the similarities in logic, which makes combining the approaches possible" (p. 240).
The distinct differences between the two research approaches notwithstanding, pacifists or pragmatists from social and behavioral sciences have posited "that qualitative and quantitative methods are, indeed, compatible" (Tashakkori & Teddlie, 1998, pp. 4-5), and that this compatibility is manifested in most research in education and evaluation. Social and behavioral scientists branded the use of both qualitative and quantitative approaches or paradigms as "mixed methods" or "mixed methodology" or "methodological mixes" (Tashakkori & Teddlie, 1998). Overall, both quantitative and qualitative research paradigms have notable roles to play in the research of this type. According to Datta (1994),
Today, evaluation standards call for stakeholder involvement, and many evaluations begin with trying to understand what different-make that many different-- stakeholders see as the issues. That is a lesson taught to us all by qualitative methodologists. As another example, evaluation standards call for methodological transparency, making public what the evaluator did in the conduct of the study: measures, instance selection, data reduction and analysis, precautions taken to achieve quality, and limitations and strengths. That is a lesson taught to us all by quantitative methodologists. (p. 55)
From the foregoing, I tend to disagree with those who strongly support one method and condemn the other. Quite simply, the key rule is understanding the nature, and appropriateness of each of the two paradigms, and entering the research or evaluation arena with an open mind. In other words, the strategies selected should suit the nature of the research being undertaken rather than making selection based on biases. There are situations when questionnaires or laboratory experiments (quantitative instruments) offer the most feasible means of collecting data. There are also situations when interviews with, or observations (qualitative instruments) of the subjects provide the most practicable source of data collection. Perhaps because of my personal interest in creativity, critical reflection, and logical conclusion, the use of both qualitative and quantitative research methodologies provided a great experience for me.
However, the implication of this article is clear. To be a competent and diversified educational researcher, one must acquire knowledge of the two worlds of research methods and demonstrate proficiency in the use of both. As concluded by Creswell (1994), "It is advantageous to a researcher to combine methods to better understand a concept being tested or explored" (p. 177). Research being a truth-finding construct aimed at verifying and authenticating phenomena, evidence abounds that the use of a combination of both quantitative and qualitative research methods results in a stronger validity of outcomes. To ensure a well grounded research (i.e., a research that is reliable and valid), I believe that a good researcher must be skillful in the design of data collecting instruments, the analysis of the data and the interpretations of results.
Given that educational research studies generally involve both animates and inanimates, the needs to experiment, experience, observe, and/or to talk to the people involved become critical phenomena. Therefore, the use of numbers and descriptions, which anchor both quantitative and qualitative research paradigms, are mutually complementary, and the strengths of both can produce a research synergy whose collective benefits are greater than that obtainable from either approach taken alone. Eisner (1981) summarizes it all, "The field of education in particular needs to avoid methodological monism. Our problems need to be addressed in as many ways as will bear fruit" (p. 9).
Among other things, when faced with the question of which method to choose in conducting a research - quantitative or qualitative, the following factors are important for consideration: matching research purposes and questions with methods; depth of study of phenomena; availability of resources (money, time, etc); availability of supporting literature; 'knowledge pay off' (i.e., which approach will produce more useful knowledge); and 'style' or preference for a method (Punch, 1998); sample population; researcher's analytical skills; utility of findings, and accessibility to - situations, relevant data, and sample population, and so and so forth.
Although details about how to conduct research using a mixed methodology are important, I have attempted, in this article, to concentrate on relating my experience in the use of both quantitative and qualitative research methods in evaluating a leadership training program. For broader understanding of using both quantitative and qualitative research methods, scholars are referred to Guba & Lincoln, 1981; Creswell 1994; Reichardt and Rallis, 1994; Bogdan and Biklen, 1998); Punch, 1998; and Tashakkori and Teddlie, 1998, to mention a few.
Bogdan, R. C., & Biklen, S. K. (1998). Qualitative research for education: An introduction to theory and methods. Boston, MA: Allen & Bacon.
Borg, W. R., & Gall, M. D. (1989). Educational research: An introduction. (5th ed.) White Plains, NY: Longman.
Creswell, J. W. (1994). Research design: Qualitative and quantitative approaches. Thousand Oaks, CA: Sage.
Datta, L. (1994, Spring). Paradigm wars: A basis for peaceful coexistence and beyond. In Reichardt, C. S., & Rallis, S. F. (Eds.). The qualitative-Quantitative debate: New perspectives. New Directions for Program Evaluation, 61, 53-70.
Eisner, E. W. (1981). On the differences between scientific and artistic approaches to qualitative research. Educational Researcher, 10(4),5-9.
Fetterman, D. M. (1988, November). Qualitative approaches to evaluating education. Educational Researcher, 17(8), 17-23.
Gall, J. G. Gall, M. D., & Borg, W. R. (1999). Applying educational research: A practical guide. New York: Longman.
Gay, L. R. & Airasian, P. (2000). Educational research: Competencies for analysis and application (6th ed.). Upper Saddle River, NJ: Prentice Hall.
Guba, E. G., & Lincoln, Y. S. (1981). Effective evaluation: Improving the usefulness of evaluation results through responsive and naturalistic approaches. San Francisco, CA: Jossey-Bass.
Hedrick, T. E. (1994, Spring). The quantitative-qualitative debate: Possibilities for integration. In Reichardt, C. S., & Rallis, S. F. (Eds.). The qualitative-Quantitative debate: New perspectives. New Directions for Program Evaluation, 61, 45-52.
House, E. R. (1994, Spring). Integrating the quantitative and qualitative. In Reichardt, C. S., & Rallis, S. F. (Eds.). The qualitative-- Quantitative debate: New perspectives. New Directions for Program Evaluation, 61, 13-21.
Miles, M. B., & Huberman, A. M. (1984, May). Drawing valid meaning from qualitative data: Toward a shared craft. Educational Researcher, 13(5), 20-30.
Philips, J. J. (1990). Quantitative evaluation: Preparation for evaluation. In L. Nadler, & Z. Nadler (Eds.), The handbook of human resource development (2nd ed.), (pp. 12.3-12.32). New York: Wiley.
Punch, K. F. (2000). Developing effective research proposals. Thousand Oaks, CA: Sage.
Punch, K. F. (1998). Introduction to social research: Quantitative and qualitative approaches. Thousand Oaks, CA: Sage.
Reichardt, C. S., & Cook, T. D. (1979). Beyond qualitative versus quantitative methods. In T. D. Cook, & C. S. Reichardt (Eds.), Quantitative and Qualitative methods in evaluation research. Newbury Park, CA: Sage.
Reichardt, C. S., & Rallis, S. F. (1994, Spring). The relationship between the qualitative and quantitative research traditions. In Reichardt, C. S., & Rallis, S. F. (Eds.). The qualitative-Quantitative debate: New perspectives. New Directions for Program Evaluation, 61, 5-11.
Rossi, P. H. (1994, Spring). The war between the quals and the quants: Is a lasting peace possible? In Reichardt, C. S., & Rallis, S. F. (Eds.). The qualitative-Quantitative debate: New perspectives. New Directions for Program Evaluation, 61, 23-36.
Rural Education and Development Association (1994a, January 17-21). Intermediate leadership skills workshop: Level two. Edmonton, AB: Author.
Rural Education and Development Association (1994b, January 24-28). Introductory leadership skills workshop: Level one. Edmonton, AB: Author.
Sechrest, L. (1992). Roots: Back to our first generation. Evaluation Practice, 13(1), 1-7.
Smith, M. L. (1994, Spring). Qualitative plus/versus quantitative: The last word. In Reichardt, C. S., & Rallis, S. F. (Eds.). The qualitative-Quantitative debate: New perspectives. New Directions for Program Evaluation, 61, 37-44.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and qualitative approaches. Applied Social Research Methods Series, Vol 46. Thousand Oaks, CA: Sage.
Weiss, C. H. (1972). Evaluation research: Methods for assessing program effectiveness. Englewood Cliffs, NJ: Prentice-Hall.…
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Selecting a Quantitative or Qualitative Research Methodology: An Experience. Contributors: Sogunro, Olusegun A. - Author. Journal title: Educational Research Quarterly. Volume: 26. Issue: 1 Publication date: September 2002. Page number: 3+. © Educational Research Quarterly Dec 2008. Provided by ProQuest LLC. All Rights Reserved.
This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.