Toward an Understanding of MIS Survey Research Methodology: Current Practices, Trends, and Implications for Future Research

Article excerpt

ABSTRACT

This study is a comparison of survey research methodology applied by research papers published during 1996 to 1999 on three major MIS journals. Through a content analysis of articles from MIS Quarterly (MISQ), Journal of Management Information Systems (JMIS) and International Journal of Information Management (IJIM). There are 75 papers that adopted questionnaire survey as the major research methodology to study for MIS research issues. With the consideration of PK list, SMA and other textbooks about business research methods, the authors have developed a checking list containing essential procedures to perform a questionnaire survey research. It consists of six parts: demographics, research design, questionnaire development, sampling procedure, data collection, and data analysis methods. Results indicate some potentially important trends in reports of research methodologies and the methodology usage is generally improved after analyzing our recording data and comparing with early assessment studies.

INTRODUCTION

A multisite, multisource research methodology is suggested for students of information management in order to attain generalizability and statistical significance in reporting findings. Such data may inform us of whether methods are becoming more standardized or more diversified over time. Examination of such patterns may provide some insights into the possible future development of research methodology. This paper describes methodologies that are appropriate for investigating the above and other contingency approaches in MIS field. It captures the dynamic aspects of environmental evolution and competitive changes and act on by changing patterns of resource allocation.

The purpose of this article was to identify possible trends in the use of various research methodologies in MIS studies. The rigor of survey research methodology had been discussed by several researchers. Pinsonneault and Kraemer (1993) considered that those investigations were done from three perspectives: constructing and illustrating research methodologies, examining the usage of methodology and identifying appropriate situation for applying survey research. This study is to investigate the survey research methodology applied to research papers published during 1996 to 1999 on three major MIS journals, including MIS Quarterly (MISQ), Journal of Management Information Systems (JMIS) and International Journal of Information Management (IJIM). There are 75 papers that adopted questionnaire survey as the major research methodology to study MIS related issues.

Pinsonneault and Kraemer (1993, will be abbreviated as PK later) had implemented an assessment of survey research methodology. They collected 122 MIS research papers published on sixteen journals during 1980 to 1990 and designed a list (will be called PK list later in this paper) to record how the research methodology were processed and described. The PK list examined survey methodologies mainly on research design, sampling procedures, and data collection. They concluded five general misapplications suffered for the survey research: (1) single-method designs where multiple methods were needed. (2) Unsystematic and often inadequate sampling procedures. (3) Low response rate. (4) Weak linkages between units of analysis and respondents. (5) Overreliance on cross-sectional surveys where longitudinal surveys were really needed.

Grover, Lee and Durand (1993, will be abbreviated as GLD) developed a list called SMA (Survey Methodological Attributes), that consists of nine Boolean statements about the technical processes of the survey methodology, such as sampling, data collection, validity and reliability of the instrument (The SMA list is included in the end of Appendix). They used SMA to evaluate 227 MIS research papers published during 1980 to 1989. The results showed that most papers have scores between two and four, and the mean score is 3. …