Academic journal article JITTA : Journal of Information Technology Theory and Application

Content Validity of Instruments in Is Research

Academic journal article JITTA : Journal of Information Technology Theory and Application

Content Validity of Instruments in Is Research

Article excerpt

ABSTRACT

This paper presents a literature review of content validity, summarizes qualitative and quantitative approaches for content validity assessment, and explores the extent to which IS researchers have content validated their developed instruments. An extensive review of scale development papers published in five major IS journals between 1989-2005 revealed that the proportion of published studies reporting content validity had indeed increased; however, qualitative assessment of content validity remains the preferred approach. To encourage the utilization of an alternative approach to content validity assessment, this paper describes a quantitative approach to evaluating the content validity of the B2E portal user satisfaction instrument.

INTRODUCTION

A valid instrument is one which measures what it is supposed to measure (DeVellis 2003). It also enables researchers to interpret variables and the relationships between variables in a more theoretically meaningful fashion (Bagozzi 1980). Therefore, the development of a valid instrument is the most fundamental aim of any instrument developer.

The issue of whether IS researchers sufficiently validate their instruments was initially raised by Straub (1989) who reported that only a few had devoted serious attention to validation. His work was replicated by Boudreau, Gefen, and Straub (2001) who investigated the extent to which IS researchers had responded to Straub.s (1989) suggestion. Their findings revealed that the number of empirical studies reporting instrument validation had indeed increased since 1989. However, the number of studies reporting content validity was the lowest among all validity types investigated. This trend is supported by Straub, Boudreau, and Gefen (2004) who affirmed that content validity is indeed infrequently assessed in IS research. This is surprising as it should be the first type of validity to be established prior to examining other types of validity when developing instruments (Ebel 1967).

This paper responds to the call for more IS research on instrument validation, particularly on content validity. It presents theoretical literature on content validity including different approaches for assessing content validity. It also investigates the extent to which IS researchers have considered content validity when developing new instruments. Finally, the use of a quantitative approach for assessing content validity of the B2E portal user satisfaction instrument is described.

CONTENT VALIDITY THEORY

Haynes, Richard, and Kubany (1995 p.238) defined content validity as "the degree to which elements of an assessment instrument are relevant to, and representative of, the targeted construct for a particular assessment purpose.. Content validity can be established through the application of a two-stage process: development and judgement (Lynn 1986). The former is divided into three sequential steps: domain identification, item generation, and instrument formation (Carmines and Zeller 1979). Initially, the construct should be conceptually defined based on literature. A set of items is then generated and these items are arranged in a suitable sequence for the next stage of preparation. The judgement process, the primary goal of content validation, involves asking a specific number of experts to evaluate the validity of individual items and the whole instrument. The aim of this process is to retain the best items which are believed to adequately measure a desired content domain (Grant and Kinney 1992). In assessing experts. feedback, qualitative or quantitative approaches can be utilized.

The main difference between qualitative and quantitative approaches lies in the method used to determine when the finalised items - after modifying, deleting, or adding the original generated items - are adequate to measure the targeted construct. When applying qualitative methods, final decisions are generally obtained after all panel experts arrive at a consensus. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.