Academic journal article Human Factors

A Paired Comparison Method for Interval Scaling

Academic journal article Human Factors

A Paired Comparison Method for Interval Scaling

Article excerpt

INTRODUCTION

There are times when a study examines subjective information about a set of items, opinions, or situations. Rank ordering the set of items, opinions, or situations provides only basic information about how important they are to a research participant. The rank ordering provides ordinal data, and many nonparametric statistical techniques have been developed to analyze rank-ordered data. There are times, however, when the researcher needs subjective information that has the properties of interval data in order to perform parametric statistical analyses or to relate the subjective information to objective information that was measured on an interval scale.

Current paired-comparison techniques collect subjective information and generate relative importance information as ratio data (Saaty, 1980; Vidulich, 1989). An example of relative importance ratio data would be the case in which a subject considered Item A to be twice as important as Item B. The importance is relative given that the values for both Item A and Item B are unknown. The benefit of ratio data is that they provide information beyond merely rank order information; they provide information about how far apart a person perceives the items to be. For example, two people can rank three items (A, B, and C) in the same order. However, one person can view Item A to be twice as much as Item B and five times as much as Item C, whereas the second person can view Item A to be four times as much as Item B and six times as much as Item C.

The problem is that this type of ratio data (not to be confused with ratio data that refer to interval data that have a true zero point) cannot be analyzed using parametric statistical analyses. Ratio data can be analyzed only using multiplication and division and cannot be analyzed using addition and subtraction (which are used extensively in parametric statistics). This article presents a new paired-comparison technique, the linear paired-comparison (LPC) method, which produces relative importance information as interval data so that they can then be analyzed using parametric statistics. Paired comparisons have been successfully used in such diverse evaluations as measuring the world influence of nations (Saaty, 1980), assigning overhead costs to products (Partovi, 1991), and evaluating workload (Vidulich, 1991). When a measure of the relative importance between each of several conditions is desired, such as in the case of workload evaluations, paired-comparison techniques have an advantage over rating scales.

Typically workload rating scales treat the research participant as a "meter" indicating the level of workload. The participant indicates a level of workload by selecting a number from the rating scale. Each number on the rating scale may be associated with a verbal description of the level of workload the number represents. Because the possible numbers on the rating scale is finite, a workload rating scale will detect workload differences only when the perceived difference in workload between two situations can be described by different rating scale numbers. With a paired-comparison technique, the differences are stated explicitly by the participants. The result is a method that has a much higher resolution than rating scales.

As an example of the benefits of increased scale resolution, the paired-comparison technique successfully found differences in the level of mental difficulty experienced by aircrew as they recovered correctly from various unusual aircraft attitudes. These differences were not indicated by the NASA Task Load Index (NASA-TLX) technique, which was also used in the evaluation (Turner, 1993). Because paired comparisons utilize the rater's internal criteria for making comparisons, the most important issue in determining the validity of the subjects' stated differences is determining the consistency of the internal criteria the subjects used in making the comparisons. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.