Academic journal article Psychological Test and Assessment Modeling

The Hybrid Rating Method: Assessment of a Novel Way to Measure Attitudes

Academic journal article Psychological Test and Assessment Modeling

The Hybrid Rating Method: Assessment of a Novel Way to Measure Attitudes

Article excerpt

Different rating methods that attempt to measure attitudes have become a hallmark of social sciences and while they are widely used, they are still heavily discussed and iterated upon. This paper proposes and assesses a new method that was developed in response to problems with existing methods. Based on the works of Thurstone (1994) and especially Likert (1932), rating scales have been heavily researched and developed. In the past, the five-point method, already found to be the most useful by Likert in 1932 and therefore called the 'Likert-type scale', was shown to be the most prevalent (Rohrmann, 1987).

Naturally, rating methods exhibit shortcomings, some of which will be elaborated upon in the following - a general overview is provided by Podsakoff, MacKenzie, Lee, and Podsakoff (2003). On a five-point scale the granularity with which an attitude can be expressed is relatively low (Cohen, Manion, & Morrison, 2000; Döring, 2006). This can cause floor and ceiling effects (see Döring, 2006; Parducci, 1965) to occur, causing attitudes that are sufficiently different to be rated equal because the available points cannot accommodate the difference. This issue might potentially be prevented by a method in which respondents do not have to fit their attitudes to a scale but rather fit the scale to their attitudes.

A related issue, the ambivalence-indifference problem, describing the difficulty interpreting ratings labelled 'neutral', causes ratings to be unclear (Kaplan, 1972). It is not possible to discern whether the respondent felt indifferent or ambivalent about the stimulus, the respondent might even have rejected the question, answered according to social acceptance, or lacked knowledge about the stimulus (Garland, 1991; Jonkisz, Moosbrugger, & Brandt, 2012; Podsakoff, et al., 2003). One might choose to remove the middlepoint to try and remedy this problem, with the result that truly neutral attitudes can no longer be expressed and data distortions occur (Garland, 1991; Jonkisz, et al., 2012). The core of this problem seems to be that respondents cannot express all aspects of their attitudes and consequently deliver inconclusive data; thus a more granular and personalised method of rating seems beneficial.

Additionally, the acquiescence bias causes ratings to 'pile up' on the positive side, increasing ambiguity (Winkler, Kanouse, & Ware, 1982). One attempt to counteract this is to create items with reversed polarity, which has shown the side effect of introducing an additional factor loading onto the items (Jonkisz, Moosbrugger, & Brandt, 2012). A more personalised rating method that allows for more differentiation might help to reduce yea-saying or nay-saying tendencies as respondents are possibly more invested and interested in the process.

Considering floor-ceiling effects, and the ambivalence-indifference problem, the alarming revelation is that out of the five points on a five-point scale, only two points yield moderately concise information, whereas the remaining three produce inconclusive data. Increasing the number of points however, may cause diminishing returns with regards to psychometric quality - Matell and Jacoby (1971) saw no change in reliability while Ciccetti, Shoinralter, and Tyrer (1985) along with Preston and Coleman (2000) observed a limited increase - and runs the risk of overburdening the respondent with too many choices (Bogner, & Menold, 2015; Döring, 2006). Preston and Coleman (2000) found that respondents preferred ten-point ratings while Hirsch (2006) found that five-point ratings were favoured. Döring (2006) and to some extent Matell and Jacoby (1971) concluded that the optimal number of points varies individually.

In the past, alternative rating methods, such as the visual-analogue scale (see Crichton, 2001), the Q-technique (Stephenson, 1953), and the Semantic Differential (Osgood, Suci, & Tannenbaum, 1957) have been devised to address some of these shortcomings. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.