Academic journal article Applied Health Economics and Health Policy

Ordering Errors, Objections and Invariance in Utility Survey Responses

Academic journal article Applied Health Economics and Health Policy

Ordering Errors, Objections and Invariance in Utility Survey Responses

Article excerpt

Utilities are the underpinning of cost-utility analysis and other decision-analytic methods that depend upon QALYs as an outcome measure. As subjective measures of well-being, utilities are, by definition, reported by individuals, usually by way of in-person, telephone or written surveys, and increasingly via the internet. Many of the challenges inherent in self-reported data are also present for utility elicitation, in addition to the complexity of the information sought. Research documenting these challenges and methods for addressing them is limited despite the widespread use of utilities in medical and policy decision making.

The methodological limitations of self-reported information are well known.[1,2] Utilities are especially challenging to measure because they are not simply information that is reported by the individual (such as height or weight, or choice of a candidate), but rather are the verbalization of values (such as a belief in the importance of independence, or the value of emotional vs physical well-being). As such, utilities require multiple levels of cognition, introspection and comparison to articulate.[3,4] While the goal of health utility surveys is to measure the individual's 'true' preferences for a health state, this may not result from the survey process because the person either does not know or is unable to articulate his/her preferences, or because the reported value is unduly influenced by some aspect of the measurement process ('construct irrelevant variation'[5]). In fact, a preference as elicited during a utility survey is, at best, an estimation of a person's feeling or value associated with a health state or condition. At worst, it is an artefact of a survey task and is unrelated to feelings or values because of tangential issues raised by the task, or misunderstanding, inattention or distraction during the articulation process. The inclusion of inaccurate utility data in analyses can result in biased estimations of population preferences for health states, while exclusion of such data can bias the sample used for analyses and threaten the external validity of results.[6]

The objective of this article is to describe the literature on the types and prevalence of errors and objections in response data in directly elicited utility surveys, including those using the standard gamble (SG), time trade-off (TTO) and visual analogue scale (VAS) techniques, and to provide insight into the reasons underlying such responses. Our analysis also applies to 'generic' utility instruments, meaning those that rely on health state classification systems to derive utility weights[7] (such as the EQ-5D and the Health Utilities Index), to the extent that they are based on directly elicited utilities. Our goal is to inform decision analysis through the identification of limitations in utility data and mechanisms to improve data validity.

1. Terminology of Error, Objection and Invariant Utility Response Types

1.1 Ordering Errors

Biased or mistaken responses in direct utility surveys can be described in two main categories: ordering errors and objections (figure 1). Ordering errors encompass responses in which the values for multiple health states in comparison with one another violate some standard of order. By definition, ordering errors can occur only when more than one state is being evaluated. Although sometimes difficult to establish, the normative order of two or more states may be determined by the investigator (inherent to the health state description) or determined by the respondent.

Fig. 1 Terminology of ordering errors and objections/invariance in responses to direct utility surveys. [Figure omitted.]

1.1.1 Investigator-Determined Order

This type of ordering exists when the health states themselves include an ordinal level of functioning or disease, such as partial blindness compared with total blindness, or early-stage cancer compared with late-stage disease. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.