Adjusting Performance Measures to Ensure Equitable Plan Comparisons

Article excerpt


In 1995, the Agency for Healthcare Research and Quality (then called the Agency for Health Care Policy and Research) initiated a cooperative agreement with RAND, Harvard, and Research Triangle Institute to conduct the CAHPS[R] study. The goals of the CAHPS[R] project included developing a standardized survey that could be used to assess the experience of consumers in different types of insurance arrangements and health care plans (Weinberger, 1999). In 1997, HCFA funded the CAHPS[R] consortium to develop a version of CAHPS[R] suitable for assessment of the experiences of Medicare beneficiaries in managed care. HCFA now uses that survey to assess Medicare managed care plans annually.

Several methodological problems complicate the measurement and reporting of health care data, particularly when reports draw comparisons among health plans, as is the case in the MMC-CAHPS[R] project. Among the challenges is the need to adjust appropriately for patient characteristics such as patients' health and sociodemographic characteristics, which are not under the control of plans and which may affect CAHPS[R] scores.

There are at least two reasons why it might be desirable to adjust plan CAHPS[R] scores. First, there are certain processes that one would expect to vary according to the characteristics of patients. For example, one CAHPS[R] question is "... how much of a problem did you have finding or understanding the information ... from your health plan?" Although it is desirable to communicate clearly with all patients, it probably is harder to do so with patients who have less education than with other patients. Second, certain personal patient characteristics might influence the response to questions, even if the process of care is the same for all patients.

To develop a case-mix adjustment model for the MMC-CAHPS[R] data, we first reviewed published studies. Next, we analyzed five data sets from surveys of health maintenance organization (HMO) populations in different parts of the United States and identified potentially important case-mix variables (Cioffi et al., 1998). Finally, we analyzed MMC-CAHPS[R] data to evaluate alternative models. We analyzed data from the first MMC-CAHPS[R] survey, based on CAHPS[R] 1.0, and from the second and third surveys, based on CAHPS[R] 2.0 (Cleary et al., 2000).


Given that there are few published studies of factors affecting health plan ratings, we reviewed studies of hospital care, ambulatory medical services, and health plans. Patient characteristics that have been identified as correlates of patient reports about their health care include (1) patient sociodemographic characteristics, (2) overall perceived health status, (3) functional status, (4) diagnoses or conditions, (5) length of relationship with provider or health plan and prior use of services, (6) whether the survey was completed by a proxy, and (7) institutional status (Cleary and McNeil, 1988; Aharony and Strasser, 1993; Weiss, 1988; Hall et al., 1990; Cleary et al., 1992; Zapka et al., 1995; Kane, Maciejewski, and Finch, 1997; Kippen, Strasser, and Joshi, 1997; Lee and Kasper, 1998).

Better self-reported health is consistently associated with higher ratings of health care services by consumers and patients (Cleary and McNeil, 1988; Aharony and Strasser, 1993; Marshall, Hays, and Mazel, 1996; Hall, Milburn, and Epstein, 1993; Roberts, Pascoe, and Attkisson, 1983; Hall et al., 1998). Current general health status tends to be the strongest predictor of patient or consumer satisfaction with health care services (Hall et al., 1990; Cleary et al., 1992; Zapka et al., 1995; Kane, Maciejewski, and Finch, 1997; Kippen, Strasser, and Joshi, 1997; Lee and Kasper, 1998). Perceived improvement in health also has a strong positive association with health care ratings (Kane, Maciejewski, and Finch, 1997; Kippen, Strasser, and Joshi, 1997).

The few studies that have investigated the relationship between satisfaction ratings and the presence of specific medical conditions have yielded inconsistent results (Hall et al. …