Performance Assessment in Community Mental Health Care and At-Risk Populations

Article excerpt

INTRODUCTION

While the health care system overall has retreated from managed care, such competition-based strategies continue to be increasingly prevalent in public mental health care (McIntyre, Rogers, and Heier, 2001). The organizational and financial arrangements associated with managed care are designed to increase efficiency and reduce waste in health care delivery. Concerns have arisen that these same incentives may lead providers to under-serve clients, particularly individuals with severe or complicated conditions (Ellis, 1998; Ware et al., 1996). To guard against such potential negative outcomes, managed care is typically supplemented with monitoring of provider performance. Yet, "[d]espite recent research on methods of risk adjustment ..., the application of this research to Medicaid populations has lagged" (Ireys, Thornton, and McKay, 2002). For instance, standard methods of performance assessment focus on average outcomes, and may not detect suboptimal quality of care provided to select groups of at-risk clients.

Our analysis is based on data from the Indiana Division of Mental Health and Addictions (IDMHA). IDMHA is the public agency that serves as payer of last resort for persons with persistent and severe mental illness in Indiana. Care is delivered through 1 of 30 not-for-profit CMHCs, which act as gatekeepers to the 6 State hospitals. In 1996, the IDMHA adopted the Hoosier Assurance Plan that reformed the delivery system along managed care principles (Family and Social Services Administration, 1997). Subsequently, IDMHA produced provider report cards that describe various aspects of the centers that reflect the quality of care provided, including differences in assessed mental health outcomes experienced by clients at these centers (Family and Social Services Administration, 2000). While the IDMHA analysis controls for baseline functioning, it ignores variance that may be due to non-clinical client factors. In addition, the IDMHA analysis produces only limited subgroup analysis, in part because it uses a stratified approach that severely limits the extent to which different subgroups can be compared. As a result, the report cards cannot identify the vulnerability of some at-risk client groups, at-risk clients cannot use the information to identify optimal choices for people most like themselves, and treatments that work best on average may be applied to some clients for whom other treatment approaches may be more appropriate.

In previous analysis of these data (Deb, Holmes, and Deliberty, 2004), we showed the importance of adjusting performance measures for non-clinical client characteristics (e.g., sociodemographic variables and income), and different rates of client attrition across CMHCs. In this article, we extend this analysis to examine whether performance differentials observed in aggregate apply to specific, vulnerable subpopulations of clients, including clients with dual diagnoses for substance abuse, comorbid disabilities, and mental illnesses that cause particularly severe functional impairment.

Methods

Typically, estimates of provider performance have been generated in a fixed effects framework. We use instead a mixed random effects model to evaluate provider performance. The model includes both fixed coefficients (which permit control of client risk factors on outcomes) and random coefficients associated with providerspecific variation. We estimate the mixed random effects model in SAS[R] with the PROC MIXED procedure (SAS Institute Inc., 1999). In addition, we adjust provider performance for different rates of client attrition using a non-linear selection equation. The formulation of the non-linear selection equation with fixed and random coefficients is described in Deb, Holmes, and Deliberty (2004) and estimated using the NLMIXED procedure in SAS[R] (SAS Institute Inc., 1999).

As a multilevel modeling technique, the mixed random effects model offers a number of advantages over standard fixed effects specifications and is particularly attractive for the objectives of this analysis. …