Ethnography in Evaluation: Uncovering Hidden Costs and Benefits in Child Mental Health

Article excerpt

This article discusses the application of ethnography to the design and implementation of an anthropologically informed cost-benefit analysis, of a program for families of children with severe emotional disorders. Ethnography proved particularly useful at revealing monetary costs and benefits for various stakeholders not included in traditional evaluations or assessments, as well as identifying costs avoided and non-quantifiable "hidden" benefits of the program to families and children, such as increased communication between family and community, improved parenting skills, and higher valuations of self-esteem of parents and children. This article contributes to the literature on evaluation anthropology in that it provides an example of how ethnography can inform the assessment and measurement of importance from the viewpoint of a program's participants, bringing their voices and concerns to the attention of program directors and policy makers.

Key words: evaluation anthropology, cost-benefit analysis, child mental health

Introduction

Anthropologists are increasingly involved in the field of evaluation, and an emerging "transdiscipline" has been identified that integrates the theory and method of ethnography with that of evaluation. One domain of evaluation is what is called "efficiency evaluation," in which the utility and effectiveness of a program is assessed relative to its costs, often in comparison to another, preexisting program. Such a study is usually represented by a cost-benefit analysis, done to investigate the effectiveness of a social intervention, or to inform policy decisions about a program's adoption, implementation, or continuation. To date, cost-benefit analyses (CBA) have typically been the domain of economists, accountants, and public administrators, not something for cultural anthropologists interested in context and meaning. We find this unfortunate, and suggest that the holistic perspective of ethnography provides a natural and valuable framework capable of uncovering hidden costs and benefits from a range of a program's participants that might otherwise go unnoticed during the course of evaluation.

In summer 2005 the authors received a competitive grant from a nearby metropolitan city to perform a cost-benefit analysis of a new model of mental health service provision for emotionally disturbed children ages 5-17 years. The program sought to implement integrated mental health services providing children with seamless, non-redundant access to care. The program, called Community Answers1, is one of many similar "systems of care" sites nationwide where this new model is being implemented. The purpose of the CBA was to assess both the benefits of the service model and how it might effectively reduce the social and economic burden to the state, in both short and long-term perspectives. It was to provide a baseline of data to be used to estimate the value of the new services, and to locate areas needed for system improvements. In all areas of the project, ethnography proved invaluable in discovering types of data not accessible through other means.

This paper details the theory and method which guided the construction and implementation of an anthropologically informed cost-benefit analysis. It reaffirms the critical relevance of ethnography within evaluation studies, in that an ethnographic approach can reveal programmatic costs and benefits that might otherwise remain hidden, among a range of users who might otherwise remain voiceless. Because CBA seems to be a new venture for anthropologists, we situate our study within evaluation anthropology in general, and attempt an extensive list of references. We hope the article may serve others interested in developing innovative approaches to evaluation that will create new space within the field for applied anthropologists.

Evaluation and Anthropology

Evaluation is ultimately a determination of the worth and utility of a particular thing, though this "thing" can be as diverse in scope as a program, product, objective, organization, policy, personnel, performance, or even a new technology (Payne 1994; Wilde and Sockey 1995). …