Using Theory to Improve Program and Policy Evaluations

Using Theory to Improve Program and Policy Evaluations

Using Theory to Improve Program and Policy Evaluations

Using Theory to Improve Program and Policy Evaluations

Synopsis

This collection of essays underscores the importance of theory in program evaluation and analyzes the nature and implications of theory-driven evaluations. Scholars from different disciplines cover a wide range of issues, such as functions of program theory, the interface between theory and methods, strategies for formulating theory, cost and resource constraints, different types of theory-driven evaluations, and the future implications of such evaluations. This broad-based analysis is designed for inter-disciplinary audiences in policy studies and all the social sciences.

Excerpt

In the 1970s, the state of the art of program evaluation was centered on experimental research designs. This emphasis was due in large part to the influence of Donald Campbell and his colleagues who conducted the excellent program on evaluation research at Northwestern University. According to the Northwestern "orthodoxy," the overriding objective of program evaluation was to determine the effects of the intervention; it was recognized that experiments often yielded treatment effect estimates of high "internal validity." Because of this, experimental designs were labeled the "Cadillac" of research designs for program evaluations (Hatry,Winnie, &Fish, 1973).

To view experiments as "Cadillacs" is a good analogy. If one has to get from point a to point B and has a well-maintained superhighway connecting those points, a Cadillac is an appropriate means of travel. in addition, Cadillacs come equipped with a "cruise control" that allows the driver to sit back and let the car drive itself. the driver is not required to use this feature, but it is very seductive. Because experiments, whether or not they are grounded in a theoretical framework, generate estimates of treatment effects that generally have high internal validity, many evaluators were (and still are) content to sit back and let research design, rather than theoretical propositions, drive their studies (Chen &Rossi, 1983). the "pride of ownership" derived from employing the Cadillac of research designs may reinforce this tendency.

The "cruise-control" feature of basic experimental evaluations led to dissatisfaction, however. the commissioners and consumers of evaluations, and evaluators themselves, began to recognize that often little is learned in such evaluations about either the intervention or the disorder or social problem to which it is directed. the basis for this dissatisfaction was captured succinctly by J. G.

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.