What Is a Successful Program?
In any study of the operation of public programs, an important but difficult question is whether the programs under study are ultimately effective. The question is difficult because results in many public programs are likely to be hard to measure, discernible only over the long term, and produced by the interaction of the program itself with a great many other influences. Therefore, convincing answers about the effect of the program itself are likely to require experimental or quasi-experimental studies carried out over long periods of time and across many outcome dimensions, studies that are rarely conducted for any program and certainly beyond the scope of this project.
However, even if it is hard to measure, program effectiveness matters a great deal, especially in a study that draws lessons for public policy from the experience of particular programs. What if these programs are really not good examples? What if they do not serve families effectively or change children's lives in the long run? As noted in Chapter 1, this study does not go as far as do some studies based on model programs -- it does not assume that everything about the study sites works well -- but it does assume that the sites offer operational practices that are effective enough in their circumstances to be worth studying. Even this more limited assumption clearly requires both a definition and evidence of operational effectiveness. Moreover, if the study is to be truly useful to policyrnakers, it also requires at least some evidence of success in a broader outcome sense as well: evidence that the programs are reasonably likely to improve the lives of poor children and families. Unless the program approaches and practices are at least consistent with the research evidence about changing life chances, there is no reason for policymakers to care about their operational effectiveness.