Evaluation and Implementation: How Long an Arm Is Needed in the "Arm's-Length" Relationship? 1
John H. Reed, Nicholas Halland Robin Calhoun
Editors' Note. This chapter makes the case for the professionalization of evaluation in relation to the challenges posed by the decisions which are dependent upon it -- specifically, the expenditure and recovery through utility rates of potentially billions of dollars. The proposed solution is organizational. The central issue of accountability to various interest groups is often addressed by isolating evaluation from implementation. The authors suggest that, contrary to this traditional view in evaluation research, energy programs succeed best when lines of communication are open and when interaction occurs among program designers, implementers, and evaluators. This organizational strategy strengthens the ability of major groups within the utility organization -- program designers, implementers, and evaluators -- to retain control over which evaluation questions they will be able to answer. The chapter specifies the questions that can and cannot be answered in the areas of process evaluation, market evaluation, and impact evaluation if certain types of data are not available. The entire discussion speaks to the professionalization of this evaluation subfield and the importance of professional integrity on the part of the evaluator.
For many years, evaluators and program implementers have stressed the need for program implementation and program evaluation to be independent of one another. The rationale for this is that the integrity