This is the second of a two-part essay. The first part, "Getting Started with Evaluation Reports: Answering the Questions, " appeared in this column in the previous issue of ETC.
ONCE A WRITER of an evaluation report has considered all the questions his readers might ask, as described in the previous installment of this column, he is ready to ground himself in the challenging task of framing the document into the narrowest yet most useful possible perspective; thus, he establishes the evaluative scope and criteria, collects and analyzes data, and organizes the report. For a hypothetical example to illustrate the elements featured in this article, I use a public school district's pilot project of a voucher program, in which parents may choose whichever school in the district they wish their child to attend.
Establishing the Scope and Criteria
Determining the scope of the report helps the writer go far in creating a structure. Having no boundaries for the report, on the other hand, will lead to labored drafting and unnecessary revisions. To structure the evaluation report efficiently, answer these three questions about the project.
1. What central issue is the project or program addressing? The answer to this question should appear in one clear sentence. For example, the school superintendent assigning the evaluation report might answer this question with the statement, "The project seeks to increase student standardized test scores by 20 percent, decrease student retention (repeating the same academic year) by 30 percent, and increase parental participation in school activities by 40 percent by the end of two school years."
Writers should not underestimate the power of the stating the issue in one clear statement. Doing so - and then keeping the statement at the fore of their mind - keeps them focused on all aspects of the project and helps them organize the report.
2. Who are all the stakeholders? The stakeholders are those who would be interested in reading the school's evaluation report because of a social, moral, intellectual, professional, or financial interest in the project. Among the many parties with a stake in the voucher project would be other school administrators, teachers, students, parents, legislators, and political organizations such as think tanks and teachers' unions.
The key word of the question to remember is all. Writers should remember that they must communicate to their least informed reader. While the superintendent may reasonably assume that a member of the general public would not be a reader of the report, she should ensure that the evaluation include terminology familiar to the media and potential funding sources not directly involved in the field of education, because school choice programs have traditionally captured the attention of news organizations, which may interpret the report for their audience.
3. What is the evaluation perspective? What institutional aspect is under examination? At least five evaluative viewpoints come to mind: system, process, performance, efficiency, and causality. Of course, a writer may be evaluating from more than one of these perspectives in a given report. Definitions and examples of each appear below:
* the system designed for achieving the objective (e.g., administrative structure, governing bylaws)
* the process by which the objective was achieved (e.g., the methodology by which students were taught or the frequency of classroom instruction)
* the performance, or effectiveness, in executing the process (e.g., classroom facilitation or student evaluation)
* the efficiency, or value of achieving the objective relative to its cost (e.g., time frame involved, cost-benefit analysis)
* the causality, or reason the objective was or was not achieved (e.g., the cause of an increase in student performance on standardized tests or a decrease in the student retention rate). …