Policymakers and funders are increasingly recognizing the central role that principals play in supporting and ensuring high-quality teaching in schools. This realization has piqued interest in efforts to improve the principalship and a desire to evaluate the effectiveness of such efforts.
Throughout our evaluation of the New Leaders program, we have grappled with a number of conceptual and practical challenges described in this report. We do not provide a neat answer or a single “best practice” that can be applied by all stakeholders in all circumstances. The evaluation of efforts that target principals presents unique challenges. The nature of those challenges and the opportunity to address them varies depending on the characteristics of the effort, such as how many schools it affects, the diversity of sites in which it is being implemented, and how long it has been in existence.
An overarching reality that policymakers and funders must come to terms with is that it will take time for efforts that target principals to show results. Whereas one might expect to see changes in student achievement from a curriculum intervention or even a targeted teacher professional development effort relatively quickly, efforts targeting principals typically influence student outcomes through the principal’s influence on teachers. In the case of a principal preparation program, these changes in student outcomes can easily take up to four years or more to appear from the time of the initiation of the program, depending on the outcome used. Even when an effort is comprehensive enough to support a rigorous outcomes-based evaluation, stakeholders will want to consider other measures that are available earlier in the effort’s life cycle.
Evaluators of efforts designed to affect the principalship face numerous challenges that make it difficult to isolate the effect of the effort. Overall, one of the most important recommendations is to use multiple measures when evaluating these efforts. Ideally, these multiple measures will include both qualitative and quantitative techniques and consist of both student outcomes and interim measures.
Policymakers also have a role to play in mitigating these challenges; they should assess the data situation and lay the groundwork for the collection and retention of data that will be needed for the purposes of evaluation at the start of an initiative. There may also be a role for state policymakers to encourage consistent and systematic reporting of core variables of interest by all districts.
The challenges that we have identified in this report can be addressed by improving the availability and quality of data, by choosing appropriate evaluation methods, and by appropriately interpreting the results of the evaluation. States, districts, and funders may find it challenging to adopt all of these recommendations in an environment of constrained resources but should bear in mind that time, money, and energy devoted to the evaluation of these types of