Policy Evaluation in a Time of Austerity: Introduction
Bryson, Alex, Dorsett, Richard, Portes, Jonathan, National Institute Economic Review
As Phil Davies (pp. R41-52) notes, the last government made a series of high-profile commitments to the idea of evidence-based policy. Central to this philosophy was a culture of evaluation whereby potential and actual policies were scrutinised and appraised in order to build and refine a knowledge base of 'what works'. While this was far from consistently applied, and the results of evaluations were not always comfortable reading for the government (for example, the very mixed results from the National Evaluation of Sure Start), on the whole there was a presumption that major social policy initiatives should be subject to independent evaluation.
The approach of the new government appears somewhat different. Although the Treasury has maintained its public commitment to evidence-based policymaking, research and evaluation budgets have been severely squeezed. Faced with a choice between service delivery and policy evaluation, it may be tempting to regard evaluation as a luxury that government can ill afford, at least in the short term.
We reflect briefly on why such a response may be shortsighted and offer some thoughts on how the approach to policy evaluation may adapt to these more straitened times. This is a theme that is taken up in a number of places by the papers in this issue.
The key reason why evaluation continues to be relevant in times of financial austerity is that it provides the evidence needed for informed decision-making. This is especially relevant when resources are scarce, a point made forcefully by Rachel Glennerster (pp. R4-14) in her review of policy evaluation in developing countries. Evaluation evidence, including the assessment of the costs and benefits of policy, offers the most direct insight to inform such decisions, as Haroon Chowdry (pp. R53-64) makes clear. Policies found through rigorous analysis to be both effective and cost-effective are strong candidates for investment. Furthermore, broader explorations--including qualitative analyses such as the one discussed by Heather Rolfe (pp. R65-76)--can provide a deeper understanding of the way policy is implemented and how and why a policy seems (not) to work, sometimes suggesting ways of adapting policies to better achieve their aims.
Evaluation can be costly but it is often a means of saving money. The recently-completed Employment Retention and Advancement (ERA) demonstration (Hendra et al., 2011) provides an edifying example of how a thorough evaluation can help guide policymakers away from wasteful spending. Davies and Chowdry outline the features of the intervention. The evaluation revealed that impacts for the first two years were large and significant but subsequently disappeared. Overall, the intervention failed the cost-benefit test for lone parents. The evidence cautions strongly against national roll-out of the policy (Chowdry). Evaluations of labour market programmes typically confine themselves to a shorter period over which to observe impacts. Had the ERA evaluation only observed outcomes for two years, say, the conclusion would have been very different, with significant financial implications.
Properly conducted, evaluations offer an objective assessment of the appropriateness of policy, thus enabling governments to deploy available funds in the most cost-effective manner. When the Work and Pensions Committee asked the employment minister, the Rt Hon Chris Grayling MP, why the Department for Work and Pensions (DWP) had decided not to commission an independent evaluation of the Future Jobs Fund (FJF) he cautioned of the "... danger in financially straitened times that we spend vast amounts of money evaluating things that we are no longer going to do." (1) The decision to discontinue the FJF was not based on evaluation evidence and, consequently, it is impossible to establish its cost-effectiveness. One argument for ending the FJF was that, from June 2011, the Work Programme has provided a unified mechanism for helping individuals move into sustained employment. …