Research Performance Evaluation: The Experience of an Independent Medical Research Institute
Schapper, Catherine C., Dwyer, Terence, Tregear, Geoffrey W., Aitken, MaryAnne, Clay, Moira A., Australian Health Review
Background. Evaluation of the social and economic outcomes of health research funding is an area of intense interest and debate. Typically, approaches have sought to assess the impact of research funding by medical charities or regional government bodies. Independent research institutes have a similar need for accountability in investment decisions but have different objectives and funding, thus the existing approaches are not appropriate.
Methods. An evaluation methodology using eight indicators was developed to assess research performance across three broad categories: knowledge creation; inputs to research; and commercial, clinical and public health outcomes. The evaluation approach was designed to provide a balanced assessment across laboratory, clinical and public health research.
Results and discussion. With a diverse research agenda supported by a large number of researchers, the Research Performance Evaluation process at the Murdoch Childrens Research Institute has, by necessity, been iterative and responsive to the needs of the Institute and its staff. Since its inception 5 years ago, data collection systems have been refined, the methodology has been adjusted to capture appropriate data, staffawareness and participation has increased, and issues regarding the methodology and scoring have been resolved.
Conclusions. The Research Performance Evaluation methodology described here provides a fair and transparent means of disbursing internal funding. It is also a powerful tool for evaluating the Institute's progress towards achieving its strategic goals, and is therefore a key driver for research excellence.
What is known about the topic? Increasingly, research funders are seeking to evaluate the impact and outcomes of research spending in order to inform policy decisions and guide research funding expenditure. However, in most instances, research evaluation activities are not undertaken by the organisation conducting the actual research and may not meet their practical needs.
What does this paper add? The paper outlines a research performance evaluation methodology specifically tailored to the needs of the medical research institute conducting the research being evaluated, as a way of evaluating research performance to drive strategic goals and disburse internal funds.
What are the implications for practitioners? This paper provides a clear approach to internal research evaluation using a process that meets the needs of the organisation actually conducting the research, and provides institutional data for strategic planning activities.
Received 10 June 2011, accepted 22 September 2011, published online 25 May 2012
Interest in defining the health, social and economic impacts of investment in health research has intensified in recent years.1 Much of this activity has focussed on evaluating the outcomes of research funding from the public sector. The Research Assessment Exercise (RAE) and now the Research Excellence Framework (REF) were established to assess the quality of research carried out in British universities.2 In Australia, the Federal government implemented the Excellence in Research for Australia (ERA) exercise in 2009,3 a national initiative to evaluate university research output. Other evaluation efforts include assessing the research outcomes of funding from bodies such as medical charities4 and an assessment of research and development centres funded by a regional National Health Service office in the UK.5 In these situations, the body funding the research, and thus seeking to assess its impact, is a separate entity to the organisation conducting the research. Independent research institutes, many of which are funded by philanthropic donors, have a similar need for accountability in investment decisions and also require an evaluation of research performance but the existing methods - generally developed for universities by external funding bodies - are not appropriate. …