Academic journal article Trames

A Framework for the Measurement and Prediction of an Individual Scientist's Performance

Academic journal article Trames

A Framework for the Measurement and Prediction of an Individual Scientist's Performance

Article excerpt

1. Introduction

Quantitative measurement of scientific output has started to play an important role in the lives of scientists. However, traditional bibliometric indicators are not well adapted to the practical tasks for which they are used.

Using simple counts of publications and citations to measure productivity and research quality of an individual scientist assumes that each scientific paper has a single author. This was once a reasonable idea, but it is fundamentally misleading today (Lindsey 1980, Price 1981, Poder 2010). Perhaps everybody agrees that it is not correct to equate the contribution of a single author of an article with the contribution of any of the 1000 co-authors of a similar article. Although it is well known how to calculate unbiased measures in that case, the majority of users simply ignore the problem of multiple authorship.

The popular h-index (Hirsch 2005) is based on an amusing mathematical idea of combining publication and citation counts, which is, however, arbitrary and unsupported by any theory or data (Lehmann et al. 2006, Van Eck and Waltman 2008). There is no rational argument why publishing, for example, 10 papers that receive 10 citations each should be valued higher than publishing five papers that receive 20 citations each. This indicator ignores the problem of multiple authors as well (Schreiber 2008). A large number of 'improved' or alternative indicators have been proposed (Panaretos and Malesios 2009, Bornmann et al. 2011). However, there is no consensus on which of these indicators are really useful, how to select a correct one, or how to combine them, for practical tasks. Frequently, the experts of bibliometrics recommend to 'take into account' different contextual factors not included in the indicators themselves (Panaretos and Malesios 2009, Bornmann and Marx 2014). One may conclude that the measurement of scientific performance is necessarily subjective.

It is important to determine the goal of our measurement. When selecting candidates for an academic position or making decisions about financial support, the main objective is to predict future performance. We have to estimate a researcher's ability to produce new qualitative papers within some future period. This is fundamentally different from the prediction of one's cumulative citation score or h-index (Hirsch 2007, Acuna et al. 2012), which are determined primarily by past performance. Obviously, the cumulative indicators are not good for the practically relevant prediction. Also, it is impossible to compare researchers of different ages using these indicators. A more practical strategy is using some fixed intervals and time series format that allows analyzing the past and predicting future performance (Fiala 2014, Schreiber 2015).

In this study, I propose a framework for the measurement and prediction of an individual scientist's performance. It is based on a simple analysis of the behavioural processes that produce bibliometric data and of the requirements of usual practical tasks. It is intended to avoid some obvious problems of traditional bibliometric measures.

2. Theory

I suppose that there are two elementary processes that can be modelled more or less separately: producing publications and collecting citations. These processes can be related to different behavioural characteristics of individual researchers.

Scientific articles can be produced either individually or in larger or smaller research groups. A larger group produces a larger number of articles per unit of time, assuming that productivity of the members of different groups is the same. The straightforward measure of an individual's productivity is the number of articles published per unit of time divided by the number of people in the group. If we have reliable information on the proportions of individual contributions of the co-authors, we can use these data for a more accurate estimation of each individual's productivity. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.