Signal Detection Theory and Roc Analysis in Psychology and Diagnostics: Collected Papers

A companion chapter (Chapter 3) derives the form of the relative (or receiver) operating characteristic (ROC) that is algebraically implied by each of a dozen or so commonly used indices of discrimination accuracy, and identifies the models of the discrimination process that are implied by the main categories of those forms. In this chapter I present a broad sample of empirical ROCs for comparison with the theoretical forms. They are drawn from discrimination tasks used in the psychology of perception, learning, memory, and cognition, and from several practical fields in which a discrimination, or diagnosis, is made in the interest of prediction, selection, or corrective action. The fields included are medical imaging, information retrieval, weather forecasting, aptitude testing, and polygraph lie detection.

The ROC, in a sentence, is a graph showing the conditional probability of choosing Alternative A when that alternative occurs (here denoted by h, for "hit") plotted against the conditional probability of choosing A when Alternative B occurs (here denoted by f, for "false alarm"). Both h and f increase as the tendency to choose Alternative A increases, or as the criterion for choosing A becomes more lenient.

The form of an ROC is best visualized on a "binormal" graph--a graph in which the usual probability coordinates are rescaled so that their corresponding normal-deviate values are linearly spaced, as in Figure 1. On such a graph, empirical ROCs are consistently fitted well by a straight line that varies in slope; the slopes are generally between 0.5 and 1.5, as indicated in Figure 1 a by dashed lines. (Other details of the figure are discussed next.)

The indices chapter (Chapter 3) shows that the form of predicted ROCs serves to sort common accuracy indices and their implied models into three categories.

-31-