Predictive Validities of Aptitude Tests
Signal detection theory and its relative operating characteristic (ROC) are firmly established in psychophysics. Interested readers can readily find systematic accounts of the theory and methodology (see, e.g., Green & Swets, 1988). For most psychologists, in all probability, there is no other known area of application. A recent review, however, has extended application of the theory and methodology to measuring the accuracy of diagnostic and predictive systems in widely different areas of research ( Swets, 1988, Chapter 4 in this volume).
Applications require that the predictor variable be continuously or quasi continuously distributed and that the criterion be a dichotomy or convertible to a dichotomy. The ratio of signal to noise changes monotonically with the level of the continuous predictor, and a prediction or decision is made concerning the presence or absence of the signal at each of multiple levels of the predictor. The accuracy of the predictions or decisions at each level of the predictor determines the ROC curve. The Az statistic is defined by the area under the curve and describes the degree of accuracy in detecting the signal. The zero point of the Az scale is .50, which is the chance level of the yes/no decision, and the maximum value is 1.00.
Chapter 4 applied the ROC and the index Az to weather forecasting and reported an Az of .89 for a forecast of extreme cold and .71 for intervals of temperature. In an information-retrieval application, the accuracy with which retrieval systems could select books and articles relevant to a selected topic and reject the irrele-