Learning: Enhancing Interaction
Ellen J. Bass and Amy R. Pritchett
Judgment is a critical component of many human activities, and technology to aid judgment is proliferating. Effectively coupling human and such automated judges involves more than merely providing the output of the automated judge to the human. Because automated judges are not perfect, human judges may face difficulties in understanding how to combine their own judgments with those of the automated judges.
Improved theoretical constructs are necessary for explaining (and quantitative methodologies for analyzing) human–automated judge interaction. Measures of interaction have been inferred from fitting observed behavior to models of judgment. For example, interaction between automated and human judges has been modeled using signal detection theory (SDT) (Robinson & Sorkin, 1985; Sorkin & Woods, 1985). This model implicitly views automated and human judgments as serial processes, each with their own sensitivity and response bias (Wickens & Flach, 1988). However, it can neither measure continuous valued judgments nor provide detailed insight into the causes of poor judgment. Fuzzy SDT (Parasuraman, Masalonis, & Hancock, 2000) can help overcome the former limitation but is still susceptible to the latter.
Pritchett and Bisantz (this volume) used judgment analysis (Cooksey, 1996) to examine human and automated judges, demonstrating the ability of the n-system lens model to assess conflict between a human judge and what an automated judge would have said and to decompose the contributions of their knowledge and cognitive control. However, this study examined only human and automated judgments made individually without any interaction.
Seong and colleagues (this volume) combined an n-system lens model approach with subjective estimates of trust in the automated judge to model human–automated judge interaction. They manipulated the automated judge's quality and the display of its feedback. Results showed that performance and assessments of trust were impacted by the decision aid's quality, and that participants used the feedback effectively to compensate for a poorperforming aid. Although they explicitly measured trust and manipulated feedback about the automated judge, they did not measure the human judge's ability to predict the aid.
A comprehensive method should be capable of many measures of human interaction with automated judgments, including: