AI, Decision Science, and Psychological Theory in Decisions about People: A Case Study in Jury Selection
Lachman, Roy, AI Magazine
A Case Study in Jury Selection 1
Al theory and its technology is rarely consulted in attempted resolutions of social problems. Solutions often require that decision-analytic techniques be combined with expert systems. The emerging literature on combined systems is directed at domains where the prediction of human behavior is not required. A foundational shift in Al presuppositions to intelligent agents working in collaboration provides an opportunity to explore efforts to improve the performance of social institutions that depend on accurate prediction of human behavior. Professionals concerned with human outcomes make decisions that are intuitive or analytic or some combination of both. The relative efficacy of each decision type is described. Justifications and methodology are presented for combining analytic and intuitive agents in an expert system that supports professional decision making. Psychological grounds for the allocation of functions to agents are reviewed. Jury selection, the prototype domain, is described as a process typical of others that, at their core, require the prediction of human behavior. The domain is used to demonstrate the formal components, steps in construction, and challenges of developing and testing a hybrid system based on the allocation of function. The principle that the research taught us about the allocation of function is "the rational and predictive primacy of a statistical agent to an intuitive agent in construction of a production system." We learned that the reverse of this principle is appropriate for identifying and classifying human responses to questions and generally dealing with unexpected events in a courtroom and elsewhere. This principle and approach should be paradigmatic of the class of collaborative models that capitalizes on the unique strengths of AI knowledge-based systems. The methodology used in the courtroom is described along with the history of the project and implications for the development of related Al systems. Empirical data are reported that portend the possibility of impressive
predictive ability in the combined approach relative to other current approaches. Problems encountered and those remaining are discussed, including the limits of empirical research and standards of validation. The system presented demonstrates the challenges and opportunities inherent in developing and using Al-collaborative technology to solve social problems.
Significant progress has been reported by AI researchers in combining decision-analytic techniques with knowledge-based expert systems. Historically, the most common AI methods for inference and decision making involved inquiry that was conducted in isolation from associated fields of study. The limitations of this approach have been examined, with the result that the methodologies of decision making under uncertainty are under continuing evaluation and are being incorporated into knowledge-based systems (Henrion, Breese, and Horvitz 1991). Applications that combine decision-theoretic approaches with expert systems include medical diagnosis, product development, system troubleshooting, and various tasks involving candidate evaluation (Durkin 1993; Mitri 1991). Overall, AI and decision-theoretic methods in combination produce superior results than either alone. Although the applications address some socially and financially significant domains that require the prediction of human behavior, there are still many pressing problems for which AI theory, concepts, and methodology provide potentially powerful solutions, yet AI methods are not currently being tested or extended for application to these problem areas.
This situation is regrettable, given the increasing maturity and coherence of the foundational paradigm of Al science that is reflected in changes in the field's presuppositions. Previously, AI systems were generally treated as single-agent entities working in isolation from the world and other agents; recently, the discipline has begun to adopt the more productive assumption that Al is one of a group of intelligent agents working in collaboration. …