differ. This will mean that sometimes experts and novices have a different vocabulary, that sometimes novices may have an impoverished understanding of the terms and the assumptions behind definitions, and that sometimes novices' and experts' conceptual structures may be different. It is important that the users of a system understand the terminology used by the system, that it is "natural" to them, and that they are able correctly to interpret input and output. It is therefore important to investigate the commonalities and differences in expert and novice terminology early on in a project. It is important to note that novices may appear to understand terms and may construct apparently correct statements, but subsequent decisions or questions may betray a lack of understanding.
Second, experts and novices are likely to have different representations of problems, assumptions, and hypotheses. They are therefore likely to use different strategies. Furthermore, the strategies used by experts are likely to be essentially different on "hard" problems versus "easy" ones. In other words, a knowledge elicitation exercise on a "hard" problem will elicit different knowledge from the same technique used on an "easy" problem. Experts are likely to be more versatile and to consider a more diverse and rich set of alternatives; they may interpret evidence in different ways and keep an open mind for longer. This is very important given the almost rigid nature of many expert system interactions. The system should not encourage novices to have confidence in bad solutions, but it should increase confidence in good ones. It may have to suggest more alternatives or issue cautionary advice without causing total confusion.
In any domain the following questions are important: Do experts and novices use terminology differently? If so, how? Can a common terminology be established? Do experts and novices use different strategies? How do experts and novices differ in the number, level, and type of hypotheses that they consider during problem solving? How do experts and novices differ in their confidence in the different hypotheses and in their perceptions of the necessary and sufficient evidences which they consider? Can novices understand a system that reasons like experts? What information can be elicited from the novices, when, and how? Can an analysis of experts and novices tell us whether an expert system is feasible for a domain? How do novices respond to different forms of interaction in terms of the numbers of alternatives suggested by the system, the levels of confidence displayed by the system, and a userdriven or system-driven interaction?
Our research is addressing some of these questions, with particular regard to the numbers of hypotheses considered by experts and novices, and their confidence in those hypotheses. After we have an understanding of these issues it should be possible to devise knowledge elicitation methods that incorporate novices and that result in models of the knowledge they can use to effect.
Adelson B. ( 1981). "Problem solving and the development of abstract categories in programming languages". Memory and Cognition, 9,422-433.
Adelson B. ( 1984). "When novices surpass experts: The difficulty of a task may increase with expertise". Journal of Experimental Psychology: Learning, Memory, and Cognition, 10,483-495.
Barfield W. ( 1986). "Expert-novice differences for software: implications for problem solving and knowledge acquisition". Behaviour and Information Technology, 5,15-29.
Breuker B., & Weilenga B. ( 1987). "Use of models in the interpretation of verbal data". In A. Kidd (Ed.), Knowledge acquisition for expert systems (pp. 17-44). New York: Plenum.
Chase W. G., & Simon H. A. ( 1973). "Perception in chess". Cognitive Psychology, 4,55-81.
Clancey W. J. ( 1983). "The epistemology of a rule-based expert system: A framework for explanation". Artificial Intelligence, 20,215-251.
de A. D. Groot ( 1965). Though and choice in chess. The Hague: Mouton.
de A. D. Groot ( 1966). "Perception and memory versus thought: Some old ideas and recent findings". In B. Kleinmuntz (Ed.), Problem solving: Research, method, and theory (pp. 19-50). New York: Wiley.
Diaper D. ( 1989). "Designing expert systems: From Dan to Beersheba". In D. Diaper (Ed.), Knowledge Elicitation: Principles, Techniques and Applications