Assessment of Intelligent
University of Pittsburgh
Over the past decade, there has been considerable research and development in applications of artificial intelligence to education and training (e.g., studies in Larkin & Chabay, 1992; Polson & Richardson, 1988; Psotka, Massey, & Mutter, 1988). In several cases, training systems have been produced that are receiving practical use (e.g., Anderson, 1990; Corbett & Anderson, 1992; Govindaraj, 1988). More commonly, so far, managers are starting to face decisions about whether a prototype research system has potential utility. In this chapter, I view the assessment of intelligent training systems from a long-term perspective, discussing the different kinds of decisions that require assessment of intelligent training technology and a number of specific assessment issues, considered in light of current theory and experience. In particular, I draw on experiences with the Sherlock coached practice environment for electronics troubleshooting ( Lajoie & Lesgold, 1990; Lesgold, Lajoie, Bunzo, & Eggan, 1992; Lesgold, Lajoie, Logan, & Eggan, 1990).
Technology assessment in the world of intelligent training systems must consider not only the effectiveness of a training system but also the likelihood that it can be assimilated by the organizations that could use it. This can be seen either superficially as a marketing problem or more deeply as a problem in changing schooling or training. In either case, though, it is not enough for a product to be effective; it also must either fit the existing organizational structure and available technology or be so attractive as to bring about adaptive changes that make it usable.