were divided randomly into two groups: Active and Passive. The Active group had to solve programming problems using the model-tracing based environment of DISCOVER, with immediate feedback supported, while the Passive group had to solve the same problems using the passive environment of DISCOVER, with feedback being delayed until the user explicitly requests it. In analyzing the results, only two measures were used: total logical errors made and total times spent while solving problems. So far, only five users in each group have completed the evaluation.
Results that have been compiled so far showed that the passive environment did not prove to be as effective as the active one. Overall, the Passive group made about 46% more errors to complete their programming problems than the Active group. Regarding the time spent on problem solving, the Passive group took about 63% longer to complete their problems than the Active group. These results may suggest that the effectiveness of the Active approach to intelligent diagnosis is achieved by maintaining strong control over the progress of the user during problem solving. Broadly speaking, these partial results seem to be consistent with an earlier evaluation conducted to test the Student Controlled version of the Lisp Tutor ( Anderson 1990).
Anderson, J. ( 1990). "Cognitive Modeling and Intelligent Tutoring". Artificial Intelligence and Learning Environments, Clancy and Soloway (Eds.), MIT/Elsevier.
Bonar, G. ( 1992). "Intelligent Tutoring with Intermediate Representations". Proceedings of the Second Conference on Intelligent Tutoring Systems (ITS-92), Canada.
Johnson, W. ( 1990). "Understanding and Debugging Novice Programs". Artificial Intelligence and Learning Environments, Clancey and Soloway (Eds.), MIT/Elsevier.
Ramadhan, H. ( 1992). "Intelligent vs. Unintelligent Programming Systems for Novices". Proceedings of the IEEE 15th International Conference on Computer Applications and Systems, USA.
Ramadhan, H. ( 1998). "Model tracing based approach to intelligent program diagnosis". SQU Journal of Science & Technology, Vol. 2.
Reiser, B. ( 1992). "Making Process Visible: Scaffolding Learning with Reasoning-Congruent Representations". Proceedings of the 2nd Conference on Intelligent Tutoring Systems (ITS '92), Montreal.