A Framework for the Evaluation of CASE Tool Learnability in Educational Environments
Senapathi, Mali, Journal of Information Technology Education
The use of Computer Aided Software Engineering (CASE) tools in the industry has shown improvements in productivity and system delivery rates (Banker & Kauffman, 1991; Finlay & Mitchell, 1994). The benefits of using CASE tools in education such as its role as a pedagogical instrument for teaching a systems development methodology (Jankowski, 1995; Linder, Steele, & Dorling, 1994) and its role in enabling students to fit into the contemporary work place much easier and faster (Cifuentes & Lockwood, 1996; Rajkumar, 1992), are well recognized by the research
community. Yet, there is insufficient understanding of the various issues that students confront while learning a CASE tool (Fowler, Armarego, & Allen, 2001). According to Boloix and Robillard (1998), "CASE tool learnability is a subject that has not received much attention from the research community".
Despite the potential benefits of CASE tools recognized by both the industry and the academia, the rate of adoption by the industry is slow (Holt, 1997). Research cites a number of reasons for this low rate of adoption, which include the difficulty in learning to use the tool (Fowler et al., 2001). The learning curve problem associated with CASE tools is also widely recognized by the academic community, who argue that most commercial CASE tools suffer from the problems of a long learning curve and do not cater well to the student learning requirements (Cowling, 1998; Jankowski, 1995).
Other reasons for the low rate of adoption include the shortcomings in the usability of functionality of CASE tools available today (Phillips, Mehandjiska, Griffin, Choi, & Page, 1998). In order to ensure a general acceptance of any system, it should provide a consistent, robust and usable interface and this is specifically true in the case of CASE tools which usually provide a highly interactive and graphics intensive environment (Phillips et al., 1998). However, not enough research exists that has paid attention to the evaluation of the user interface in CASE tools (Phillips et al., 1998).
While the frameworks that exist for the evaluation of educational software are suitable and relevant for the evaluation of educational technologies such Computer Assisted Learning (CAL) systems (Jones et al., 1999; Squires & Preece, 1996), the existing evaluative methods and frameworks for CASE tools are mainly suitable for evaluation in industrial environments. The existing frameworks for the evaluation of CASE tools are mainly intended for its evaluation and selection in organizations (Misra, 1990; Mosley, 1992). They specify an exhaustive set of evaluation criteria, and Usability or learnability evaluation is not the single main focus of these frameworks. In general, these frameworks provide insufficient support for the evaluation of CASE tool learnability in educational environments. A new evaluation framework has been proposed to support the learnability assessment of CASE tools in educational environments.
The framework is used to find out the i) effect of contextual factors such as user, task and learning environment on CASE tool learnability and ii) whether the CASE tool allows learners with different characteristics to successfully learn the tool. The framework is specific to educational environments and oriented towards CASE tools and incorporates learnability principles. It is customizable and extensible to allow changes. The framework may be used by researchers to evaluate CASE tool learnability in different educational environments.
The application of the framework to evaluate the learnability of Rational Rose, a CASE tool used in an undergraduate Systems Analysis and Design course, at the University of Auckland, New Zealand is described. It is believed that the results and findings from this research will allow for the experiences to be shared and add value to a better understanding of CASE tool learnability in educational environments. …