Academic journal article Journal of Educational Multimedia and Hypermedia

Re-Evaluating the Effectiveness of a Web-Based Learning System: A Comparative Case Study

Academic journal article Journal of Educational Multimedia and Hypermedia

Re-Evaluating the Effectiveness of a Web-Based Learning System: A Comparative Case Study

Article excerpt

Much is unknown about the effectiveness of networked technology in education. Although a great deal of research exists dealing with the effectiveness of various types of web-based learning systems, the focal question of most evaluation studies does not concern the innovation of the delivery model itself and the factors that contribute to its effectiveness. Moreover, there is a lack of a theoretical or conceptual framework in many existing studies. The authors applied a systematic evaluation method (called CADMOS-E), and present the quantitative and qualitative results of a re-evaluation study on the effectiveness of a web-based learning system used at the National Technical University of Athens (NTUA), Greece, over a period of three years (1998, 1999, and 2000). The system supports an introductory course in software engineering offered by the Software Engineering Laboratory of the Electrical and Computer Engineering Department at the NTUA. The findings were compared with the results from a study done in 1997. According to that study the "quality of the learning material" alone explained almost 28% of the system's effectiveness (R2adj. = .278). After its first evaluation the system underwent a number of revisions and additions suggested by the results from quantitative and qualitative analysis of students' feedback. In this article, among other things, the authors demonstrate--using quantitative results--that the changes made in the design and presentation of the learning material resources improved the learning effectiveness of the revised system, from 28% to 37.5%.

**********

Networked technology is having and will continue to have a profound impact on education around the globe. It holds significant potential in advancing the interactivity between learners and tutors, in offering flexibility for the means of learning, and in providing easy, one-stop maintenance and reusability of resources (Lowe & Hall, 1999; Nielsen, 1995). However, the educational community has much to learn regarding how and in what ways technology can enhance the instructional process. While there is a large amount of writing devoted to research on the impact of technology in education, there is much that we don't know about its effectiveness. Moreover, there are certain gaps in these research efforts which require further investigation, specifically the lack of a theoretical or conceptual framework (Institute for Higher Education Policy, 1999).

In most evaluation studies, the focal question concerns the comparative effectiveness of various types of web-based learning systems when measured against traditional ones rather than the innovation of the delivery model itself and the factors that contribute to its effectiveness (Learning Technology Dissemination Initiative, 1998; Institute for Higher Education Policy, 2000). Also, there is often little detail on how the evaluation studies have been conducted and on the criteria that were used. In our study we applied a systematic evaluation method, called CADMOS-E (evaluation), to evaluate the learning effectiveness of Web technology in the instructional process as well as singling out the factors that influence its efficacy (Retalis & Skordalakis, 2001).

CADMOS-E method is part of a development methodology for web-based learning systems, called CADMOS (web-based Courseware Development Methodology for Open Learning Systems) (Retalis, 1998). CADMOS methodology embraces the whole development lifecycle of web-based courseware. CADMOS-E is a stepwise evaluation method supported by specially developed pretest and posttest questionnaires that provide data for both quantitative and qualitative analysis. The focus of the evaluation is on the learning effectiveness of the system as a whole as well as the identification of extensions and revisions that need to be made to it. Learning effectiveness is conceptualized as being related to a multiple measurement index consisting of cognitive and attitudinal outcomes (Makrakis, Retalis, Koutoumanos, Papaspyrou, & Skordalakis, 1998). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.