Web-Based Course Management Software: An Empirical Study of Faculty Usage
Jones, Gary H., Jones, Beth H., International Journal of Instructional Media
There is no question the Internet can be a valuable educational resource. As the U.S. Congress' Web-Based Education Commission project director extols:
"In the course of our work, we heard from hundreds of educators, policymakers, Internet pioneers, education researchers, and ordinary citizens who shared their powerful visions and showed us the promise of the Internet ... We heard that the Internet enables education to occur in places where it normally does not, extends resources where there are few, expands the learning day, and opens the learning place. We experienced how it connects people, communities, and resources to support learning. We witnessed how the Internet adds graphics, sound, video, and interaction to give teachers and students multiple paths for understanding". (Fulton, 2001, p. 1)
But how can such promise be realized? One type of system designed to assist teachers in harnessing the Web's capabilities is Web-based course management software (course management software). CourseInfo, WebCT and similar commercial software teaching aids provide teachers the ability to design and maintain advanced Web components for their courses with minimal Web design experience and negligible knowledge of hypertext markup language (HTML). "The software's ease of use is the biggest thing," stated Aaron Goldstein, a Client Relationship Manager for Blackboard. "We have a vision of helping universities and faculty focus on education so that as we come out with [new features], it allows other things to fade into the background and education to step forward. That's always been the center of Blackboard. It's a really powerful vision and it's something that we really try to focus on" (A. Goldstein, personal communication, December 17, 2004).
Business Wire, July 14, 2004, summarized the results of a WebCT survey conducted by Boston-based Atlantic Research & Consulting in April, 2004 as follows: "Thirty-seven percent of the survey's 416 respondents say they have implemented e-learning institution-wide, up from 25 percent in 2002 ... Student participation in e-learning is growing at a 31 percent clip, and faculty members are catching up to the demand with a 44 percent aggregate growth rate in c-learning participation, according to the survey. The survey results indicate e-learning is no longer a peripheral part of education at colleges and universities around the world" (WebCT, 2004, p. 1). The educational technology research company MDR stated in its 2002-2003 College Technology Review: "Contributing to the increasing use of electronic communication on campus, course management systems are present in virtually all (94%) of colleges." (MDR, 2002-2003, p. 3).
Considering the vast amounts of money, time, and energy being invested in Web-based teaching programs across our nation's educational institutions, a greater understanding of their use by faculty is needed. Which features do faculty find most useful and which capabilities, if any, are being virtually ignored? Such knowledge can aid university software implementers and support staff. It can help software designers build more effective systems. It may also be useful to faculty considering adopting this software for their classroom room as well as new faculty users. It is certainly of interest to university administrators who are, have, or might be responsible for approving the purchase of course management software. The purpose of the present study was a preliminary determination of faculty usage of the CourseInfo instructional technology package at a Midwestern regional comprehensive university.
Oliver (2000) surveyed Virginia Tech faculty using CourseInfo to supplement on-campus courses to determine how faculty were using this tool (Oliver, 2000). Sample findings from 38 survey respondents showed 82% placed content online for students to access, but only 29% grouped students in electronic teams for document sharing and cooperative tasks. Further, 97% used the tool for online announcements, but only 29% used digital drop boxes to exchange papers and provide formative feedback on student artifacts. While some faculty used CourseInfo to modify and extend their teaching and learning practices, many utilized the system only to promote efficiency and increase information access. The authors questioned whether such foci alone promote higher-order thinking and improved student learning.
A few studies have reported on aspects of faculty opinions concerning their use/nonuse of the Web as an educational resource. Grankovska and Heines (2002) surveyed both faculty who use (125 respondents) and those who do not use (25 respondents) course Websites. Both groups claimed a lack of time (85% and 64%, respectively) as a serious obstacle. They reported three additional obstacles that prevented them from "making their course Website everything they want it to be" or from using course Websites altogether. In order, these were the lack of know-how, technical assistance, and tools (those respondents lacking "tools" most likely did not have course management software available).
Witt (2003) surveyed the 36 instructors using supplemental Websites for a classroom course at an urban campus in the South. (Course management software was not available at this institution.) Twenty-four responded with data concerning 30 course Websites. Seven questions were asked, basically to assess "Are course Websites worth the trouble?" Most Websites (77%) were created by the instructors at zero dollar cost (90%), but had taken an average of 13.6 hours to construct. The majority of instructors indicated that their goals for the Websites had been entirely or largely achieved (60%) and considered the sites to be essential to successful course design (60%). Only two respondents indicated that the attempt to maintain a course Website was more trouble than it was worth. Type of usage was not measured specifically.
These studies show that university faculty have positive opinions of the World Wide Web as an educational tool and that they are integrating its use into their classrooms in a variety of ways. The present study adds to this line of research by examining all active CourseInfo Websites at one university and surveying faculty users concerning their usage.
The objective of this project was to gain an understanding of faculty CourseInfo use at a comprehensive regional Midwest university. The questions to be answered objectively (by examination of Websites) were: who was using it, which features were they using, and to what extent were they using these features? Additional questions, addressed by a survey, were what feature(s) did faculty perceive to be most important and why.
Research questions posed by this study were addressed by individual examination of university CourseInfo Websites and by survey of faculty CourseInfo users. The institution where the research was conducted is a four-year regional comprehensive Midwest public institution with a declared liberal arts mission. The student body size is approximately 6,000 undergraduate students, with virtually 100% of the student body consisting of the traditional 18-22 year-olds.
In Phase I of the study, a total of 213 Websites were examined. An "active" site was defined as one having at least one posting in any of these five "button" areas: Announcements, Course Information, Staff Information (information on the person teaching the course), Course Documents, Assignments, or any site that had at least one External Link (a link any other web page). Breadth-of-use was measured by the number of software features used (of the six areas listed above). A feature was considered "used" if there was at least one posting under that feature, or in the case of External Links, at least one link to another Website. Extent-of-use was measured by the total number of postings or links found.
By default, guest access does not permit visitation of the Communication and Student Tools features of the sites. A few instructors restricted access to other sections (and, in one case, the entire site) as well. In the case of breadth-of-use, it was assumed restricted features were being used, and these were included in the count of software features utilized. When counting total number of postings for the extent-of-use measure, restricted features could not be included. The number of restricted sites was relatively small, however, ranging from 0 to 36 (depending upon the specific feature) or 0% to 19% of the 181 sites determined to be "active."
After faculty users were identified in Phase 1, the second phase of the project was performed. This consisted of a short survey formatted for the Web and posted on the principal author's university Website. A link to this survey, with brief cover letter and permission request, was e-mailed to all 97 professors found to be using CourseInfo.
Of the two-hundred thirteen (213) total CourseInfo sites established on the university computer network, 26 did not meet the "active" criteria (i.e., did not have a single posting or link) and 6 were administrative sites. The remaining 181, included in the following analysis, represent 97 faculty (55 faculty had multiple sites), or nearly 25% of the entire faculty of 400. A summary of usage by academic division can be seen in Table 1.
It can be seen that the academic division with the least involvement in CourseInfo is the Education area, with only one user out of twenty-four faculty. As a percent of total division faculty, the Business Division clearly reflects the most interest in CourseInfo, with almost 40% of its faculty using it to some extent. The division with the next highest usage is Science with 32% of its faculty using it. Faculty in two divisions had little interest in the software. Education, as mentioned, and Fine Arts which only had four out of 36 faculty using CourseInfo. About a fourth of the faculty in each of the remaining four divisions used it.
Once the users of CourseInfo were identified, the next question was, how were they using it? As discussed previously, two measures were used to determine usage: breadth-of-use refers to the number of features used and extent-of-use refers to the total number of postings. Table 2a shows the number of courses in which each feature was used, broken down by academic division. Table 2b begins with a sum of these features from Table 2a, that is, it gives the total number of features used by division, the total number of courses using CourseInfo by division, and an average number of features used by course per division. The purpose of these tables is to see what features were most popular overall and if certain divisions were making more use of the various features than other divisions.
Breadth-of-use by division correlated with the number of CourseInfo courses taught, as seen in Table 2b. The Language and Literature division, with the most CourseInfo courses (50) had the highest number of features used (161), followed by Social Science (135 features, 35 courses), Science (86 features, 30 courses) and Business (70 features, 18 courses). Overall, on average 3.5 features were used per course. With the exception of the sole Education course that used only one feature, course breadth-of-use in each division ranged around this mean (2.8-4.1 features used per course.)
For another perspective on breadth-of-use, the number of features used on each faculty Website was counted. Some faculty chose to use just one feature of the CourseInfo template. Specifically, thirty-four out of the 181 Websites (19%) used a single feature only. But much more often faculty used more than a single feature. On twenty-nine sites (16%) two features were used, on 23 sites (13%) three features were used, 41 (23%) sites used four features, 34 (19%) used five features, and 26 (14%) used all six features. Overall, the number of features used per faculty CourseInfo Website was fairly evenly divided over the range of one feature to the six features investigated in this study. These results show that faculty who chose to use CourseInfo did, for the most part, make use of several aspects of it.
Another measure of usage was extent-of-use, where the total number of postings in each CourseInfo area was examined. Table 3 shows the usage by number of postings. It can be seen that faculty most commonly posted between I and 3 items for every CourseInfo feature they used. For Course Information and Staff Information, few faculty posted more than 1-3 documents. For all four of the other CourseInfo features, many Websites had more than three postings. In the case of course documents, for example, 25 sites had seven or more items posted.
Course Documents and Assignments were used more extensively by faculty who chose to use these features. Respectively, 6.1 and 5.8 postings per Website were found under these buttons. Adding more understanding to the usage of External Links was the number of links found per course. While fewer than expected faculty used external links, those that did used it more heavily than any other feature. There was also the greatest disparity in extent-of-use in External Links, with some professors posting over 40 links and others posting none. This disparity might be attributed to the fact professors who are frequent Web browsers will inherently discover, and thus be able to post, many more pertinent links than those who do not search the Web often. The course's subject matter must also be taken into consideration since there are probably more relevant links available for, say, a course on the military campaigns of World War II than an advanced Calculus course.
As with breadth-of-use, the extent-of-use by division correlates to the number of courses taught in each division, that is, the greatest number of postings are found in divisions with the greatest number of CourseInfo courses.
In Phase II, faculty who actively used CourseInfo were asked to complete a brief on-line survey. Of the 97 active users, 44 responded. The breakdown of respondents by academic division is shown in Table 4.
Three of the survey questions were Likert-scale questions asking about faculty usage. These questions were: (1) I am an active CourseInfo user, (2) I am confident in my ability to use CourseInfo, and (3) CourseInfo is a useful educational tool. (See Figures 1a-1c, respectively).
All faculty users agreed that CourseInfo was a useful educational tool, with the exception of one who was "neutral." This stands to reason, as faculty who do not believe the system is useful would likely consider their effort learning and using it wasted energy. The vast majority of faculty agreed or strongly agreed that they were active users and felt confident in their ability to use the software. About 10% of the respondents (5 out of 44 in the questions of active use and 4 out of 44 in the question of confidence in using) were either neutral or tended to disagree.
[FIGURE 1A OMITTED]
[FIGURE 1B OMITTED]
[FIGURE 1C OMITTED]
Another question asked faculty to rank the top three (only) features of the Courseinfo program. Faculty ranked "links to internal course documents" and "teacher-to-student email" as 1 and 2, respectively. The next three features, in order of ranked usefulness were: links to external Web resources, student-to-student email, and online grade sheet.
When asked the most important factor in choosing CourseInfo features to in their courses, 22 faculty (50%) indicated "general usefulness" and 34% marked "relevance/contribution to course content." The remaining 16% of responses were about equally split between "knowledge of how to use the feature" and "time required to set up/maintain the feature."
Measured by percent of faculty who used a CourseInfo Website for at least one course, the Business Division showed the highest usage level (39%), followed by Science (32%). The divisions with the least interest in the software were Education (4%) and Fine Arts (11%). In each of the remaining divisions, approximately one-fourth of the faculty used it. Apparently, either the software lends itself more easily for use in more technical areas or the faculty in those areas are more open to incorporating technology into their teaching.
In terms of breadth-of-use 'announcements' was the most frequently used feature. Also heavily used were the Course Documents, Course Information, and Assignments areas. Less used was External Links, with less than half of the sites (41%) including such links. Rarely (19%) did a CourseInfo site make use of only one feature. Faculty generally used more than one feature, with the number of features used per faculty CourseInfo Website fairly evenly divided over the range of features. The conclusion is that faculty who chose to use CourseInfo did, for the most part, make use of several aspects of it.
In terms of extent-of-use, Course Documents and Course Assignments were used most extensively by faculty who chose to use these features. While fewer than half the faculty used External Links, those that did used this feature more heavily than any other feature.
Results on the three survey questions were all very positive, as one might expect given that only users were surveyed. Faculty users agreed that CourseInfo was a useful educational tool, agreed they were active users and felt confident in their ability to use the software. On these last two questions, only 10% of the respondents were either neutral or tended to disagree.
Both the direct Website analysis and the survey indicated that 90% of faculty who chose to use CourseInfo not only considered themselves active users of the software, but were found to be active users. Approximately 10% of faculty users did not make extensive use of the software--the total sum of their activity was to post one announcement or one syllabus. The vast majority of faculty who choose to use CourseInfo tended to use it actively, in the manner described above.
One limitation of this study is that it was conducted at only one university. While this university may very well be a good representative of course management software across the country, there is no guarantee that these results are generalizable to other institutions.
This study does not attempt to provide an assessment of the actual benefits or effectiveness of the CourseInfo program. One cannot assume that simply because a particular feature is most frequently used or highly regarded by faculty members that it is, in fact, the most relevant, important, or useful feature on the system. Popularity does not always equate to utility.
In the survey portion, participation among members of that population was voluntary, introducing a self-selection bias. As a result, it is likely that faculty samples were biased in the direction of computer literacy.
Direct Reprint Requests to:
Gary H. Jones, Ph.D.
College of Business
Western Carolina University
Cullowhee, NC 28723
Grankovsak, S. & Heines, J. (2002). Course Web sites: State-of-the-Art. USDLA Journal [On-line], 16(12). Available: http://www.usdla.org/html/joumal/DEC02_Issue/article0l.html
Fulton, K. (2001). From promise to practice: Enhancing student Internet learning [Online]. Available: http://www.infotoday.com/MMSchools/mar0l/fulton.htm
MDR (Market Data Retrieval, a Company of D&B). (2002-2003). The college technology review, 2002-2003 academic year [On-line]. Available: http://www.black board.comJdocs/MDR2003.pdf
Oliver, K. (2000). Studying and improving faculty use of a Web course management system: Flashlight case study. F-LIGHT.- E-Newsletter for the Flashlight Program for the Study and Improvement of educational Uses of Technology [On-line]. Available: http://www.tltgroup.org/resources/F-LIGHT/f-light_Aug2000.html.
WebCT user survey shows significant increase in enterprise-wide implementations, usage and user satisfaction. (2004, July 14). Business Wire, p. 1. Retrieved from North Carolina Libraries for Virtual Education, InfoTrack OneFile Plus, September 9, 2004.
Witt, P.L. (2003). Enhancing classroom courses with Internet technology: Are course web sites worth the trouble? Community College Journal of Research and Practice, 27(5), 429-438.
GARY H. JONES, PH.D & BETH H. JONES, PH.D.
Western Carolina University
TABLE 1. NUMBER OF COURSEINFO WEBSITES AND INSTRUCTORS BY DIVISION Number Number of Faculty Average Number Academic Division of Courses (approx. Courses per % of total Instructor division faculty) Business 18 11 (39%) 1.6 Education 1 1 (4%) 1 Fine Arts 12 4 (11%) 3 Human Potential & Performance 17 10 (26%) 1.7 Language & Literature 50 23 (23%) 2.2 Mathematics & Comp. Science 18 8 (21%) 2.2 Science 30 21 (32%) 1.4 Social Science 35 19 (24%) 1.8 Total 181 97 1.9 TABLE 2A. NUMBER OF COURSES USING EACH FEATURE BY DIVISION Course Course Division Information Staff Documents Assignments Business 14 5 15 11 Education 0 0 0 0 Fine Arts 7 0 7 7 Human Potential 14 5 14 11 & Performance Language & 29 15 30 29 Literature Mathematics & 13 9 15 16 Computer Science Science 17 6 20 15 Social Science 21 17 22 23 Total 115 57 123 112 External Division Links Announcements Business 9 16 Education 0 1 Fine Arts 2 11 Human Potential 8 12 & Performance Language & 25 33 Literature Mathematics & 7 14 Computer Science Science 4 24 Social Science 20 32 Total 75 143 TABLE 2B. AVERAGE NUMBER OF FEATURES USED BY COURSES IN EACH DIVISION Total Average Features Used Number of Number of Division (Sum of Table Courses Features Used 2a rows) by Course Business 70 18 3.8 Education 1 1 1.0 Fine Arts 34 12 2.8 Human Potential & 64 17 3.8 Performance Language & Literature 161 50 3.2 Mathematics & 74 18 4.1 Computer Science Science 86 30 2.9 Social Science 135 35 3.9 Total 625 181 3.4 TABLE 3. FACULTY USAGE OF COURSEINFO FEATURES BY NUMBER OF POSTINGS Number of Courses with: 1-3 4-6 7+ Unknown Courselnfo Postings Postings Postings (restricted Features sites) Course information 83 14 1 17 documents Staff information 34 0 0 23 documents Course documents 48 18 25 32 Assignment 43 16 22 31 documents * External links 22 9 22 22 Announcements 92 30 21 0 Total No. of Total Number Course Using of Postings on Courselnfo this Feature unrestricted Features sites) Course information 115 187 documents Staff information 57 38 documents Course documents 123 557 Assignment 112 471 documents * External links 75 432 Announcements 143 524 Excluding one outlier (141 documents) TABLE 4. SURVEY RESPONDENTS, FACULTY, FREQUENCY AND PERCENT BY ACADEMIC DIVISION Academic Division Frequency Percent Business & Accountancy 5 11.4 Education 1 2.3 Fine Arts 2 4.5 Human Potential & Performance 6 13.6 Language & Literature 12 27.3 Math & Computer Science 4 9.1 Science 9 20.5 Social Science 4 9.1 Missing 1 2.3 Total 44 100.0…
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Web-Based Course Management Software: An Empirical Study of Faculty Usage. Contributors: Jones, Gary H. - Author, Jones, Beth H. - Author. Journal title: International Journal of Instructional Media. Volume: 35. Issue: 3 Publication date: Summer 2008. Page number: 251+. © 1999 Westwood Press, Inc. COPYRIGHT 2008 Gale Group.
This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.