Academic journal article The Journalism Educator

Computer-Mediated Communication: An Experimental Study

Academic journal article The Journalism Educator

Computer-Mediated Communication: An Experimental Study

Article excerpt

Computer-mediated communication systems (CMCS) are widely used in government and industry to deliver electronic mail and conduct conferences on line. Home use is growing as well. Prodigy, one of several large public systems, claims 1.75 million subscribers (Ziegler, 1992). Many academics use computers to communicate with colleagues across campus and around the world (Parker, 1991).

This study investigated the effectiveness of a computer-mediated communication system in supplementing traditional instruction in a media law course.

Computer-mediated communication seemed especially appropriate for the media law class, since its relatively large section size and infrequent meeting schedule made close contact with students by traditional means difficult.

The technology has drawn the interest of some interpersonal communication researchers who have focused on how differences in the medium affect relationships that develop among those who communicate on line rather than face-to-face. Walther and Burgoon (1992) provide an extensive bibliography of such research.

At a few schools, courses (Hiltz, 1988; Kaye, 1990) or entire degree programs (University of Phoenix, 1990) are delivered by computer, often to students living at great distances from the campus. Such systems have also been used in some high school journalism classes (Oates, 1987; Perkins, 1991.

Most computer-mediated communication research has been published in education, computer science, psychology or management journals, rather than those of the communication field. In addition, as Wells (1990) noted in a review of more than 250 literature references on educational uses of computer-mediated communication, "rigorous experimental studies...are a rarity" (p. 3).

Only a few reports exist of the use of computer-mediated communication in journalism education. Lieb (1990) found that over three-fourths of the students in a feature-writing class that used computer conferencing said they would like to use it in other writing classes. He also reported his informal assessment that students did much higher quality work in sections of the class that featured computer conferencing.

Smith, Kim, and Bernstein (1992) offered anecdotal evidence that students in a large lecture class were enthusiastic about electronic mail as a technique for contacting the instructor; that students in a television news reporting class found e-mail helpful in scheduling equipment and finding partners to help shoot stories; and that e-mail proved useful for facilitating peer evaluation of stories in a news writing class.

However, in a study involving several non-journalism classes, Hiltz (1988) found that despite greater satisfaction among students in courses using computer-mediated communication, there were no significant differences in scores measuring mastery of course material.

A different study, of four commercial CMCS, found that female users were more satisfied with the systems than males (Hiltz & Johnson, 1990). This contrasts with findings of numerous studies of other aspects of computerization in which males are generally the more active and satisfied users (Canada & Brusca, 1991).

Computer-mediated communication systems can deliver traditional mass media content, from news articles to research reports. They also can serve as research tools to explore full text and bibliographic databases. However, such functions were not available on the system used at the research site. So, these aspects of computer-mediated communication are not included in this study.

Hypotheses

Based on existing research it was hypothesized that:

1. Students using computer-mediated communication would report more satisfaction with their version of the course than students in traditional sections.

2. Students with greater prior computer experience would be more satisfied with the computer-mediated communication process than students with less experience.

3. Female students would be more satisfied than males with the computer-mediated communication process.

4. Students in the computer-mediated communication treatment condition would not achieve higher scores on the final exam in the course than students in the traditional sections.

In addition, the research sought student comments on various aspects of the course design.

Method

One hundred thirty undergraduate journalism majors in three sections of a media law class participated in the experiment as part of their regular class work. Students in two sections had the course delivered in the traditional way, while students in the third section received the computer-mediated communication treatment.

The course is taught in a lecture and discussion format and meets once a week for 190 minutes. The three sections were offered in different quarters of the same calendar year, but were taught by the same instructor, used the same text, covered the same content and required similar assignments. Students were not aware when they signed up for the course whether the section in which they enrolled would use the experimental or traditional approach.

The computer software used for the experiment was a mainframe-based conferencing system available on the campus for several years. The conferencing software was based on the CoSy system developed at the University of Guelph, Ontario, but had been modified locally. Students could access the system using terminals or personal computers located in several labs on the university's main and branch campuses. Students with personal computers and a modem could also connect to the system from their homes or workplaces.

Students in the experimental section were required to visit the campus computer center to sign up for a free account on the conferencing system. They also received instructional materials from the computer center on system operation. The instructor devoted an initial hour of class time to discussing the system's operation and held several follow up discussions to respond to student questions about the system. A planned hands-on class session in system operation was cancelled when it was discovered that a new campus computer lab, the only one large enough to accommodate the entire class at once, had not yet been connected to the conferencing system.

The conferencing software allowed students to send private electronic mail messages to any person on campus with an account on the system--including the instructor and all other students in the class. In addition, the instructor established a number of on-line bulletin boards and discussion areas. Some were "read only" where the instructor posted rules for the course or assignments. Others were "read/write," so students could post their own comments and read those posted by others. Separate discussion areas were created for general class discussion of key topics and as places for smaller teams of students preparing for in-class research presentations to post material.

The instructor devised a series of weekly on-line quizzes based on readings for the class. Students were expected to use the computer system to get the questions and post their answers. This was designed to assure student participation in the on-line component of the course and give the instructor information about what students understood from the readings before the class meeting each week. However, many students complained about difficulty gaining access to the system and composing answers on line. In response, the instructor made questions for later quizzes available at the department office as well as on line. In addition, responses were accepted on paper if the student could show evidence of having tried to use the system.

Three different types of quizzes were used--a short-essay format early in the quarter, multiple short-answer questions toward the middle of the quarter and participation in an on-line discussion forum late in the quarter.

Students in the computer-mediated communication section were asked to complete one-page questionnaires about the experiment early in the quarter, at the midpoint and again toward the end of the quarter.

At the end of the course, students in all three sections completed a brief evaluation survey used for all the department's classes.

Scores on identical true-false and multiple-choice questions asked in the final exam for each section provided the basis for comparing student performance.

Experimental group surveys

Computer experience levels. Responses to demographic questions in the first survey indicated most students were familiar with computers, but not with conferencing software.

The class was composed primarily of juniors and seniors. Of the 44 students responding, all but one reported having used a computer before, generally either an Apple Macintosh, an IBM compatible, or both.

Just over half reported prior use of campus computer labs. Users averaged one visit a week.

Thirty-nine percent had a computer available in their living quarters. Only 14 percent had a computer with the modem necessary for telecommunications. Sixteen percent reported having used telecommunications software. Only one student reported going on line more than once a week.

Eight students reported having an account on the campus computer conferencing system when the course began. Seven said they had taken a Western Civilization course taught by the History department, one of the few other courses at the university which used the conferencing system. Of those, only three reported computer conferencing had improved the course.

Experience and attitudes. An index was created from the seven "yes/no" questions about computer usage. Students with very high computer experience, indicated by four or more "yes" answers, were compared with those who had less computer experience in their attitudes toward the experiment. Marginally significant differences were found in responses to three attitudinal questions, as shown in Table 1. Responses to three other questions, about satisfaction with the on-line quizzes, showed no significant differences.

Gender and attitudes. No significant differences emerged between males and females in attitudes toward the experiment, except that males (M = 3.84) were marginally more likely to report being "pretty comfortable" using the conferencing system than females (M = 3.29), t(34) = 1.51, p< .1.

Learning-style preferences. The students in the first survey were asked to rank five different techniques for learning about a subject. Of the choices offered, "making something yourself--writing a story, building a model" and "participating in a discussion" were nearly tied as the most popular choices (M = 2.10; M = 2.12). "Hearing a lecture" ranked third (M = 2.87), collaborative work, "making something with others" ranked fourth (M = 3.08) and "reading a textbook" was fifth (M = 3.72). However, even the least popular choices were ranked either first or second by more than 20 percent of the students.

Quiz-type preferences. Students reported only modest differences in their assessment of the effectiveness of the three different on-line quiz types as a learning tool. They were asked to consider how time consuming the quiz was and how much they learned from it. On a five-point scale from least to most effective, the highest effectiveness value was 3.64, for the short-answer quizzes. The discussion-format quizzes scored 3.55 and the short essays scored 3.29.

By contrast, when asked which quiz type they liked best, six said the essays, 17 the short-answer format and 20 the discussions. Asked to indicate why they liked a particular quiz type, nine students said they liked the short-answer quizzes because they were quick or easy. Two said such quizzes encouraged them to read the textbook, and one said printouts of the short-answer questions were an effective tool in reviewing for the midterm exam. Proponents of the essay quizzes tended to say they learned the most from essay-style quizzes or found them thought-provoking. Those who liked the discussion questions best tended to say they found them the most thought-provoking, interesting or challenging and that testing their own ideas in discussion against those of others helped them learn more.

Did on-line work help? Most students (n = 29) said they believed that the on-line features of the class helped rather than hurt their learning in the class. But six said those features hurt their learning and seven offered no response or said they neither helped nor hurt. This fairly substantial negative response showed up more dramatically on the departmental course evaluation survey.

Departmental survey

The departmental survey's questions are framed as seeking reactions to the instructor rather than the course itself, but since the instructor and other aspects of the class were the same for all sections, it is reasonable to attribute variations to the computer-mediated communication experiment. Asked their overall rating of the instructor, students in the traditional sections reported significantly more satisfaction (M = 2.96) than did students in the experimental section (M = 2.23, t(87) = 4.01 p< .001.

In addition, 25 of the 35 students completing the evaluation of the experimental section, when asked if they had any suggestions or other comments, volunteered negative observations about the computer conferencing system. Only two offered favorable comments about it.

Some of the negative comments objected to using the computer system for required quizzes, but even more objected to using the system at all. One seemed to express the views of many others in urging the instructor to "forget" the system. "it's too much of a hassle," the student said.

Exam results, No significant difference was found in final exam results between the traditional (M = 32.6) and on-line (M = 33.6) sections, t(119)= 1.02.

Discussion and conclusion

The mixed results on the measures of satisfaction with the computer-mediated communication version of the course are discouraging. While most students ultimately concluded that the new technology "helped," a substantial minority hoped never to be subjected to it again. Their dissatisfaction is especially troubling, since the experiment followed a pattern noted by Hiltz (1988) and others in requiring the instructor and students to spend more time on the course than did the traditional approach. More work and less satisfaction are not an appealing combination.

The somewhat higher satisfaction levels for experienced computer users are to be expected, and offer hope that as computer usage becomes more widespread, resistance to computer-mediated communication for instruction may diminish. While females still reported marginally less satisfaction on one measure, the differences as a whole were less than those noted in many other studies.

While not unexpected, the lack of significant improvement in exam scores is also cause for concern. It suggests that at least in this instance the computer-mediated communication strategy failed to further the central goal of any course--increasing knowledge of the course subject matter. It's possible that the objective examination failed to capture some of the distinctive learning that took place on line, but we lack proof of that. It's also possible that better software, an improved introduction to the system and an improved course design might boost the slightly higher scores reported for the on-line section to statistically significant levels. Proof of such speculation will have to await further research.

At the same time, it's also noteworthy that the system did not have a negative impact on learning. This experiment should in no way discourage use of computer-mediated communication for distance education or other settings where the alternative may be to not deliver the course to a given audience at all.

Implications for the future

This experiment suggests caution in future efforts to augment a traditional class with computer-mediated communication techniques. Several concerns must be addressed.

Adequate hardware and software. Many mainframe-based computer-mediated communication systems available on college campuses are extremely unfriendly, especially for novice users. Documentation frequently is sparse and confusing. Developing custom documentation will add to the preparation time for the course, but may be important to its success (Phillips & Santoro, 1989). Instructors generally are not in a position to remedy limitations in the software itself, but they may wish to become advocates for the purchase of improved systems.

Introduction to system. Some researchers in the distance-education field have concluded that bringing students together for an initial introduction to the computer system is unnecessary. But for on-campus courses, it would seem that time devoted to a hands-on introduction to the computers and software would be highly beneficial. The last-minute forced decision to drop it from this experiment is seen by the instructor as a likely strong negative influence on the outcomes. Such hand-holding would appear to be especially necessary for systems using older command lines rather than contemporary graphical interfaces.

Course design. On-line computer conferencing appears easiest to add to a course in which students already expect to regularly use the same computers for news writing or database searching. Instructors of courses that don't already use computers need to carefully assess whether computer access on their campus is adequate to support a successful computer-mediated communication experience. Commuter students who don't have a computer and modem at home may be especially disadvantaged if campus labs are overcrowded or inadequately maintained.

Computer-mediated communication advocates often promote the systems as ideally suited to encouraging a collaborative style of learning. Questionnaire responses in this study, while not settling the issue, do raise some questions about how favorably disposed most journalism students are to engage in collaborative projects. It may simply be that they've had little exposure to such approaches; or had unsatisfying past experiences. But it is also possible that those who seek careers as writers may be less inclined toward collaborative work than those in other disciplines. By contrast, group discussions emerged as a favored quiz style among the study participants.

Class size. While computer-mediated communication can make it possible for many students to participate in a discussion, it does not create more time for the instructor to attend to their comments. An instructor who finds a discussion format manageable in a seminar of 15 may find converting a lecture class for 50 to an on-line discussion format quite daunting. Some researchers suggest one instructor or tutor per 25 students is an upper limit if active discussion is expected (Mason, 1988).

Participation incentives. Although experiments in the workplace suggest that being ordered to use a computer-mediated communication system increases dissatisfaction with it (Hiltz & Johnson, 1990), instructors tend to conclude that students who aren't required to do something won't do it. The usual coercion is to link some portion of the grade for the class to system usage.

In a large class, essay, short-answer and multiple-choice quiz formats are not well suited for on-line use with basic conferencing systems. The instructor in this experiment found it time-consuming to grade such quizzes and post individual on-line responses for the students. Students, especially early in their system-usage experience, frequently found on-line quizzes extremely stressful. (When asked at mid-quarter, 89 percent of students in the experimental group said they would prefer on-line delivery of the quizzes to be made optional.) Other computer programs could generate, score and provide feedback on quiz questions much more effectively off-line. On-line features might then be used to deliver a summary of the quiz results to the instructor for use in planning classroom lectures and discussions.

Students were more satisfied with the discussion-forum approach. However, the conferencing software used did not provide summary statistics for the instructor on how often students contributed to the discussion. Such data had to be tallied manually while scanning the conference messages. The system also provided no information about whether students had read the posted messages without adding their own--what some researchers refer to as "lurking" in a conference. Lurking, if considered a problem, could be cured with a system described by Hiltz (1986) in which students were required to answer a quiz question before viewing the responses of others.

While quantitative measures of discussion participation are crude, the instructor in this experiment concluded that providing a qualitative evaluation of each student's conference participation would be too time-consuming to be practical.

Instructors need to carefully assess whether the benefits of computer-mediated communication systems are great enough to successfully support such an experiment.

Canada, K., & Brusca, F. (1991). "The Technological Gender Gap: Evidence and Recommendations for Educators and Computer-Based Instruction Designers." Educational Technology Research and Development, 39(2), 43-51.

Hiltz, S. R. (1986). "The 'Virtual Classroom': Using Computer-Mediated Communication for University Teaching." Journal of Communication, 36(2), 95-104.

Hiltz, S. R. (1988, June). Collaborative Learning in a Virtual Classroom: Highlights of Findings. Paper presented at the Computer Supported Cooperative Work Conference. (ERIC Document Reproduction Service No. ED 305 895)

Hiltz, S. R., & Johnson, K. (1990). "User Satisfaction with Computer-Mediated Communication Systems." Management Science, 36(6), 739-764.

Kaye, A. R. (1990). Computer Conferencing and Moss Distance Education (CITE Rep. No. 98). Milton Keynes, U.K.: Open University, Centre for Information Technology in Education. (ERIC Document Reproduction Service No. ED 320 531)

Lieb, T. (1990). "Computer Conferencing Offers New Way to Think about Writing." Journalism Educator, 45(2), 32-37.

Mason, R. (1988). The Use of Computer-Mediated Communication for Distance Education at the Open University, 1988 (CITE Rep. No. 56). Milton Keynes, U.K.: Open University, Centre for Information Technology in Education. (ERIC Document Reproduction Service No. ED 320 531)

Oates, R. H. (1987). "Computer-Mediated Communication for High School Classroom." Quill and Scroll, 61(2), 8-10.

Parker, E. S. (1991). "Computer conferencing offers boundless geography, time." Journalism Educator, 45(4), 49-55.

Perkins,. (1991). "Bring the "Real World" into the Classroom." Communication: Journalism Education Today, 24(4), 18-19.

Phillips, G. M., & Santoro, G. M. (1989). "Teaching Group Discussion Via Computer-Mediated Communication." Communication Education, 38(2), 151-161.

Smith, C., Kim, H., & Bernstein, J. (1992, August). Because It's Time: Teaching Computer-Mediated Communication. Paper presented at the meeting of the Association for Education in Journalism and Mass Communication, Montreal, Canada

University of Phoenix (1990). University of Phoenix on-line Education Delivery System No. San Francisco: Author.

Walther, J. B., & Burgoon, J. K. (1992). "Relational Communication in Computer-Mediated Interaction." Human Communication Research, 19(1), 50-88.

Wells, R. A. (1990). Computer-Mediated Communications for Distance Education and Training: Literature Review and International Resources. Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences. (ERIC Document Reproduction Service No. ED 343 569)

Ziegler, B. (1992, September 10). "Prodigy to Add Services, Raise Rates." Associated Press wire service.

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.