Academic journal article The Journal of Research in Business Education

The Effects of Systematic Changes to a Business Course over a Three Year Period

Academic journal article The Journal of Research in Business Education

The Effects of Systematic Changes to a Business Course over a Three Year Period

Article excerpt


The purpose of this study is to examine the measurable impacts of shortening lectures and increasing engaged learning activities in an undergraduate marketing course. Class pedagogy is progressively changed over six semesters from a lecture-based format to a hybrid of lecture and engaged learning activities. The researchers ask whether these pedagogical changes positively affect course outcomes as measured by Student Evaluation of Teaching (SET) scores and exam scores. The intent was to engage students, increasing their perceived value of class time activities, and consequently favorably impacting student satisfaction and performance. Evidence supports increased SET scores, but exam scores did not show consistent improvement.


Student Evaluations of Teaching (SET) scores are widely used by institutions of higher education to evaluate teaching effectiveness (Chen, Gupta, & Hoshower, 2004) and to measure broadly classroom quality management (Kwan, 1999). SET scores are also used at the departmental-level to help make decisions about tenure, promotions, and raises (Chen, Gupta & Hoshower, 2004; Read, Rama & Raghunandan, 2001; Simpson & Siguaw, 2000). Instructors acknowledge the importance of SET scores for improving their own teaching. In a faculty survey, Simpson and Siguaw found that 85% of the professors believed that SET scores were important to their departments, and 92% said that their SET scores were important to themselves. As suggested, most academic institutions rely heavily on SET scores to evaluate instructors. This measurement is often used in conjunction with other measures, but some institutions use it as the sole or primary measure on which personnel decisions are justified. In many cases, SET scores are used to assist instructors in improving course content or pedagogy. However, considerable conflicting opinion exists on the validity of SET scores as a measure of teaching effectiveness as it relates to student performance.

A substantial body of research supports a positive relationship between SET scores and student performance. In a meta-analysis of multi-section validity studies Cohen (1981, p. 300) found a "strong support for the validity of student ratings as measures of teaching effectiveness." Marsh and Roche (1997) support the effectiveness of SET scores as an indicator of effective teaching when using a multidimensional measure, while rejecting a single criterion approach. Often the student performance measure used has been based on the student's perception of learning (e.g., Clayson & Haley 1990).

Numerous studies, including some very recent ones, have called into question the direct relationship between SET scores and student performance. Beyers (2008, p. 106) reasons that "the real lesson is that students are not generally well versed in pedagogy, so their evaluations often have more to do with their emotional experiences than learning." In a study of undergraduate accounting students, Yunker and Yunker (2003) found a negative relationship between student evaluations and student performance. After discussions with a number of faculty and upon reviewing several studies, Wilson (1998, p. 212) stated that "faculty (too often) dumb down material and inflate grades to get good reviews." In a direct comparison of SET and final exam scores, Gramlich and Greenlee (1993, p. 12) report that the "commonly used SETs and the degree to which students actually learn the course material - the two do not seem to be closely related." Sproule (2002, p. 287) suggests that the use of student evaluation data to determine instructor performance "is tantamount to the promotion and practice of pseudoscience . . . ." This paper addresses the relationship among pedagogical applications designed to increase student engagement and the impact on both SET and exam scores.

Literature Review

Literature support of engaged learning pedagogy is substantial. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.