Counseling Students' Decision Making regarding Teaching Effectiveness: A Conjoint Analysis
Pietrzak, Dale, Duncan, Kelly, Korcuska, James S., Counselor Education and Supervision
The authors examined the relative importance of 4 attributes of decision making for student evaluation of teaching effectiveness: perceived knowledge base of the professor, professor's delivery style, course organization, and course workload. Participants were 234 counseling graduate students from 6 midwestern universities in the United States. Data were analyzed with classic conjoint analysis using an additive model with full-profile method. Results indicated that decisions about teaching effectiveness are complex and based on multiple attributes. There was sufficient variability in the decision-making process to warrant caution in overgeneralizing results. Implications for faculty development and faculty hiring are considered.
For better or worse, student input into the evaluation of faculty members has become routine. Typically, institutions of higher education use student surveys of instruction to gather student input on teaching effectiveness (Centra, 2003; Jacobsen, 1997; Wolfer & Johnson, 2003). In turn, data generated from student surveys are often used in the tenure and promotion process and to determine merit pay (Obenchain, Abernathy, & Wiest, 2001). Despite their near ubiquity, the role of student evaluations in the faculty evaluation process as a whole is controversial (Ballantyne, Borthwick, & Packer, 2000; Chen & Hoshower, 2003; Krantz-Girod et al., 2004; Marsh, 2007b; Sheehan & DuPrey, 1999; Yunker & Yunker, 2003). Furthermore, it is unclear which variables influence student evaluations (Vargas, 2001), a state of affairs that perhaps colors the conversation regarding the reliability and validity of such evaluations as measures of teaching effectiveness. Wilson (1998) indicated that, as of the late 1990s, more than 2,000 articles had been published on the topic of student evaluation of teaching. Absent from the literature are studies investigating counseling students' evaluations of teaching effectiveness.
Prevalence and Utility of Teaching Evaluations
Using student surveys to evaluate university faculty teaching performance is common practice in the United States (Saroyan & Amundsen, 2001; Wolfer & Johnson, 2003; Wollert & West, 2000) and in other countries. Saroyan and Amundsen indicated that 94% of Canadian universities used student surveys alone or in combination with other methods for evaluating teaching performance. Although some research has indicated that universities in the United States have tried to incorporate additional methods of teacher evaluation such as self-evaluation, review of the course materials, submission of teaching portfolios, and direct class observation by fellow professors, student evaluation via opinion survey predominates (Wolfer & Johnson, 2003).
If the prevalence of student opinion surveys seems certain, the utility of their data, in isolation from other methods, to improve teaching effectiveness appears dubious. Marsh (2007a) conducted the only multilevel longitudinal study of the impact of teaching experience on student evaluations of teaching effectiveness. The results of this study showed that years of faculty members' teaching experience did not improve their student evaluation scores beyond the first 2 years of their teaching career. Moreover, faculty members who received feedback from student opinion surveys alone showed no improvement of scores on subsequent teaching evaluations. Although evaluations might identify those who need to improve in some way, Marsh (2007a, 2007b) suggested that more direct and purposeful intervention was needed to aid faculty members' development of teaching effectiveness. The exact role of student opinion surveys in the faculty evaluation process is still being refined (Marsh, 2007a, 2007b).
Ways in Which Students Perceive Teaching Effectiveness
Researchers have attempted to identify how students evaluate teaching effectiveness and which factors affect their ratings of this variable (Arbuckle & Williams, 2003; Chonko, Tanner, & Davis, 2002; Marsh, 2007b; Webster, 1990). …