Evidence on Instructional Technology and Student Engagement in an Auditing Course

Article excerpt

ABSTRACT

The introduction of the instructional technology initiates the shift of the focus of learning environment from instructors to students. The availability of the online learning tools provides the chance for instructors to enhance the student engagement in the learning process. This paper provides evidence on the impact of instructional technology on student engagement in the learning process, measured by student activities on Blackboard Vista in a face-to-face auditing course. Further analysis indicates that improved course interactions are associated with the introduction of the new function of instructional technology. In addition, the students with high motivations are more likely to have better performance in the auditing course.

This paper contributes to existing literature and teaching practice. First, the research design captures the unique features of objective data with measurement of reality, which are rarely included in self-reported research methodology. Second, the study supports the view that the integration of instructional technology into face-to-face courses enhances student learning engagement and motivation, hence has the potential of improving student performance.

INTRODUCTION

Multiple platforms have made it possible to utilize web-based instructional tools to facilitate the interactions between faculty members and students between classes, and to enhance the student motivations and engagement in the learning process. The introduction of the instructional technology initiates the shift of the focus of the learning environment from instructors to students. While prior studies have examined the effect of student motivation and engagement on course performance (Vruwink & Otto, 1987; Elikai & Baker, 1988), little attention has been directed to this research question in the era of instructional technology. The purpose of this paper is to examine how instructional technology influences learner motivation and engagement, proxied by student activities on the Blackboard Vista course site.

It is evident that a number of technological changes have irreversibly changed the landscape of the traditional classroom setting of higher education. Most previous research generally provides positive results regarding the influence of information technology. Prior studies in accounting education employ the research methodology of surveys and questionnaires, which are subject to the limitations of self selection and biased inference of the results. To fully address the research question, this paper utilizes research methodology using objective data. This paper employs student tracking statistics at Blackboard Vista and provides an analysis on the role of instructional technology in a face-to-face auditing course.

LITERATURE REVIEW AND HYPOTHESIS DEVELOPMENT

The use of various instructional technologies in higher education has been on the rise at an accelerated pace. Despite the prominence of the impact of technology on the traditional classroom, limited evidence is available on the incremental contribution of these technologies to the in-class student learning environment. Previous research has focused on the influence of instructional technology on distance education, when face-to-face meetings are not required. While the quality assurance of online classes and the trade-off between quality and convenience are constantly called into question (e.g., Ryan, 2000), the merits of the use of instructional technology in college education should not be diminished.

There has been constant debate on the effectiveness of the delivery of management courses in the format of distance education. Leidner and Jarvenpaa (1995) provide a theoretical view on the influence of information technology on management school education. They point out that advanced technology would facilitate the display and access of information, thus increase the process of "sharing and construction" of knowledge (Leidner & Jarvenpaa, 1995).The evaluation results of e-learning via WebCT by Halawi, McCarthy, & Pires (2009) also indicate that e-learning is an effective instructional tool for MIS courses.

The study by De Lange et al. (2003) examines the student evaluations of Virtual Learning Environment (VLE) products in higher education in an attempt to determine whether such tools produce benefits with regards to student learning outcomes. Their survey concludes that accounting undergraduate students show a higher level of satisfaction after using such tools as lecture notes, bulletin boards, and online assessments. Their findings further indicate that overall student learning motivation and engagement is enhanced by the features generated from the implementation of instructional technology (De Lange, Suwardy, & Mavondo, 2003). Empirical research within accounting education is limited with regards to the effectiveness of technology applied to in-class courses. McVay et al. (2008) employ a survey instrument and find that classroom configuration and information technology have positive effects on student learning experience. Prior studies also find that most accounting students do not have difficulty in adapting to the learning environment aided by instructional technology (e.g., Basile & D'Aquila, 2002).

Another stream of literature on accounting education focused on the association between student engagement and learning. The theory of learning developed by Norman (1981) suggests that pedagogical techniques with a feedback system are associated with improved learning outcomes (Norman, 1981). Vruwink and Otto (1987) report that accounting instructors have observed that homework assignments and quizzes assist in motivating students in the learning process. Based on this learning theory, Elikai and Baker (1988) provide empirical evidence that quizzes associated with rewards can be used to improve student performance. I develop hypothesis 1 as follows:

Hypothesis 1: The availability of instructional technology functions is associated with increased student engagement in the learning process.

In a learner-centered environment, Chang and Smith (2008) examine course-related interactions in the context of long distance education. Using data collected from a survey, they find that a high student satisfaction level is associated with all three interactions, namely studentinstructor, student-student, and student-content interactions, in a computer science course (Chang & Smith, 2008). The research by Gagne and Shepherd (2001) also shows that primary communications between the instructor and the students are emails for both online and in-class students. I predict that the introduction of a new instructional tool would have an effect on course interactions.

Hypothesis 2: Increased usage of information technology is associated with increased course-related interactions.

Lammers, Kiesler, & Curren (2005) examine the impact of student effort on performance in college coursework. They find that students expect to study hard to obtain satisfactory grades. Consequently, it is expected that the students with higher grades are more likely to make the best out of the instructional technology available to them. On the other hand, students with low grades may be neutral in regards to exposure to enhanced instructional technology. I develop the following hypothesis:

Hypothesis 3: Increased student engagement is more likely to be associated with students with higher grades.

BACKGROUND

Auditing is a required course for students in the Bachelor of Science in Accounting at a regional campus of a public university. This course is also an elective course for students in Bachelor of Science in Management with a concentration in accounting. This campus was a commuter campus until August 2005, when student housing became available. The student body is a mixture of traditional students and non-traditional students, who work full time and attend college part time. To enroll in the Auditing course, students are required to have a grade of C or better in Intermediate Accounting II. Some students with extraordinary performance in Intermediate Accounting I may gain administrative permission to enroll in the course. Those students generally take Auditing and Intermediate Accounting II in the same semester.

This Auditing course aims at giving the student an understanding of the philosophy and environment of the auditing profession. The course highlights the nature and economic purpose of the auditing profession, auditing standards, professional conduct, legal liability, audit evidence, audit planning, internal control, and audit working papers. One section of the course was offered in spring 2009 and two sections were offered in fall 2009. All students who completed Auditing during the spring and fall semesters of 2009 are the subjects of the study. The same instructor taught all classes and collected the data. The students used the same textbook (Whittington & Pany), from which nine chapters were included in the lecture. The description of the chapters covered in the course is detailed in Table 1 . Although different editions were used for the two semesters (the 16th edition was used for the spring semester and the 17th edition was used for the fall semester), the quizzes and exams were very similar except in regards to the delivery method.

In spring 2009, the instructor used paper based quizzes to monitor student attendance and performance. The use of the course site on Blackboard Vista was limited to the functions of course material and email. The instructor posted course materials including chapter outlines, lecture slides, and solutions to the in-class exercises for students' review. Blackboard emails remained the primary communication channel between the instructor and the students. The instructor sent a group email to the students after each class, summarizing the material covered and the plan for the next class. Group emails were also used to remind the students of important due dates or to clarify questions raised by the students.

In addition to the above routine practices, the instructor introduced an application of instructional technology in fall 2009. Using this new application, the instructor prepared a self assessment quiz for each chapter, which could only be accessed on Blackboard Vista. Each self assessment quiz consisted of various numbers of objective questions. The students were allowed multiple attempts at taking the quizzes and they were also able to view the correct answers to the questions after each attempt. The students were not required, but were encouraged to take the online quizzes. In other words, the student usage of the online assessment tools was strictly voluntary and the student performance on the online quizzes was not counted towards the course grade. As a follow-up measure, the instructor prepared in-class quizzes, which incorporated some of the objective questions from self-assessment quizzes and one essay question from the lectures. Twelve quizzes were given throughout the semester, including nine quizzes on the chapters, two evaluation quizzes, and one take-home quiz. The four lowest grades were dropped at the end of the semester.

RESEARCH METHODOLOGY

The study is based on the instructor's experience of using an instructional tool on Blackboard Vista. The instructional tool was made available for the voluntary use of the students. Based on the comparison of student activities in two semesters with and without the utilization of the instructional tool, the paper examines changes of student activities and courserelated interactions. The study also investigates the association between student activities on Blackboard Vista and grades.

The data on student activity were collected from the tracking feature at Blackboard Vista. Table 2 presents the variable definitions provided by the Blackboard Vista administration. The dataset also contains student performance data from the Blackboard grade book.

Table 3 presents the descriptive statistics for student activities on Blackboard Vista. As illustrated, there were 3 1 subjects and 44 subjects in the spring and fall semesters, respectively. The statistics for variables related to the assessment tool (assessment began, assessment finished, and time for assessment) are only available for fall 2009. The normality tests of the variables (skewness and kurtoisis) show that the variables are not normally distributed.

The Kruskal- Wallis test is a nonparametric test and this test is normally used when the assumption of normal distribution is violated. As discussed previously, the variables included in the dataset are not normally distributed, thus the Kruskal-Wallis test is appropriate. The null hypothesis of the test assumes that the groups are from identical populations. Rejection of the null hypothesis indicates that the variables are not statistically similar in the groups. The tables of mean comparison are prepared using statistics derived from the Kruskal-Wallis tests. The tables also illustrate the mean ranks for each variable, and the corresponding Chi Square of the mean comparison across the groups.

The hypotheses are tested by comparing the means of the various groups. In testing Hypothesis 1 on the association between usage and the availability of additional technical application, I perform a mean comparison between the spring and fall semesters. Table 4 shows that variables of the number of sessions, the time spent on the sessions, content folders viewed, and files viewed are significantly different across the two semesters. Further examination of the descriptive statistics of Table 3 reveals that the means of the above variables of the fall semester exceed those of the spring semester. Empirical results support Hypothesis 1 , illustrating that increased student activities are associated with the availability of the new function on Blackboard.

Similarly, Table 4 indicates that the interactions within the course are also different in the two semesters. Consequently, Hypothesis 2 holds because the means of the variables "number of mail read" and "number of mail sent" in fall 2009 (mean of number of mail read =111; mean of number of mail sent = 13.26) have increased from those of spring 2009 (mean of number of mail read = 54.35; mean of number of mail sent = 8.26). The results suggest that improvement in the interactions is related to the introduction of the online assessment tool.

The study also examines the relationship between enhanced instructional technology and student grades. As indicated in Table 5, students with grades A and B have different levels of activity across the two semesters. Students with a grade of C are similar in all activities on Blackboard. It is suggested that the existence of the online assessment instrument does not change the behavior of the students with a grade of C. On the other hand, the students with grades A and B are motivated to participate in the voluntary-based online assessment activities. The empirical results support Hypothesis 3, indicating that increased usage of information technology is more likely to be associated with students with higher grades.

DISCUSSION

This study investigates the effect of instructional technology on student activities on Blackboard. The empirical results show that, first and foremost, the application of instructional technology is a prominent factor associated with increased student engagement in the learning process. The results also indicate that the enhanced interactions are also associated with the new function of instructional technology.

The findings provide insight on how instructional technology impacts the students in various grade categories. Relative to the students with a grade of C, students with grades A and B are more likely to use the instructional tools efficiently and effectively. Interestingly, when comparing students with grades A and B separately, the student interaction in terms of "mail sent" remains statistically similar in the two semesters (Table 5). I also perform the KruskalWallis test on separate dataseis to examine the differences between various grades in the two semesters. The results show that the student activities are insignificantly different across the grade categories in spring 2009 (untabulated). Instead, Table 6 reveals that in fall 2009, the students in different grade categories can be differentiated by major. As illustrated, the variable time of sessions is significantly different across the grade categories. It is implied that the better performers did not necessarily log in the course site more, but they spent more time to gather information they needed. Also, the better performers tended to have more assessment sessions but completed each session more quickly.

The analysis indicates that better performance is associated with students who are willing to take advantage of advances in instructional technology to improve their grades. The relationship between the improved student performance and student engagement sheds light on the practice to motivate the students to be involved in the learning process. While face-to-face courses are traditionally designed to be instructor-centered in the classroom, the development in instructional technology provides opportunities for students to be an active component in achieving their own learning objectives.

SUMMARY

Developments in instructional technology provide new opportunities for improving teaching and learning in accounting and auditing courses. Web-based instructional tools give students more opportunities to interact with instructors and to engage and motivate themselves in the learning process. Similar to other education papers, the application of the results to other learning environments should be exercised with caution. This paper nevertheless provides a starting point to investigate how instructors may effectively integrate instructional technology into conventional classroom courses. The incorporation of web-based instructional tools in teaching accounting and auditing courses will continue to grow. The continuing challenge to instructors will be learning how to use technology, together with other teaching strategies, to further motivate and engage students in the learning process.

This paper contributes to existing literature and teaching practice. First, the research design captures the unique features of objective data with measurement of reality, which are rarely included in self-reported research methodology. Second, the study supports the view that the integration of instructional technology into face-to-face courses enhances the student learning engagement and motivation, hence has the potential of improving student performance.

[Reference]

REFERENCES

Basile, A. & J. M. D'Aquila (2002). An Experimental Analysis of Computer-Mediated Instruction and Student Attitude in a Principles of Financial Accounting Course. Journal of Education for Business, 11 (3), 137143.

De Lange, P., T. Suwardy & F. Mavondo (2003). Integrating a Virtual Learning Environment into an Introductory Accounting Course: Determinants of Student Motivation. Accounting Education, 12(1), 1-14.

Chang, S. H. & R. A. Smith (2008). Effectiveness of Personal Interaction in a Learner-Centered Paradigm Distance Education Class Based on Student Satisfaction. Journal of Research on Technology in Education, 40 (4), 407-426.

Elikai, F. & J. Baker (1988). Empirical Evidence on the Effectiveness of Quizzes as a Motivational Technique. Issues in Accounting Education, 3(2), 248-254.

Gagne, M. & Shepherd, M (2001). A Comparison Between a Distance and a Traditional Graduate Accounting Class. Transforming Education Through Technology Journal, April, 58-64.

Halawi, L., R. V. McCarthy & S. Pires (2009). An Evaluation of ?-Learning on the Basis of Bloom's Taxonomy: An Explanatory Study. Journal of Education for Business, July/ August, 374-380.

Lammers, H. B., T. Kiesler, M. T. Curren, D. Cours & B. Connett (2005). How Hard Do I Have to Work? Student and Faculty Expectations Regarding University Work. Journal of Education for Business, March/ April, 210-213.

Leidner, D. & S. L. Jarvenpaa (1995). The Use of Information Technology to Enhance Management School Education: A Theoretical View. MS Quarterly, September, 265- 291.

McVay, G. J., P. R. Murphy & S. W. Yoon (2008). Good Practices in Accounting Education: Classroom Configuration and Technological Tools for Enhancing the Learning Environment. Accounting Education: an international journal, 17 (1), 41-63.

Norman, D. A. (1981). What is Cognitive Science? In D. A. Norman (Ed.), Perspectives on Cognitive Science (p. 40-50), Ablex Publishing Co..

Palocsay, S. W. & S. P. Stevens (2008). A Study of the Effectiveness of Web-Based Homework in Teaching Undergraduate Business Statistics. Decision Sciences Journal of Innovative Education, 6 (2), 213- 232.

Ryan, R. (2000). Student Assessment Comparison of Lecture and Online Construction Equipment and Methods Classes. Transforming Education Through Technology Journal, 21 (6), 78-83.

Vruwink, D. R & J. R. Otto (1987). Evaluation of Teaching Techniques for Introductory Accounting Courses. The Accounting Review, LXII (2), 402-408.

[Author Affiliation]

Songtao Mo, Purdue University Calumet

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.