Academic journal article Educational Technology & Society

Visualizing Topic Flow in Students' Essays

Academic journal article Educational Technology & Society

Visualizing Topic Flow in Students' Essays

Article excerpt

Introduction

Writing is an important learning activity common at all educational levels and disciplines. But incorporating writing activities into the curricula faces many challenges, including the cost of providing meaningful and timely feedback as the one provided in assessment. Technically researchers are tackling these challenges by producing automated feedback (including grades) directly targeted to the students or support for human assessors who then write the feedback. In this line of research studies have shown tools and techniques for automated feedback in academic writing (Beals, 1998; Graesser & McNamara, in press; Thiesmeyer & Thiesmeyer, 1990; Wade-Stein & Kintsch, 2004; Wiemer-Hastings & Graesser, 2000). Feedback can be genre specific, as for example, in argumentative writing. Many studies in this area have focused on argument visualization (Kirschner, Shum, & Carr, 2003) where the students are visually presented with the way in which claims are interrelated, showing evidential structures. Other forms of feedback focus on quality measures that apply to multiple genres and disciplines. For example, automatically generated visualizations can be used as support material that students can use to reflect on a set of trigger questions designed around issues with which students normally have difficulty (Calvo & Ellis, 2010). Thanks to new cloud computing technologies these computationally intensive forms of support can be provided in real-time, at any stage of the writing process and to large numbers of students (Calvo, O'Rourke, Jones, Yacef, & Reimann, 2011).

Many different features of a document can be quantified and therefore represented visually as a form of feedback or support to handling writing activities. The challenge is to find visual representations of features that are both meaningful to actual writing tasks (e.g. putting together evidence into an argument) rather than mathematical artifacts, and that do actually provide useful information that correlates with the quality of writing. Several linguistic features of quality writing have been identified with evidence that they can predict high and low proficiency essays. For example, McNamara (2010) provided evidence for syntactic complexity (measured by the number of words before the main verb), lexical diversity, and word frequency (as measured by Celex, logarithm of all words). While other measures, such as cohesion indices, have received much attention in the literature, they generally offer more of a guide to a text's connectedness, which does not necessarily correlate well to that of experts (Crossley & McNamara, 2010). The way an argument is structured and the flow in a composition are important quality features. We evaluate here different techniques for producing topic flow visualizations. These were proposed by O'Rourke and Calvo (2009b) as a way of helping students reflect about their writing, particularly issues related to the flow in the composition. The visualization techniques include several steps, all of which must be taken into account if the actual visual representation (the final outcome) is to be semantically valid and useful.

In most Natural Language Processing techniques, each text segment is converted into a high dimensional representation using the Vector Space Model. This high dimensional space is then reduced by using one of several mathematical techniques that preserve as much of the original information as possible. The first challenge is to find the optimum dimensionality reduction technique that produces meaningful visualization. The optimum value is the measure of how it relates to a writing quality attribute (e.g. flow). This reduced space can then be used to produce a 2-dimensional space that can be made into a visual representation. In our topic flow visualizations, textual units (e.g. paragraphs) are represented in a two dimensional space, with their distances being proportional to the distance between topics (in the reduced dimensionality space). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.