Combining Formal and Exploratory Methods for Evaluation of an Exploratory Geovisualization Application in a Low-Cost Usability Experiment
Demsar, Urska, Cartography and Geographic Information Science
The importance of geovisualization for exploration and analysis of spatial data has been widely recognized (Andrienko and Andrienko 2006), and the potential and limitations of information visualization tools have been explored in numerous controlled experiments (Plaisant 2004). Yet there exists certain skepticism towards visualization techniques among data analysts. One of the reasons for this is that visual data exploration is essentially a very complex process, and at present not much is known about how visual tools actually support humans in data exploration (Andrienko and Andrienko 2006). While usability testing in controlled conditions using the principles of human-computer interaction remains the main approach to evaluating visualization tools (Plaisant 2004), such experiments are not always sufficient in the domain of geovisualization, due to their exploratory and interactive nature (Andrienko and Andrienko 2006). One way to approach this issue is to combine formal and exploratory usability evaluation methods in one experiment to assess how data exploration is performed with geovisualization tools. This paper presents an experiment that combines both approaches. Besides investigating how users explore a spatial data set with a visual data-mining system, the study also introduces a low-cost methodology for usability evaluation, developed after discount usability principles by Nielsen (1994).
Usability is the extent to which a computer system supports users to achieve specified goals and does so effectively, efficiently, and in a satisfactory way (Ivory and Hearst 2001). In human-computer interaction, which investigates interaction between human users and information systems (Preece et al. 2002), usability forms a small part of the larger issue of system acceptability. System acceptability is the answer to the question of whether the system is good enough to satisfy all the needs and requirements of the users. The term combines social and practical acceptability, and the state of acceptability is further subdivided into several categories, such as usefulness, cost, reliability, and compatibility with existing systems.
Usefulness denotes whether the system can be used to perform some defined task in order to achieve some desired goal. It is divided into utility and usability. Utility describes whether system functionality can do what is needed for the defined task, whereas usability refers to how well the users can use the functionality. The outline of the system acceptability model is presented in Figure 1 (Nielsen 1993).
[FIGURE 1 OMITTED]
Usability evaluation is the process of systematically collecting data on how a particular user or a group of users uses the system for a particular task in a particular environment. There are four main methodologies for performing the usability assessment: "quick and dirty" evaluation, usability testing, field studies, and predictive evaluation (Preece et al. 2002).
In a "quick and dirty" evaluation the designers receive informal feedback from users at any stage of the design process. Usability testing includes methods that measure the performance of the users and their experience with the tool. Field studies are conducted in natural settings with the aim of understanding how technology impacts the users in their daily routines, using qualitative techniques, such as interviews, observation, participant observation and ethnography. Predictive evaluation predicts usability problems by asking the experts to apply their knowledge of typical users to a particular usability issue, while the users do not have to be present. The most commonly used method for this purpose is heuristic evaluation (Hackos and Redish 1998; Preece et al. 2002).
The experiment performed in this study attempts to combine formal evaluation methods with exploratory usability methods. Formal evaluation methods are tests that measure the performance of the users on carefully prepared tasks. …