An Empirical Approach to Automated Web Site Evaluation

Article excerpt

Abstract: The Web enables broad dissemination of information and services, yet most sites have in-adequate usability and accessibility. Numerous automated evaluation methodologies and tools have been developed to help designers to improves their sites. We describe the state-of-the-art in automated web site evaluation and then elaborate on our WebTango approach, which entails deriving design guidelines by mining empirical data. We compute over 157 measures, which assess many web interface aspects, and then use these measures along with expert ratings from Internet professionals as input to data mining algorithms. This mining process enables us to derive statistical models of highly rated web interfaces, such that the models reflect effective design patterns that are used on them. We then deploy the models so that designers can use them in the automated analysis of their sites.

Keywords : World Wide Web, WWW--Empirical Studies, Automated Usability Evaluation, Web Site Design

1 Introduction

The World Wide Web plays an important role in our society-enabling broader dissemina-tion of information and services than was previously available. However, many sites provide inadequate usability and are inaccessible to users with disabilities [Forrester Research, 1999; Jackson-Sanborn et al., 2002]. Much has been said about the way in which to design web sites so that they can be used and accessed by users who have a broad range of abilities, backgrounds, and skills, but building a high-quality site is a challenging task. Furthermore, it is a challenge that many non-professional or occasional designers need to undertake.

There are numerous approaches to address the current state of the Web:

* Design guidelines, patterns, and methods to help designers to structure their design pro-cesses and to inform their design decisions (e.g., [Comber,1995; Computer Science and Telecommunications Board, 1997; Detweiler and Omanson, 1996; Farkas and Farkas, 2000; Flanders and Willis, 1998; Fleming, 1998; Levine, 1996; Lyardet et al., 1999; Lynch and Horton, 1999; National Cancer Institute, 2001; Nielsen, 1998; Nielsen, 1999; Nielsen, 2000; Paciello and Paciello, 2000; Rosenfeld and Morville, 1998; Sano, 1996; Schriver, 1997; Shedroff, 2001; Shneiderman, 1997; Spool et al, 1999; Thatcher et al., 2002; Turns and Wagner, 2002; van Duyne et al., 2002; Williams, 2000; World Wide Web Consortium, 1999]).

* Web site templates to help designers to implement sites within HTML authoring environments.

* Form-based site creation services to build sites for designers (e.g., autowebmaker [autowebmaker, 2003] and easiwebmaker [Easiwebmaker, 2003] for e-commerce sites, Yahoo! GeoCities [Yahoo! Inc., 2003] and Terra Lycos Tripod [Lycos, Inc., 2003] for personal sites, and WebCT [WebCT, Inc., ] and Blackboard [Blackboard, Inc.,] for course sites).

* Automated tools to critique designers' web pages in the late design stages [ivory, 2003a; Ivory and Hearst, 2002b].

* Sketch-based tools to help professional designers to draw site maps, storyboards, and schematics in the early design stages [Lin and Landay, 2002; Lin et al., 1999; Newman and Landay, 2000].

* Automated tools to enable users to transform web pages to fit their needs [ivory et al., 2003].

* Each of these approaches contribute to helping designers to build better sites and to help-ing users to have better experiences on-line. In this article, we examine the contribution of automated web site evaluation tools and methodologies (fourth item above). Our research on the development, use, and role of automated evaluation tools has shown that they are important for helping designers to learn about effective design practices, to apply evaluation criteria consistently and broadly across entire sites, and to reduce the cost of non-automated evaluation methods like usability testing [ivory, 2001; Ivory, 2003a; Ivory and Hearst, 2001; Ivory and Hearst, 2002b; Ivory et al, 2003]. …