Academic journal article Journal of Digital Information Management

Study of Effectiveness of Implicit Indicators and Their Optimal Combination for Accurate Inference of Users Interests

Academic journal article Journal of Digital Information Management

Study of Effectiveness of Implicit Indicators and Their Optimal Combination for Accurate Inference of Users Interests

Article excerpt

ABSTRACT: Retrieval and filtering systems may apply relevance feedback to gain information on users' needs in order to improve their ad-hoc queries or long term profiling. Explicit relevance involves explicit ratings of documents or terms by the users and disrupts their normal patterns of browsing and searching. The alternative non-disruptive method is implicit feedback inferring users' needs and interests by monitoring their regular interaction with the system. Some implicit indicators of interest, such as reading time, have been investigated in previous studies and were found indicative to the relevance of documents but not sufficiently accurate to substitute explicit ratings. In this paper we present several new relative implicit feedback indicators and examine their effectiveness as well as the effect of combining several implicit indicators. The paper describes a large-scale user study on which users' searches were observed by a specially developed browser that recorded their behavior (implicit indicators) as well as their explicit ratings. The relationship between implicit indicators and explicit ratings was analyzed and found that a certain combination of implicit indicators achieved higher correlation with the explicit ratings than any of the individual indicators. We have also found that the newly suggested relative indicators are more indicative to the level of interest of the user in an information item than the non-relative indicators.

Categories and Subject Descriptors

H.3.3 [Information Search and Retrieval]: Relevance feedback; H.3.4 [Systems and Software]: User profile and alert services

General Terms

User profiling, Query formulation

Keywords: Relevance feedback, Implicit indicators, User studies, User Modeling

1. Introduction

Relevance feedback (RF) is typically used to acquire information about users for obtaining their accurate information needs. RF is used for building or updating user models in information filtering or recommender systems, and for expanding or modifying ad-hoc queries (Salton and Buckley, 1990). Explicit relevance feedback requires the user to explicitly provide feedback by marking or rating documents and terms for their relevancy. One main problem with this traditional feedback technique is the disruption to the regular work of users. To provide explicit ratings users are required to change their normal browsing and searching patterns and to perform additional activities (e.g., a few more mouse clicks). Since the benefit of providing feedback is not easily perceived, many users tend not to provide evaluations, thus resulting in lower effectiveness of the systems that rely on these evaluations (Shapira et. al., 2001).

Implicit relevance feedback techniques obtain information on users by monitoring their natural interaction with the system without interruption. Every interaction is recorded to infer users' interests and preferences. Analysis of users' behavior consists of examination of relevance indicators such as reading and scrolling time and activities such as printing, bookmarking, etc.

Despite the clear advantage of implicit feedback due to the removal of cost to the user, implicit feedback is still known to be less accurate than explicit rating in predicting users' interests (Nichols 1997). Although there is a growing body of studies related to implicit feedback, the optimal set of implicit indicators and their relative importance for an accurate derivation of a user profile has not yet been determined (Kelly and Teevan, 2003, White et. al., 2002, Clyapool et. al., 2001 and others). From the vast literature on implicit feedback one can conclude that not all implicit measures are equally effective and it might very well be that some are effective only when combined with others (Kelly and Teevan, 2003). In this paper we investigate the effectiveness of implicit indicators. We describe a user study that examine the usefulness of known and newly suggested implicit indicators and determine the optimal combination of these indicators for an accurate inference of users' interest. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.