The Impact of Web Search Engines on Subject Searching in OPAC
Yu, Holly, Young, Margo, Information Technology and Libraries
This paper analyzes the results of transaction logs at California State University, Los Angeles (CSULA) and studies the effects of implementing a Web-based OPAC along with interface changes. The authors find that user success in subject searching remains problematic. A major increase in the frequency of searches that would have been more successful in resources other than the library catalog is noted over the time period 2000-2002. The authors attribute this increase to the prevalence of Web search engines and suggest that metasearching, relevance-ranked results, and relevance feedback ("more like this") are now expected in user searching and should be integrated into online catalogs as search options.
In spite of many studies and articles on Online Public Access Catalogs (OPAC) over the last twenty-five years, many of the original ideas about improving user success in searching library catalog have yet to be implemented. Ironically, many of these techniques are now found in Web search engines. The popularity of the Web appears to have influenced users' mental models and thus their expectations and behavior when using a Web-based OPAC interface. This study examines current search behavior using transaction-log analysis (TLA) of subject searches when zero-hits are retrieved. It considers some of the features of Web search engines and online bookstores and suggests future enhancements for OPACs.
* Literature Review
Many studies have been published since the 1980s centering on the OPAC. Seymour and Large and Beheshti provide in-depth overviews on OPAC research from the mid-1980s through the mid-1990s. (1) Much of this research has addressed system design and user behavior including:
* user demographics,
* search behavior,
* knowledge of system,
* knowledge of subject matter,
* library settings,
* search strategies, and
* OPAC systems (2)
OPAC research has employed a number of data-collection methodologies: experiment, interviews, questionnaires, observation, think aloud, and transaction logs. (3) Transaction logs have been used extensively to study the use of OPACs, and library literature reflects this. While the exact details of TLA vary greatly, Peters et al. define it simply as "the study of electronically recorded interactions between online information retrieval systems and the persons who search for the information found in those systems." (4) This section reviews the TLA literature relevant to the study.
* Number of Hits
TLA cannot portray user intention or actual satisfaction since relevance, success, or failure are subjectively determined and require the user to decide. Peters recommends combining TLA with another technique such as observation, questionnaire or survey, interview, or focus group. (5) In spite of the limitations of TLA, many studies (including this one) rely on it alone. Typically, these studies define failure as zero hits in response to a search. Generalizing from several studies, approximately 30 percent of all searches result in zero hits. (6) The failure rate is even higher for subject searches: Peters reported that about 40 percent of subject searches failed by retrieving zero hits. (7)
Some researchers also define an upper number of results for a successful search. Buckland found that the average retrieval set was 98. (8) Blecic reported that Cochrane and Markey found that OPAC users retrieve too much (15 percent of the time). (9) Wiberly, Daugherty, and Danowski (as reported in Peters) found that the median number of postings considered to be too many was fifteen, although when fifteen to thirty postings were retrieved, more users displayed them all than abandoned the search. (10)
* Subject Searching
Some studies have specifically looked at subject searching. Hildreth differentiated among various types of searches and defined one hundred items as the upper limit for keyword searches and ninety as the upper limit for subject searches. …