The Indexed Web: Engineering Tools for Cataloging, Storing and Delivering Web-Based Documents

Article excerpt

My associate Gerald Edgar and I have spent countless hours downing innumerable Seattle espressos during lengthy discussions over the issues of Knowledge Management, Meta Data Insertion Tools, and the use of Controlled Vocabularies in the assignation of human intelligence to web resources.

Since the spring of 1996, we had been looking for a small-scale, subject specific venue to apply our ideas about Java-based, medium-neutral, and platform-independent tools and methodologies to the organization of information in the web environment. We sought to provide a means of assigning subjective human intelligence to that organization, and a vehicle to store and disseminate such information in compliance with emerging data definition and exchange standards such as XML (eXtensible Markup Language) and RDF (Resource Definition Format).

In June of 1997, the Law Library Resource Xchange (LLRX) approached us with a request to install a keyword search engine on their web site ( We could easily provide such a service (and did in fact). We also spoke about the viability of creating tools that would enable the use of controlled vocabularies when subject indexing their web resources, and cataloging software and search interfaces. These tools would allow users to browse subject headings and authors while searching for relevant documents.

This article describes the development of Java-based software to use controlled vocabularies and subject headings in the cataloging of web-based articles on a subject-specific legal resources web site. The goals of that project, the tools that came out of it, and the next steps in the development process are described here.

First Principles:

The Internet is not self-organizing; Internet users are.

* Solutions to the problems of organizing, storing, and disseminating Internet resources include subjective human analysis of information, not merely objective technological intervention in the form of filters and agents.

* Librarians will re-assert their role in engineering the tools and methodologies enabling the organization and dissemination of information in this Internet-enabled world.

* Solutions based on principles rather than transient technologies succeed.

Project Profile

The Law Library Resource Xchange is an Internet resource for the legal and law library community. LLRX was founded by two law librarians, Sabrina Pacifici (Washington, DQ and Cindy Chick (Los Angeles, CA). It is a valuable compendium of articles and columns contributed by authors from the legal information community. It is updated twice monthly and includes a library of published articles and columns. The success of LLRX and the growth of that library of articles prompted the need for organization beyond web navigation and some sort of search capability.

Pacifici approached us in June 1997 to consider installing and configuring a keyword search for the LLRX web site. LLRX originally asked for a simple relevance ranking search tool. While we might easily install and configure a keyword search engine, this didn't sound particularly interesting. However, it did sound like an opportunity for a practical application of some of the first principles noted above, namely, the use of human intelligence in the analysis and organization of web resources using controlled vocabularies and subject headings and creating a searchable catalog of the subsequent records. It also presented an opportunity to test Java as the programming language of choice in the development of unified interactive software. We offered to go beyond her request and create the tools to subject index and catalog targeted resources, with the down stream goal of creating an organizational, storage, and dissemination environment. The small scale of the site (200+ articles), as well as the subject specificity afforded a good test bed for our ideas. LLRX agreed to our proposal, so we began in earnest to outline the goals, methods, and structure of the project. …