To Merge and Not to Merge: Israel's Union List of Monographs in the Context of Merging Algorithms

Article excerpt

ALEPH (Automated Library Expandable Program Hebrew University), Israel's research library network, has always been a system that reflected the needs and limitations of the country it was designed to serve: "Like Israel itself, a country always long on problems and short on financial resources, both the designers of ALEPH and the decision-makers governing the network structure have been influenced every step of the way by the constraints of stark pragmatism.... The approach was, necessarily, better to get it done than to get it perfect."(1)

This highly pragmatic approach, which led to a decentralized network consisting of nearly thirty separate files of records for individual university libraries and special projects, has been described in detail in an earlier article on ALEPH.(2) The current article presents the equally pragmatic approach that led to a solution, albeit partial and imperfect, of the problem of searching in these separate files. Like ALEPH itself, the solution is unique: a union catalog that was never intended to replace the individual records it merges, but rather to serve as a supplementary search tool. ALEPH's Union List of Monographs (ULM) is, in fact, a whimsical misnomer, neither a union list nor limited to monographs. What it is, in effect, is a union index for locating bibliographic items by author or title, in order to ascertain where they can be found without searching each library's file separately or choosing libraries by guesswork. Once a particular bibliographic item is located by one of these fields, the ULM retrieves a short cataloging record, with a list of libraries holding the item and pointers to the full cataloging record in each library's file. The user can then retrieve the full record by inputting the pointer associated with a particular library, in the form of that library's code and the system number of the record.

The title of the present paper, therefore, derives from the curious situation that engendered the ULM. In seeking to make the decentralized system, which lacked the union catalog most American bibliographic networks offer, more useful, it was decided to attempt, in the words of Monty Python, "something completely different." An algorithm was sought that would merge matching records into a short record listing the library code and system number of each record merged--cheaply, automatically, and reasonably accurately--without replacing the original records. What emerged from this merging-while-not-merging was a location device that, at least as far as this author was able to determine from the literature, is unique.

* The Literature of Merging

The professional literature on merging online records explores four issues: (1) the goals of merging, (2) problems and solutions in merging records from diverse cataloging systems using disparate software, (3) selection and testing of merging algorithms, and (4) problems involved in matching and merging records.

The Goals of Merging

Stephen Toney, in a 1992 article on deduplication of an international bibliographic database, enumerates the three basic options available for merging records determined to be duplicates: 1. One record is chosen as the master record and the others are deleted. 2. All records are kept but clustered with a master record. 3. One record is chosen as the master record and variant fields from the duplicates are added to the master.(3)

Toney stresses that different techniques work for different databases and that the need for testing in designing a merging project cannot be overemphasized. Furthermore, testing must be an iterative process, with tests made on samples of records. The method is modified based on the results of each test, and the cycle repeated until an acceptable level of success is achieved. The sensible rule he suggests in testing fields for inclusion in the match keys is to try the simplest approach until it proves not to work. …