Semantic Notation and Retrieval in Art and Architecture Image Collections
Stanchev, Peter L., Green, David Jr.,, Dimitrov, Boyan, Journal of Digital Information Management
Abstract: In this paper, we analyze various methods used for semantic annotation and search in a collection of art and architecture images. We discuss the Art and Architecture Thesaurus, WordNet, ULAN and Iconclass ontology. Systems for searching and retrieval art and architecture image collections are presented. We explore if the MPEG 7 descriptors are useful for art and architecture image annotations. For illustrations we use images from Antoni Gaudi architecture and Claude Monet paintings.
Categories and Subject Descriptors
H.2.8[Database Applications]: Image Databases ; H.3.1 [Content Analysis and Indexing]; H.5.1 [Multimedia Information System]; 1.4 [Image Processing and Computer Vision]
Semantic Notation, Architecture Images, MPEG 7 descriptors, ontology
Keywords: Image retreival, video retrieval, Semantic Notation, Architecture image collection, thesaurus, meta data
Recent advances in computing, communications, and storage technology have made multimedia data prevalent. Many museums offer their collections on the web. The rich content of multimedia data built through the synergies of the information contained in different modalities calls for innovative methods for modeling, processing, mining, organizing, and indexing these data. Content-based image retrieval and Content-based video retrieval are two research areas in multimedia systems that have been particularly popular in the last years. The MPEG-7 standard gives a set of descriptions that have been used to facilitate this research.
Over the past decades, many researchers from the image processing, computer vision and database communities have investigated possible ways of retrieving visual information based solely on its content. Instead of being manually annotated using keywords, images and video clips could be indexed by their own visual content, such as color, texture, and objects' shape and movement. Many research groups in leading universities, research institutes, and companies are actively working in this field. Their ultimate goal is to enable users to retrieve the desired image or video clip from massive amounts of visual data in a fast, efficient, semantically meaningful, friendly, and location-independent environment.
There is evidence that different image features work with different levels of effectiveness depending on the characteristics of the specific image data set. For example, color layout, like color structure, perform badly on monochrome images, while dominant color descriptor performs equally well on several of data sets.
Existing systems are limited by the fact that they can only operate at the primitive feature level, while users operate at a higher semantic level. This mismatch is often referred to as semantic gap. It is possible to increase the retrieval effectiveness by a proper choice of image features from the MPEG-7 standard .
There are many works devoted to the art and architecture image semantics. Koivunen and Swick  presented it from the prospectives of shared collaborations. Handschuh and Staab  provide manual and semi-automatic techniques for semantic annotation. Hyvonen et al  proposed ontology-based image retrieval. The word "ontology" originates from the Greek words ontos = "being" and logos = "knowledge", and means "knowledge of being". It was first used in the 17th century, from Christian Wolff, for the branch of metaphysics of existing. The most acceptable definition of ontology according to  is the Gruber  definition: "formal specification of a conceptualization", and is shared within a specific domain. When we ask of existing works of art or architecture we are asking about the ontology of artworks. Ontologies can be used for annotation and search in image collections. In this paper various ontologies used for art and architecture image collections such as AAT, WordNet, Iconclass and ULAN are reviewed in Section 2. …