Magazine article AI Magazine

Report on Representations for Multimodal Generation Workshop

Magazine article AI Magazine

Report on Representations for Multimodal Generation Workshop

Article excerpt

The Representations for Multimodal Generation Workshop was held on April 23-25, 2005, at Reykjavík University, Reykjavík, Iceland. The overall goal of this workshop is to further the state of research on multimodal generation by enabling (and getting) people in the field to work together on building systems capable of real-time face-to-face dialog with people. This report summarizes the activities and progress of that meeting.

Human communicative behaviors span a broad set of skills, from natural language generation and production to eye gaze control and facial expression. People produce multimodal behavior with ease in real time in a broad range of circumstances. In April 2005 a group of practitioners in the area of multimodal communication and computer graphics came together for three days at Reykjavik University to further the integration and development of multimodal generation skills for artificial humans.

The group started by specifying the necessary planning stages and knowledge structures for multimodal behavior generation and control. While we felt it was important to form clear interfaces at separate levels of abstraction in order to modularize the system, we also did not want to force a particular microarchitecture on anyone. The solution was to identify abstraction levels at the macro level and treat the processing between them as black boxes or (more appropriately) as open research questions. This resulted in a focus on specifying the particular type and form of information that needs to be represented in the gaps between these levels.

We came up with what we call the SAIBA framework (situation, agent, intention, behavior, animation), according to which macroscale multimodal generation comprises three processing stages, or levels: (1) Intent planning, (2) behavior planning, and (3) behavior realization. The interface between intent planning (1) and behavior planning (2) describes communicative intent without any reference to physical behavior; the interface between behavior planning (2) and behavior realization (3) describes the physical behavior that carries out that communicative intent. The group analyzed how existing representation languages, APML, MURML, RRL, and older BML/FML, contribute to these interfaces and drafted new functional markup language (FML) and behavioral markup language (BML) specifications.

The group feels that a high degree of flexibility is achieved by separating the planning into these three levels in this particular way, and SAIBA is now being used as the foundation for further work by several of the attendees. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.