Magazine article AI Magazine

Report on Representations for Multimodal Generation Workshop

Magazine article AI Magazine

Report on Representations for Multimodal Generation Workshop

Article excerpt

Human communicative behaviors span a broad set of skills, from natural language generation and production to eye gaze control and facial expression. People produce multimodal behavior with ease in real time in a broad range of circumstances. In April 2005 a group of practitioners in the area of multimodal communication and computer graphics came together for three days at Reykjavik University to further the integration and development of multimodal generation skills for artificial humans.

The group started by specifying the necessary planning stages and knowledge structures for multimodal behavior generation and control. While we felt it was important to form clear interfaces at separate levels of abstraction in order to modularize the system, we also did not want to force a particular microarchitecture on anyone. The solution was to identify abstraction levels at the macro level and treat the processing between them as black boxes or (more appropriately) as open research questions. This resulted in a focus on specifying the particular type and form of information that needs to be represented in the gaps between these levels.

We came up with what we call the SAIBA framework (situation, agent, intention, behavior, animation), according to which macroscale multimodal generation comprises three processing stages, or levels: (1) Intent planning, (2) behavior planning, and (3) behavior realization. The interface between intent planning (1) and behavior planning (2) describes communicative intent without any reference to physical behavior; the interface between behavior planning (2) and behavior realization (3) describes the physical behavior that carries out that communicative intent. The group analyzed how existing representation languages, APML, MURML, RRL, and older BML/FML, contribute to these interfaces and drafted new functional markup language (FML) and behavioral markup language (BML) specifications.

The group feels that a high degree of flexibility is achieved by separating the planning into these three levels in this particular way, and SAIBA is now being used as the foundation for further work by several of the attendees. Developing a common representation of the content to be exchanged by each planning level, without prescribing how the processing that produces these is done, opens the possibility of combining solutions from different researchers without too much code modification. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.