Magazine article AI Magazine

Editorial Introduction to the Special Articles in the Winter Issue: Turn-Taking and Coordination in Human-Machine Interaction

Magazine article AI Magazine

Editorial Introduction to the Special Articles in the Winter Issue: Turn-Taking and Coordination in Human-Machine Interaction

Article excerpt

Along-standing pursuit in the field of artificial intelligence has been to enable humans and computers to collaborate seamlessly on shared tasks through natural interaction. Autonomous or semiautonomous systems with collaborative competency enable a myriad of current and future applications from virtual personal assistants embedded in smartphones to robots assisting surgeons in the operating room.

At the core of any collaborative process lies the ability to coordinate individual contributions to the process. Humans involved in a collaborative activity continuously coordinate with each other to establish and maintain mutual understanding, called grounding, and to coconstruct solutions for their shared tasks. Such coordination happens along multiple dimensions. Due in part to the inherent structure of tasks and in part to cognitive and resource limitations of the interaction channel, the actions produced by collaborators are temporally coordinated. The relative order and timing with which collaborators perform actions is critical for the success of the collaboration. Beyond the temporal domain, participants must coordinate with each other to establish and maintain a shared cognitive basis for future actions and future understanding.

A quintessential example of fine-grained temporal coordination in human-human interactions is the process by which we take turns to speak in a group conversation. This conversational turn-taking process is largely unconscious, as we rarely think of it, allowing us to know intuitively when it is our time to talk, seamlessly finish each other's sentences, and have the ability to inject a quip in a group conversation at exactly the right moment. Research in sociolinguistics, psycholinguistics, and conversational analysis has revealed that turn-taking is a mixed-initiative, locally coordinated process, in which a variety of verbal and nonverbal cues such as eye gaze, body pose, head movements, hand gestures, intonation, hesitations, and filled pauses play a very important role. We continuously produce and monitor each other for these signals and can coordinate seamlessly at the scale of hundreds of milliseconds across these different channels with multiple actors. Can we achieve such precise but powerful coordination with machines?

Beyond turn-taking, coordination on concepts is commonly exhibited in interactions between experts and novices, where the novices quickly learn to adapt to the terminology used by the experts, for example when naming tools. Can we achieve this type of learning and implicit teaching with machines?

Five of the articles in this issue of AI Magazine focus on issues of turn-taking, coordination, and collaboration in human-machine interaction. The contributing authors have been working in interrelated fields of spoken dialog systems, intelligent virtual agents, human-computer interaction, humanrobot interaction, and semiautonomous collaborative systems, exploring core mechanisms that facilitate coordinating speech and actions with virtual agents, robots, and other autonomous systems. Several of the contributors participated in the AAAI Spring Symposium on Turn-Taking and Coordination in Human-Machine Interaction, held in March 2015, and several articles in this issue are extensions of work presented at the symposium.

Nigel G. Ward and David DeVault open this issue with an article outlining a core set of 10 challenges for highly interactive dialog systems. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.