Toward Conversational Human-Computer Interaction

Article excerpt

* The belief that humans will be able to interact with computers in conversational speech has long been a favorite subject in science fiction, reflecting the persistent belief that spoken dialogue would be the most natural and powerful user interface to computers. With recent improvements in computer technology and in speech and language processing, such systems are starting to appear feasible. There are significant technical problems that still need to be solved before speech-driven interfaces become truly conversational. This article describes the results of a 10-year effort building robust spoken dialogue systems at the University of Rochester.

The term dialogue is used in different communities in different ways. Many researchers in the speech-recognition community view "dialogue methods" as a way of controlling and restricting the interaction. For example, consider building a telephony system that answers queries about your mortgage. The ideal system would allow you to ask for what you need in any way you chose. The variety of possible expressions you might use makes this system a challenge for current speech-recognition technology. One approach to this problem is to have the system engage you in a dialogue by having you answer questions such as, "What is your account number?" "Do you want your balance information?" On the positive side, by controlling the interaction, your speech is much more predictable, leading to better recognition and language processing. On the negative side, the system has limited your interaction. You might need to provide all sorts of information that isn't relevant to your current situation, making the interaction less efficient.

Another view of dialogue involves basing human-computer interaction on human conversation. In this view, dialogue enhances the richness of the interaction and allows more complex information to be conveyed than is possible in a single utterance. In this view, language understanding in dialogue becomes more complex. It is this second view of dialogue to which we subscribe. Our goal is to design and build systems that approach human performance in conversational interaction. We believe that such an approach is feasible and will lead to much more effective user interfaces to complex systems.

Some people argue that spoken language interfaces will never be as effective as graphic user interfaces (GUIs) except in limited special-- case situations (for example, Schneiderman [2000]). This view underestimates the potential power of dialogue-based interfaces. First, there will continue to be more and more applications for which a GUI is not feasible because of the size of the device one is interacting with or because the task one is doing requires using one's eyes or hands. In these cases, speech provides a worthwhile and natural additional modality (Cohen and Oviatt 1995).

Even when a GUI is available, spoken dialogue can be a valuable additional modality because it adds considerable flexibility and reduces the amount of training required. For example, GUI designers are always faced with a dilemma-either they provide a relatively basic set of operations, forcing the user to perform complex tasks using long sequences of commands, or they add higher-level commands that do the task the user desires. One problem with providing higher-level commands is that in many situations, there is a wide range of possible tasks; so, the interface becomes cluttered with options, and the user requires significant training to learn how to use the system.

It is important to realize that a speech interface by itself does not solve this problem. If it simply replaces the operations of menu selection with speaking a predetermined phrase that performs the equivalent operation, it can aggravate the problem because the user would need to remember a potentially long list of arbitrary commands. Conversational interfaces, however, would provide the opportunity for the user to state what he/she wants to do in his/her own terms, just as he/she would do to another person, and the system takes care of the complexity. …