The AR technology is demonstrated by superimposing the WTK graphics on top of incoming video signal (see Fig. 2). The display can be easily modified for a see-through system. Using a speech command the user can ask the system to highlight a specific component. If the component is within view, the selected component is flashed on the display. If the component is outside the field of view, a 3D audio icon is played and moved towards the location of the component (from a canonical location in front of the user) to direct the user's attention. Using dictation recognition, the user can compose a virtual note and attach it to a component. Previously attached virtual notes may be viewed or deleted using speech commands.
We have demonstrated a distributed Augmented Reality (AR) system whose display is based on video processing, and with speech and 3D audio interaction. The powerful AR technology allows information to be displayed where it is most relevant: visually superimposed on the real world. By displaying information in 3D in the user's natural environment, HCI is enhanced to a new level in which the user's interaction with the computer is mostly transparent. The distributed architecture of the system ultimately allows for scalability and mobility (user equipped with only a light wearable computer and display). As an example, a lightweight system with AR technology can replace service manuals for maintenance applications by providing on-line information just in time and just in place.
There are several challenges to make this technology serve the vision described above and in the Introduction. First, video processing is computation-intensive, and requires fast, dedicated processors. However, with increasing demand for multimedia information, we believe that this need will be heavily addressed in the near future. Second, reliable tracking is crucial to this technology. To increase the reliability of tracking, we are currently engaged in research to supplement the video tracking with other tracking technologies (inertial and magnetic). Finally, high bandwidth wireless communication will hasten the adoption of this technology.
Espiau, B., Chaurmette, F., and Rives, P. ( 1992). A new approach to visual servoing in robotics. IEEE Trans. On Robotics and Automation, 8 (3), 313-326.
Sundareswaran, V. and Behringer, R. ( 1998). Visual Servoing-based Augmented Reality. In Procs. Intl. Workshop on Augmented Reality, San Francisco, USA, Nov 1, 1998.
-469-
