made of Light -- Principles, Metaphors and Ex
Today's interfaces to technical systems commonly rely on keyboards, monitors, mice, switches or buttons that are mostly difficult to understand, to learn and to operate. We will describe a new "Come as you are"-style interface ( Krueger 1991), called VirtualTouchscreen, for contactless and natural man-machine interaction. The video-based system is able to detect human hand gestures and movements in real-time and uses them to control a projected graphical user interface (GUI).
For the input channel we developed a vision-based real-time gesture recognition system. It runs on state-of-the-art personal computers without the need for special hardware and requires only a moderate amount of the available computational power thus forming an affordable novel input device. On the output channel the VirtualTouchscreen uses a video beamer to project information on a wide range of arbitrary shaped and sized surfaces. The projected GUI may be designed to be a standard window-based display or may be tailored to the specific needs of the system to be controlled. Through this approach, any input hardware like keyboard, mouse or traditional touch screen can be replaced. Hand gestures within the projected GUI control the desired application.
First approaches for using a computer vision system as an input device for computer applications include Videodesk ( Krueger 1991) and DigitalDesk ( Wellner 1991). Further research explored 3D hand interaction ( Rehg & Kanade 1994; Segen, & Kumar 1998) and even full body interaction as in the MIT Pfinder project ( Wren, Azarbayejani, Darrel & Pentland 1997).