Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
1998-12-09
2001-03-20
Vo, Cliff N. (Department: 2772)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C345S440000
Reexamination Certificate
active
06204852
ABSTRACT:
BACKGROUND OF THE INVENTION
I. Field of the Invention
This invention pertains to a method and apparatus for inputting commands to a computer using hand signals. More particularly, the present invention relates to a video gesture-based computer interface wherein images of hand gestures are used to control a computer and wherein motion of the user's hand or a portion thereof is tracked in a three-dimensional coordinate system with five degrees of freedom.
II. Description of the Related Art
Various types of computer control and interface devices exist for inputting commands to a computer. Such devices may for example take the form of a computer mouse, joystick or trackball, wherein a user manipulates the interface device to perform a particular operation such as to select a specific entry from a menu of options, perform a “click” or “point” function, etc. A significant problem associated with such interface devices is that a surface area is needed for placement of the device and, in the case of a mouse, to accommodate device movement and manipulation. In addition, such interface devices are generally connected by a cable to a computer CPU with the cable typically draped across the user's desk, causing obstruction of the user's work area. Moreover, because interface device manipulation for performing operations is not consistent with common communication movements, such as the use of a pointing finger hand gesture to select a menu entry, as opposed to maneuvering a mouse until the cursor rests on the desired menu entry, a user must become comfortable and familiar with the operation of the particular interface device before proficiency in use may be attained.
To address these drawbacks, a video interface system for enabling a user to utilize hand gestures to issue commands to a computer has been developed and is described in commonly assigned U.S. patent application of Segen entitled “Video Hand Image Computer Interface”, Ser No. 08/887,765 filed Jul. 3, 1997 (hereinafter “Segen”), which is hereby incorporated herein by reference in its entirety. The Segen system, by way of preferred example, utilizes a video camera or other video input device connected to an image processing computer, with the camera positioned to receive images of an object such as a user's hand. The image processing capabilities of the computer act upon predetermined recognized hand gestures as computer commands. Hand images from the camera are converted to a digital format and input to the computer for processing. The results of the processing and attempted recognition of each image are sent to an application or the like for performing various functions or operations.
However, the use of both traditional two-dimensional input devices and the Segen system is problematic in advanced computer-based three-dimensional object selection and manipulation applications. In such applications, a virtual three-dimensional environment is typically displayed to the user with one or more displayed virtual objects and command menus positioned within the virtual environment. The user may delete, move and otherwise change the objects in the virtual environment or create new objects. The user may also select various commands from the command menus. Other functions may be performed in the virtual environment such, for example, as drawing curves. Traditional input devices are extremely difficult to use in such a virtual environment because traditional devices control only two degrees of freedom and, thus, a combination of several input devices or, with a single device, more than one operation, is required to control three or more degrees of freedom as is necessary in three-dimensional applications. Such a combination control scheme is cumbersome, unintuitive and requires significant training on the user's part. The Segen system provides for three degrees of freedom which is more than adequate for issuing commands, but may not be sufficient for use in certain three-dimensional applications where interaction with three-dimensional objects is necessary.
It would thus be desirable to provide a computer interface that enables common and intuitive hand gestures and hand motions to be used for interacting with a three-dimensional virtual environment. It would further be desirable to provide a system and method for tracking hand gestures and hand motions in a three-dimensional coordinate system with five degrees of freedom.
SUMMARY OF THE INVENTION
This invention relates to the use of images input to a computer for effectuating computer control. The present invention provides a system and method for tracking the three-dimensional position and orientation of a finger of the user's hand with five degrees of freedom. The tracking of the user's finger by the inventive system enables the user to utilize and control computer applications requiring three-dimensional interaction.
The system includes a computer for controlling operation of the system, having image processing capabilities and a display for displaying a virtual environment to a user, and at least two video input devices, such as video cameras, connected to the computer for acquiring video images and for enabling the computer to operatively identify, interpret, and track motions and gestures of a user's hand made by the user within an identification zone monitored by the video input devices (hereinafter “cameras”). The identification zone is an area that is typically, although not necessarily, near the computer system towards which the first and second cameras are oriented. The configuration and functionality of the cameras and the computer are similar to the above-incorporated Segen system, except that while the Segen system employs only a single camera, the system of the present invention utilizes two or more cameras.
It should be noted that while the inventive system is largely based on the above-incorporated Segen system, the system of the present invention provides important additional inventive features that are essential for enabling the user to control applications requiring three-dimensional interaction. These additional features are possible because the inventive system uses two or more cameras to simultaneously capture images of the user's hand, while the Segen system employs only a single camera to capture a single image.
During operation of the system, hand images from the cameras are continually converted to a digital format and input to the computer for processing. As in the Segen system, the results of the processing and attempted recognition of each image are then sent to an application or the like for performing various functions or operations. However, when the inventive computer recognizes a hand gesture as a “point” gesture with one finger extended, the computer uses information derived from the images to track three-dimensional coordinates of the extended finger of the user's hand with five degrees of freedom. Essentially, the computer utilizes two-dimensional images obtained by each camera to derive three-dimensional position (in an x, y, z coordinate system) and orientation (azimuth and elevation angles) coordinates of the extended finger. This feature is not possible in the Segen system because at least two cameras are necessary to generate the information required for three-dimensional position and orientation tracking. Thus, the present invention provides an advantageous supplement to the functionality of the Segen system because, in addition to issuing commands to a computer using hand gestures (as described in detail in Segen), the inventive system enables user control of applications that require three-dimensional interaction.
The inventive method of tracking three-dimensional coordinates of the user's finger with five degrees of freedom is performed as follows. Initially, the two cameras are calibrated to derive a perspective projection matrix for each camera. The computer then simultaneously acquires a first image of the user's hand from the first camera and a second image of the user's ha
Kumar Senthil
Segen Jakub
Lucent Technologies - Inc.
Vo Cliff N.
LandOfFree
Video hand image three-dimensional computer interface does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Video hand image three-dimensional computer interface, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Video hand image three-dimensional computer interface will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2468656