Computer graphics processing and selective visual display system – Display driving control circuitry – Controlling the condition of display elements
Reexamination Certificate
2000-04-17
2003-09-23
Cabeca, John (Department: 2173)
Computer graphics processing and selective visual display system
Display driving control circuitry
Controlling the condition of display elements
C345S156000, C345S157000, C345S158000
Reexamination Certificate
active
06624833
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to man-machine interfaces, and more particularly to gesture-based input interface systems for communicating information to computers or other display-based processing systems via user hand gestures.
BACKGROUND OF THE INVENTION
Speech and gestures are the most commonly used means of communication among humans. Yet, when it comes to communicating with computers, the typical home or business user is still bound to devices such as the keyboard and the mouse. While speech recognition systems are finding their way into low-cost computers, there is a real need for gesture recognition systems that provide robust, real-time operation at low cost, so as to be readily available to the typical home or business user.
Interest in vision-based gesture recognition has been growing since early 1990s. See T. Huang and V. Pavlovic, “Hand Gesture Modeling, Analysis and Synthesis,” Proc. International Conference on Automatic Face and Gesture Recognition, pp. 73-79, June 1995, for a review of various conventional techniques.
Much of this effort has been devoted specifically to gesture-based computer interfaces, as described in, e.g., A. Azarbayejani, T. Starner, B. Horowitz, and A. Pentland, “Visually Controlled Graphics,” IEEE Transactions on Pattern Recognition and Machine Intelligence, 15(6):602-605, June 1993, R. Kjeldsen and J. Kender, “Visual Hand Recognition for Window System Control,” Proc. International Conference on Automatic Face and Gesture Recognition, pp. 184-188, June 1995, R. Kjeldsen and J. Kender, “Towards the use of Gesture in Traditional User Interfaces,” Proc. International Conference on Automatic Face and Gesture Recognition, pp. 151-156, October 1996, M. W. Krueger, “Artificial Reality II,” Addison-Wesley, 1991, C. Maggioni, “GestureComputer—New Ways of Operating a Computer,” Proc. International Conference on Automatic Face and Gesture Recognition, pp.166-171, June 1995, J. M. Rehg and T. Kanade, “DigitalEyes: Vision Based Human Hand Tracking,” CMU Tech Report CMU-CS-93-220, 1993, W. T. Freeman and C. D. Weissman, “Television Control by Hand Gestures,” Proc. International Conference on Automatic Face and Gesture Recognition, pp. 179-183, June 1995, A. Utsumi and J. Ohya, “Multiple-Hand-Gesture Tracking Using Multiple Cameras,” Proc. International Conference Computer Vision and Pattern Recognition, pp. 473-478, June 1999, M. Kohler, “System Architecture and Techniques for Gesture Recognition in Unconstraint Environments,” Proc. Int. Conf. Virtual Systems and Multimedia, 1997, H. Nishino et al., “Interactive Two-Handed Gesture Interface in 3D Virtual Environments,” Proc. ACM Symp. Virtual Reality Software and Technology, 1997, J. Segen, “Controlling Computers with Gloveless Gestures,” Proceedings of Virtual Reality Systems, 1993, V. J. Vincent, “Delving in the depth of the mind,” Proc. Interface to Real and Virtual Worlds, 1991, D. Weimer and S. K. Ganapathy, “Interaction Techniques using Hand Tracking and Speech Recognition,” Multimedia Interface Design, ed. M. Blettner and R. Dannenbergc, pp.109-126, Addison-Wesley, 1992, P. Wellner, “The DigitalDesk Calculator: Tangible Manipulation on a Desktop Display,” Proc. ACM Symposium on User Interface Software and Technology, November 1991.
By way of example, the above-cited C. Maggioni reference describes a system using two cameras, that detects the position of the palm of a user's hand in three dimensions (3D). The system can recognize six static gestures, and is used as interface to a virtual environment. As another example, the above-cited R. Kjeldsen and J. Kender references describe a neural net based gesture recognition and hand tracking system that can be used in place of a mouse to move and resize computer windows.
A gesture-based input interface system is described in U.S. patent application Ser. No. 08/887,765; filed Jul. 3, 1997, now U.S. Pat. No. 6,252,298, issued Jun. 26, 2001, in the name of inventor J. Segen, which application is commonly assigned herewith and incorporated by reference herein.
A known multiple-camera gesture-based input interface system referred to as GestureVR is described in J. Segen and S. Kumar, “GestureVR: Vision-Based 3D Hand Interface for Spatial Interaction,” Proc. Sixth ACM International Multimedia Conference, Bristol, U.K., September 1998, which is incorporated by reference herein. This system provides a number of advantages over the other systems noted above.
Additional details regarding the GestureVR system and other gesture-based input interface systems are disclosed in U.S. patent application Ser. No. 09/208,079 filed Dec. 9, 1998, now U.S. Pat. No. 6,204,852, issued Mar. 20, 2001, in the name of inventors S. Kumar and J. Segen and entitled “Video Hand Image Three-Dimensional Computer Interface,” and U.S. patent application Ser. No. 09/208,196, filed Dec. 9, 1998, now U.S. Pat. No. 6,147,678, issued Nov. 14, 2000, in the name of inventors S. Kumar and J. Segen and entitled “Video Hand Image Three-Dimensional Computer Interface With Multiple Degrees of Freedom,” both commonly assigned herewith and incorporated herein by reference.
It is also known in the art to utilize shadows in computer vision image processing applications. An example of one such application is in the area of extracting buildings from aerial images, with shadows being used to generate or verify building hypotheses and to estimate building heights. Such techniques are referred to as “shape from shading” techniques. See, e.g., D. G. Lowe and T. O. Binford, “The Interpretation of Geometric Structure from Image Boundaries,” ARPA IUS Workshop, pp. 39-46, 1981, and C. Lin and R. Nevatia, “Building Detection and Description from a Single Intensity Image,” Computer Vision and Image Understanding, 72(2):101-121, 1998. Shadows have also been used to infer object shapes, as described in, e.g., S. A. Shafer and T. Kanade, “Using Shadows in Finding Surface Orientations,” CVGIP, 22:145-176, 1983, J R. Kender and E. M. Smith, “Shape from Darkness: Deriving Surface Information from Dynamic Shadows,” Proc. ICCV, 1987, D. Raviv, Y. Pao, and K. A. Loparo, “Reconstruction of Three-Dimensional Surfaces from Two Dimensional Binary Images,” IEEE Trans. Rob. and Auto, 5(10):701-710, 1989, and L. Wang and J. J. Clark, “Shape from Active Shadow Motion,” Proc. SPIE Conf. on Intelligent Robots and Computer Vision: Active Vision and 3D Methods, Boston, Mass., 1993. Compared to “shape from shading” techniques, these “shape from shadow” techniques have an advantage in that they do not require surface reflectance maps.
Although shadow processing has been applied in the above-noted computer vision applications, it has not heretofore been applied to improving detection of gestures in a gesture-based input interface system.
In view of the foregoing, a need remains for a gesture-based input interface system that utilizes shadow processing and is capable of providing robust, real-time operation in a low-cost manner more readily accessible to typical home and business users.
SUMMARY OF THE INVENTION
The present invention provides an improved gesture-based input interface system which meets the above-identified need.
An input interface system in accordance with the invention provides gesture-based user control of an application running on a computer. Image signals generated by a camera are processed to determine if the image signals contains one of a number of designated user gestures, e.g., a point gesture, a reach gesture and a click gesture, each of the gestures being translatable to a particular control signal for controlling the application.
In accordance with the invention, if a given image signal is determined to contain a point gesture, the image signal is further processed to determine position and orientation information for a pointing finger of a hand of the user and its corresponding shadow. The position and orientation information for the pointing finger and its shadow are then utilized to generate a three-dimensional pose estimate for the pointing figure in the given gesture.
Kumar Senthil
Segen Jakub
Basom Blaine
Cabeca John
Lucent Technologies - Inc.
LandOfFree
Gesture-based input interface system with shadow detection does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Gesture-based input interface system with shadow detection, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Gesture-based input interface system with shadow detection will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3061938