Data processing: presentation processing of document – operator i – Operator interface – Gesture-based
Reexamination Certificate
2008-04-18
2010-10-26
Hailu, Tadeese (Department: 2173)
Data processing: presentation processing of document, operator i
Operator interface
Gesture-based
C715S857000, C715S860000, C345S179000
Reexamination Certificate
active
07823089
ABSTRACT:
The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.
REFERENCES:
patent: 4839838 (1989-06-01), LaBiche
patent: 5177311 (1993-01-01), Suzuki et al.
patent: 5181181 (1993-01-01), Glynn
patent: 5310192 (1994-05-01), Miyake
patent: 5453758 (1995-09-01), Sato
patent: 5454043 (1995-09-01), Freeman
patent: 5528263 (1996-06-01), Platzker et al.
patent: 5554980 (1996-09-01), Hashimoto et al.
patent: 5555003 (1996-09-01), Montgomery
patent: 5598187 (1997-01-01), Ide et al.
patent: 5600765 (1997-02-01), Ando
patent: 5666138 (1997-09-01), Culver
patent: 5719622 (1998-02-01), Conway
patent: 5724106 (1998-03-01), Autry
patent: 5748186 (1998-05-01), Raman
patent: 5757360 (1998-05-01), Nitta et al.
patent: 5825350 (1998-10-01), Case et al.
patent: 5875257 (1999-02-01), Marrin et al.
patent: 5878274 (1999-03-01), Kono et al.
patent: 5884249 (1999-03-01), Namba et al.
patent: 5920024 (1999-07-01), Moore
patent: 5929844 (1999-07-01), Barnes
patent: 6021403 (2000-02-01), Horvitz et al.
patent: 6053814 (2000-04-01), Pchenitchnikov
patent: 6072467 (2000-06-01), Walker
patent: 6111580 (2000-08-01), Kazama et al.
patent: 6150947 (2000-11-01), Shima
patent: 6162123 (2000-12-01), Woolston
patent: 6184863 (2001-02-01), Sibert
patent: 6229526 (2001-05-01), Berstis
patent: 6241609 (2001-06-01), Rutgers
patent: 6251011 (2001-06-01), Yamazaki
patent: 6256047 (2001-07-01), Isobe et al.
patent: 6287198 (2001-09-01), McCauley
patent: 6300933 (2001-10-01), Nagasaki
patent: 6312335 (2001-11-01), Tosaki et al.
patent: 6344861 (2002-02-01), Naughton et al.
patent: 6345111 (2002-02-01), Yamaguchi
patent: 6369794 (2002-04-01), Sakurai
patent: 6419580 (2002-07-01), Ito
patent: 6469633 (2002-10-01), Wachter
patent: 6499025 (2002-12-01), Horvitz
patent: 6517438 (2003-02-01), Tosaki et al.
patent: 6545661 (2003-04-01), Goschy
patent: 6569019 (2003-05-01), Cochran
patent: 6601055 (2003-07-01), Roberts
patent: 6720949 (2004-04-01), Pryor
patent: 6727885 (2004-04-01), Ishino
patent: 6734847 (2004-05-01), Baldeweg et al.
patent: 6744420 (2004-06-01), Mohri
patent: 6791531 (2004-09-01), Johnston et al.
patent: 6821206 (2004-11-01), Ishida et al.
patent: 6890262 (2005-05-01), Oishi et al.
patent: 6894716 (2005-05-01), Harrington
patent: 6906700 (2005-06-01), Armstrong
patent: 6918829 (2005-07-01), Ikariko
patent: 6949024 (2005-09-01), Kaku et al.
patent: 6951515 (2005-10-01), Ohshima
patent: 7028269 (2006-04-01), Cohen-Solal
patent: 7036462 (2006-05-01), Cohen
patent: 7102616 (2006-09-01), Sleator
patent: 7183480 (2007-02-01), Nishitani et al.
patent: 7331856 (2008-02-01), Nakamura et al.
patent: 2002/0004422 (2002-01-01), Tosaki
patent: 2007/0091084 (2007-04-01), Ueshima
patent: 0357909 (1990-03-01), None
patent: 0919906 (1999-02-01), None
patent: 61161537 (1986-07-01), None
patent: 8038741 (1996-02-01), None
patent: 9519031 (1995-07-01), None
patent: 9803907 (1998-01-01), None
patent: 9942920 (1999-08-01), None
patent: 0073995 (2000-12-01), None
patent: 0140807 (2001-06-01), None
patent: 0169365 (2001-09-01), None
patent: 0215560 (2002-02-01), None
European Search Report, Application No. EP09006844, dated Sep. 30, 2009, 66 pp.
Aimone, C., R. Fung, A. Khisti, M. Varia. The Head Mounted Control System. University of Toronto CSIDC Competition Submission, May 2001.
Fisher, R.B., A.P. Ashbrook, C. Robertson, N. Werghi, A low-cost range finder using a visually located, structured light source, Proc. 2nd Int'l Conf. on 3-D Digital Imaging and Modeling, Ottawa, Canada, Oct. 1999, pp. 24-33.
Jojic, N., B. Brummiott, B. Meyers, S. Harris, and T. Huang, Estimation of Pointing Parameters in Dense disparity Maps. In IEEE Intl. Conf. on Automatic face and Gesture Recognition, (Grenbole, France, 2000).
Kohler, M.R.J. System Architecture and Techniques for Gesture Recognition in Unconstraint Environments, Proc. Of the 1997 Int. Conf. on Virtual Systems and Multimedia (VSMM'97), Geneva, Sep. 10-12, 1997, pp. 137-146.
Masaaki, Fukomoto, et al.: “Finger-Pointer: Pointing Interface by Image Processing,” Computers and Graphics, Pergamon Press Ltd., Oxford, GB, vol. 18, No. 5, Sep. 1, 1994, pp. 633-642, XP000546603, ISSN: 0097-8493 *the whole document*.
Masui, T. and I. Siio, Real-world graphical user interface, Proceedings of the Internatioinal Symposium on Handheld and Ubiquitous Computing (HUC2000, Sep. 2000, pp. 72084.
McKenzie, Mill K., et al.: “Integrating speech and two-dimensional gesture input—a studyof redundancy between modes,” Computer Human Interaction Conference, 1998. Proceedings. 1998 Australian Adelaide, SA, Australia Nov. 30-Dec. 4, 1998, Los Alamitos, CA USA IEEE Comput. Soc, US, pp. 6-13, XP010313626, ISN: 0-8186-9206-5 *the whole document*.
Michael Johnston and Srinivas Bangalore, “Finite-state Multomedia Parsing and Understanding,” AT&T Labs-Research Shannon Laboratory, Jul. 2000.
Olsen, D.R.J., T. Nielsen, Laser Pointer Interaction. In Proceedings CHI'2001:Human Factors in Computing Systems (Seattle, 2001), 17-22.
Oviatt, S., Mutual disambiguation of recognition errors with multimodal architecture, Proc. Of the CHI '99 Conference on Human Factors in Computing Systems, 1999, pp. 576-583, ACM Press.
Oviatt, S.L. Taming Speech Recognition Errors Within a Multimodal Interface. Communications of the ACM, 43(9). 45-51.
Pavlovic, V. I., et al.: “Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review,” IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Service Center, Los Alamitos, CA, US, vol. 19, No. 7, Jul. 1997, pp. 677-695, XP000698168. ISSN: 0162-8828 *the whole document.
Pearl, J. Probabilistic Reasoning in Intelligent Systems, Morgan Kaufmann, San Mateo, California, 1988.
Priyantha, N.B., A. Chakraborty, H. Balakrishnan, The Cricket Location-Support System. In Proceedings 6th ACM MOBICOM, (Boston, MA, 2000).
Rabiner, L.R. and B-H. Juang., An Introduction to Hidden Markov Models. IEEE ASSP Magazine (Jan. 1986) 4-15.
Randell, C. and Henk Muller. Low Cost Indoor Positioning System. In Ubicomp 2001:Ubiquitous Computing, (Atlanta, Georgia, 2001), Springer-Verlag, 42-48.
Starner, T., et al.: “The Gesture Pendant: A Self-Illuminating, Wearable, Infrared Computer Vision System for Home Automation Control and Medical Monitoring,” Intl. Symposium on Wearable Computers. Digest of Papers, No. 4, Oct. 16, 2000, pp. 87-94, XP002907652 pp. 87-88.
Swindells, C., K. Inkpen, J. Dill, M. Tory, That one there! Pointing to establish device identify, Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology, 2002 Paris, pp. 151-160.
Wexelblat, A., “An Approach to Natural Gesture in Virtual Environment,” ACM Transactions on Computer-Human Interaction, Avol. 2, No. 3, Sep. 1995, pp. 179-200.
Sharon Oviatt et al, “Multimodal Interfaces That Process What Comes Naturally, ” Communications of the ACM, vol. 43, No. 3, Mar. 2000, pp. 45-53.
Hailu Tadeese
Microsoft Corporation
Shook Hardy & Bacon L.L.P.
LandOfFree
Manipulating objects displayed on a display screen does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Manipulating objects displayed on a display screen, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Manipulating objects displayed on a display screen will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-4196646