Data processing: presentation processing of document – operator i – Operator interface – Gesture-based
Reexamination Certificate
2005-07-19
2009-06-23
Hailu, Tadeese (Department: 2173)
Data processing: presentation processing of document, operator i
Operator interface
Gesture-based
C715S866000, C715S771000, C704S270000, C704S275000
Reexamination Certificate
active
07552403
ABSTRACT:
The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.
REFERENCES:
patent: 5177311 (1993-01-01), Suzuki et al.
patent: 5181181 (1993-01-01), Glynn
patent: 5454043 (1995-09-01), Freeman
patent: 5528263 (1996-06-01), Platzker et al.
patent: 5600765 (1997-02-01), Ando et al.
patent: 5719622 (1998-02-01), Conway
patent: 5748186 (1998-05-01), Raman
patent: 5825350 (1998-10-01), Case, Jr. et al.
patent: 5878274 (1999-03-01), Kono et al.
patent: 5884249 (1999-03-01), Namba et al.
patent: 5920024 (1999-07-01), Moore
patent: 6021403 (2000-02-01), Horvitz et al.
patent: 6053814 (2000-04-01), Pchenitchnikov et al.
patent: 6111580 (2000-08-01), Hisashi et al.
patent: 6150947 (2000-11-01), Shima
patent: 6241609 (2001-06-01), Rutgers
patent: 6300933 (2001-10-01), Nagasaki et al.
patent: 6312335 (2001-11-01), Tosaki et al.
patent: 6344861 (2002-02-01), Naughton et al.
patent: 6345111 (2002-02-01), Osamu et al.
patent: 6369794 (2002-04-01), Sakurai et al.
patent: 6469633 (2002-10-01), Wachter
patent: 6499025 (2002-12-01), Horvitz et al.
patent: 6517438 (2003-02-01), Tosaki et al.
patent: 6601055 (2003-07-01), Roberts
patent: 6720949 (2004-04-01), Pryor et al.
patent: 6744420 (2004-06-01), Mohri
patent: 6894716 (2005-05-01), Harrington
patent: 6918829 (2005-07-01), Ikariko
patent: 6949024 (2005-09-01), Kaku et al.
patent: 7028269 (2006-04-01), Cohen-Solal et al.
patent: 7036462 (2006-05-01), Cohen
patent: 7183480 (2007-02-01), Nishitani et al.
patent: 2002/0004422 (2002-01-01), Tosaki et al.
patent: 2007/0091084 (2007-04-01), Ueshima et al.
patent: 0 357 909 (1989-07-01), None
patent: 0 919 906 (1999-06-01), None
patent: 0919906 (1999-06-01), None
patent: 95/19031 (1995-07-01), None
patent: WO-9519031 (1995-07-01), None
patent: WO-9803907 (1998-01-01), None
patent: 99/42920 (1999-08-01), None
patent: WO-9942920 (1999-08-01), None
patent: WO-0073995 (2000-12-01), None
patent: 01/69365 (2001-09-01), None
patent: WO-0169365 (2001-09-01), None
patent: WO-0215560 (2002-02-01), None
Sharon Oviatt, “Mutual Disambiguation of Recognition Errors in a Multimodal Architecture,” CHI'99 , pp. 576-583.
Alan Wexelblat, “An Approach to Natural Gesture in Virtual Environment,” ACM Transactions on Computer-Human Interaction, vol. 2No. 3, Sep. 1995, pp. 179-200.
Michael Johnston and Srinivas Bangalore, “Finite-state Multimedia Parsing and Understanding,” AT&T Labs—Research Shannon Laboratory, Jul. 2000.
Aimone, C., R. Fung, A. Khisti, M. Varia, The head mounted control system, University of Toronto CSIDC Competition Submission, May 2001.
Fisher, R.B., A.P. Ashbrook, C. Robertson, N. Werghi, A low-cost range finder using a visually located, structured light source, Proc. 2nd Int'l Conf. on 3-D Digital Imaging and Modeling, Oct. 1999, pp. 24-33, Ottawa, Canada.
Jojic, N., B. Brummitt, B. Meyers, S. Harris, and T. Huang, Detection and estimation of pointing gestures in dense disparity maps, IEEE Int'l Conf. on Automatic Face and Gesture Recognition, 2000, Grenbole, France.
McKenzie, M. K., J. L. Alty, Integrating speech and two-dimensional gesture input—A study of redundancy between modes, Proc. of Comp. Human Interaction Conf., Nov. 30-Dec. 4, 1998, pp. 6-13. Dept. of Comput. Studies, Loughborough Univ. of Technol.
Olsen, D. R. J., T. Nielsen, Laser pointer interaction, Proc. CHI 2001, Human Factors in Computing Systems, 2001, pp. 17-22.
Oviatt, S., Mutual disambiguation of recognition errors in a multimodal architecture, Proc. of the CHI '99 Conference on Human Factors in Computing Systems, 1999, pp. 576-583, ACM Press.
Oviatt, S. L., Taming speech recognition errors within a multimodal interface, Communications of the ACM, pp. 45-51, vol. 43, No. 9, published in 2000.
Priyantha, N. B., A. Chakraborty, H. Balakrishnan, The cricket location-support system, Proc. 6th ACM MOBICOM, Boston, MA, 2000.
Rabiner, L. R., and B.-J. Juang, An introduction to hidden markov models, IEEE ASSP Magazine, Jan. 1986, pp. 4-15.
Randell, C., and H,. Muller, Low cost indoor positioning system, Ubicomp 2001: Ubiquitous Computing, pp. 42-48, Springer-Verlag, Atlanta, Georgia.
Starner, T., J. Auxier, D. Ashbrook, M. Gandy, The gesture pendant: A self-illuminating, wearable infrared computer vision system for home automation control and medical monitoring, Int'l Symposium on Wearable Comp., 2000, pp. 87-94, IEEE Comp. Soc., Atlanta, GA.
Wexelblat, A., An approach to natural gesture in virtual environments, ACM Transactions on Computer-Human Interaction (TOCHI), vol. 2, No. 3, Sep. 1995, pp. 179-200, ACM Press, New York, NY.
Tadesse, H., Office Action, Mar. 16, 2005, pp. 1-12.
Kohler, M.R.J. System architecture and techniques for gesture recognition in unconstraint environments, Proc. of the 1997 Int. Conf. on Virtual Systems and MultiMedia (VSMM'97), Geneva, Sep. 10-12, 1997, pp. 137-146.
Fukumoto, M., Y. Suenaga, K. Mase, “Finger-Pointer”: Pointing interface by image processing, Computers & Graphics, 1994, p. 633-642, vol. 18, No. 5.
Pavlovic, V. I., R. Sharma, and T. S. Huang, Visual interpretation of hand gestures for human-computer interaction: A review, IEEE Transactions on Pattern Analysis and Machine Intelligence, Jul. 1997, pp. 677-695, vol. 19, No. 7.
European Search Report, Application No. 03002829.4-2415, Date of Completion: Oct. 30, 2007, Date of Mailing Nov. 7, 2007.
Toshiyuki Masui and Itiro Siio, “Real-World Graphical User Interfaces,” Proceedings of the International Symposium on Handheld and Ubizuitous Computing (HUC2000), Sep. 2000, pp. 72-84.
Colin Swindells, Kori M. Inkpen, John C. Dill, and Melanie Tory, “That one there! Pointing to establish device identity,” Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology, 2002 Paris, pp. 151-160.
Fisher, R.B., A.P. Ashbrook, C. Robertson, N. Werghi, A low-cost range finder using a visually located, structured light source, Proc. 2nd Int'l Conf. on 3-D Digital Imaging and Modeling, Ottawa, Canada, Oct. 1999, pp. 24-33.
Masaaki, Fukomoto, et al.: “Finger-Pointer: Pointing Interface by Image Processing,” Computers and Graphics Pergamon Press Ltd., Oxford, GB, vol. 18, No. 5, Sep. 1, 1994 (Sep. 1, 1994), pp. 633-642, XP000546603, ISSN: 0097-8493 *the whole document*.
Michael Johnston and Srinivas Bangalore, “Finite-state Multomedia Parsing and Understanding,” AT&T Labs-Research Shannon Laboratory, Jul. 2000.
Olsen, D.R.J., T. Nielsen, Laser Pointer Interaction. In Proceedings CHI'2001:Human Factors in Computing Systems (Seattle, 2001), 17-22.
Pavlovic, V. I., et al.: “Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review,” IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Service Center, Los Alamitos, CA, US, vol. 19, No. 7, Jul. 1997 (Jul. 1997), pp. 677-695, XP000698168. ISSN: 0162-8828 *the whole document.
Pearl, J. Probabilistic Reasoning in Intelligent Systems, Morgan Kaufmann, San Mateo, California, 1988.
Hailu Tadeese
Microsoft Corporation
Shook Hardy & Bacon L.L.P.
LandOfFree
Controlling an electronic component within an environment... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Controlling an electronic component within an environment..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Controlling an electronic component within an environment... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-4057445