Image analysis – Pattern recognition – Feature extraction
Reexamination Certificate
1998-02-19
2001-06-19
Grant, II, Jerome (Department: 2624)
Image analysis
Pattern recognition
Feature extraction
C382S203000, C382S153000, C345S182000, C345S156000, C345S173000
Reexamination Certificate
active
06249606
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to the field of computer implemented user interfaces. More specifically, the present invention relates to the field of computer implemented recognition of user input information.
2. Related Art
In modern computing technology, there is always a need to provide mechanisms to facilitate user interaction with computing devices. By increasing the mechanisms by which persons interact with computer systems, the number of potential computer users and the number of potential computer applications expand. Further, by facilitating user interface mechanisms, applications become easier to use and more efficient. Today, users can communicate with computer systems using a number of various devices including refreshable display screens (cathode ray tubes, liquid crystal displays etc.), alphanumeric keyboards, keypads, cursor directing devices, microphones, etc. Keyboards are used for conveying information in alphanumeric form to a computer from a user.
The cursor directing device is used in conjunction with an animated cursor image that is rendered on the display screen. The cursor image is animated in that it can move across the display screen in real-time tracing the motion of the cursor directing device. Cursor directing devices, e.g., mouse devices, trackballs, etc., are used to direct the position of the cursor image on a display screen according to user interaction. In operation, a hand held or user directed mouse device is displaced across a mouse pad and the corresponding displacement and direction are simultaneously traced out by the cursor image as rendered on the display screen. When the cursor image is directed over a particular portion of the display screen, one or more buttons on the mouse device can be depressed to “activate” the cursor image which generally invokes a computer action related to the screen portion. The areas on the display screen that invoke a computer command when the cursor image is positioned thereon and activated have been called “hot spots.” In the past, to convey information to the computer system, cursor directing devices have been used in conjunction with hot spots located on the display screen.
More particularly, when the cursor image is activated in prior art user interfaces, the computer system performs a relatively routine task of checking the screen coordinates of the cursor image against the screen coordinates of a number of recorded hot spots to determine which enabled hot spot was selected by the cursor activation. In performing the check to determine which hot spot is selected, the computer system typically does not care about the screen path in which the cursor image passes through in order to reach the hot spot. Further, in performing the check to determine which hot spot is selected, the computer system typically does not care about the speed in which the cursor image was directed to the hot spot. All that is checked by the computer system is the screen coordinate of the cursor image when the cursor image is activated (e.g., when a mouse button is depressed). Thus, in the past, the real-time displacements of the mouse device between mouse activations are largely ignored.
Since the cursor directing device allows a substantial amount of real-time free style user-directed movement within the plane of the mouse pad, it would be advantageous to provide a user interface mechanism that could make use of more information from the cursor directing device than merely the display screen coordinate of the cursor image upon cursor activation.
Accordingly, the present invention provides a system and method of interfacing a user with a computer system that provides more information to the computer system from a cursor directing device than merely the screen coordinate of the cursor image upon cursor activation. The present invention provides a system and method of interfacing a user with a computer system that conveys information regarding the path through which a cursor directing device is spatially displaced and the relative speed of such spatial displacement. The above information provides a unique and advantageous mechanism by which information can be conveyed to the computer system from the user. These and other advantages of the present invention not specifically mentioned above will become clear within discussions of the present invention presented herein.
SUMMARY OF THE INVENTION
A computer implemented method and system are described for gesture category recognition and training. In one embodiment, a cursor directing device is used. Generally, a gesture is a hand or body initiated movement of a cursor directing device which outlines a particular pattern, in particular directions, and can comprise one or more strokes. The present invention allows a computer system to accept input data, originating from a user, in the form gesture data that are made using a cursor directing device. In one embodiment, a mouse device is used, but the present invention is equally well suited for use with other cursor directing devices (e.g., a track ball, a finger pad, an electronic stylus, optical tracking device, etc.). In one embodiment, the computer system is queued to accept a new gesture data by pressing a key on the keyboard and then moving the mouse (e.g., while a mouse button is depressed) to trace out a gesture that is associated with a gesture category. Coordinate position information of the mouse and time stamps are recorded as gesture data in memory based on the user gesture. More than one gesture can be associated with a gesture category (e.g., as negative and positive examples).
The present invention then determines a multi-dimensional feature vector based on the gesture data. The multi-dimensional feature vector is then passed through a gesture category recognition engine that, in one implementation, uses a radial basis function neural network to associate the feature vector to a preexisting gesture category. Once identified, a set of user commands that are associated with the gesture category (e.g., a macro) are applied to an application of the computer system. The user commands can originate from an automatic process that extracts commands that are associated with the menu items of a particular application program. The present invention also allows user training so that userdefined gesture categories, and the computer commands associated therewith, can be programmed into the computer system.
More specifically, an embodiment of the present invention includes a method of providing a user interface in an electronic system having a processor, a memory unit, an alphanumeric input device, the method comprising the computer implemented steps of: a) receiving gesture data representing a gesture performed by a user with the cursor directing device, the gesture data comprising coordinate positions and timing information and having one or more individual strokes; b) generating a multi-dimensional feature vector based on the gesture data; c) providing the multi-dimensional feature vector to a radial basis function neural network for recognition, the radial basis function neural network associating the multi-dimensional feature vector with a gesture category from a predefined plurality of gesture categories and supplying the gesture category as an output value; and d) applying a set of predetermined commands to the electronic system, the set of predetermined commands being associated with the gesture category output from the radial basis function neural network.
Embodiments include the above and wherein the step b) comprises the steps of: b1) normalizing the gesture data; b2) dividing each stroke of the gesture data into a plurality of segments, N; b3) determining first feature elements for each stroke of the gesture data based on an end point of a respective stroke and a start point of a next stroke; b4) determining second feature elements for each segment of each stroke of the gesture data based on an orientation of each segment with respect to a reference line, wherein the multi-
Dobler Ervin
Kiraly Jozsef
Grant II Jerome
Kianni Kevin C
Mindmaker, Inc.
Wagner , Murabito & Hao LLP
LandOfFree
Method and system for gesture category recognition and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and system for gesture category recognition and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and system for gesture category recognition and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2518893