Computer graphics processing and selective visual display system – Display driving control circuitry – Controlling the condition of display elements
Reexamination Certificate
1999-06-21
2001-11-27
Jankus, Almis R. (Department: 2672)
Computer graphics processing and selective visual display system
Display driving control circuitry
Controlling the condition of display elements
Reexamination Certificate
active
06323884
ABSTRACT:
FIELD OF INVENTION
The present invention relates to improved ease of use for graphical user interfaces (GUIs), and in particular to assisting the user in the operation of selecting a destination or element within a GUI.
BACKGROUND OF THE INVENTION
Apparatus is known which tracks the eye movement of a user of a GUI to determine what position within the interface is being viewed, using reflections of signals from the user's retina. When the user's eye focuses on a single position corresponding to a selectable element of the interface for a predefined period of time (for example 0.3 seconds), this is interpreted as selection of the element. Example selectable GUI elements are icons such as buttons, and items in a context menu or scrollable list.
There is a problem with the known eyeball tracking mechanisms, which is that they are not very accurate. The accuracy can be adequate for very simple interfaces in which the GUI elements are each very large, but a requirement for very large GUI elements is often unacceptable for real applications where there is limited display space available for each element and it is likely to result in an unattractive and simplistic GUI. Furthermore, eye tracking selection mechanisms may not perform well even for large GUI elements if the user focuses near an edge of the GUI element. A mechanism is needed for dealing with the inaccuracy of the eye tracking mechanisms to allow eye tracking to be used with any desired interface design.
There are other types of apparatus for which fast and accurate selection of a GUI element is difficult to achieve. For example, touch-response screens and any apparatus which has a small display screen and relies on user-controlled positioning of a pointer for selection of GUI elements has the problem of positioning the pointer within the small screen area of the desired GUI element. Examples of such apparatus are laptop computers and some personal digital assistants (PDAs).
The positioning problem is a particular concern for users of GUI selector mechanisms who have physical limitations affecting their ease of use, but ease of use is of major importance for all users.
Warping of an input device pointer to a newly created window is used, for example, by the Motif(TM) Window Manager (which is the default window manager of The Open Group's Motif user interface standard). However, this is an inflexible action which is only taken when the window is created.
U.S. Pat. No. 5,745,100 discloses an input device pointer remapping facility which causes an input device pointer, when entering a region on a computer display, to jump to a specific point within the region such as the centre of the region. Further movements of the pointer through the region are then adjusted to increase the likelihood that the pointer will remain in the region. There is at most only one predefined fixed point which is an active jump point for any position of the pointer within the display screen.
U.S. Pat. No. 5,737,555 discloses rapid repositioning of a display pointer to a predefined location on a display screen in response to an input stimulus. This saves the user from physically moving the pointer but relies on a predefined order of display pointer movement between elements of a GUI (i.e. a predefined action sequence) to identify the next predefined pointer location.
IBM Technical Disclosure Bulletin article “Method for simplifying the accurate positioning of a mouse pointer”, P. Hanna, vol.36, No.4, April 1993, pages 339-342, discloses using an heuristic to predict the intended destination of a user-controlled mouse pointer movement and then automatically moving the pointer to that destination. The mouse pointer's initial position and directed acceleration are used to calculate a force vector which is then matched to actual positions of mouse-selectable areas. The acceleration must exceed a predefined threshold for the automated movement to be initiated. However, the initial user-controlled acceleration of an input device pointer may not be proportional to the distance that the user intends to move the pointer, and so the reliance on measured acceleration will tend to produce erroneous predictions. Secondly, no account is taken of the current application program state and so the pointer may be moved to mouse-selectable areas which are not currently valid selections.
DISCLOSURE OF THE INVENTION
The invention provides a software-implemented mechanism for assisting a GUI user with the task of selecting a GUI element. The software controls the operation of a computer to execute processes to identify those GUI elements which are potentially the next GUI element that the user will interact with, to compare a predefined set of characteristics for the identified GUI elements to predict which of these GUI elements will be the next element that the user will interact with, and to visually indicate the result of the prediction.
The identified potential next GUI elements are preferably the complete set of GUI elements that are valid next selections for the current application state. Alternatively, the identified GUI elements may be the set of validly selectable GUI elements which are within the currently active window within the GUI, and the software implementing the invention may be invoked in response to opening of a new window to predict which GUI element within the window the user is likely to interact with. The identified GUI elements could equally be the subset of GUI elements which satisfy any predefined criteria.
The predefined set of characteristics preferably include a distance measurement between a user indicated position within the GUI and the position of each potentially selectable GUI element. The user indication of a position may comprise focusing of the eye, as measured by an eye tracking mechanism, orientation of the user's head or some other measurable body movement, or user-controlled positioning of an input device pointer. The step of comparing a set of characteristics is preferably initiated when the user indicated position moves to within a predefined distance of a selectable GUI element.
Alternatively or additionally, the set of characteristics may include a determination of whether the identified GUI elements are within a user indicated region of the GUI, the user indication including an initial user-controlled movement of a selection pointer and a vector representing the direction of movement and starting position of the pointer being calculated by the software.
According to a preferred embodiment, the set of characteristics include weightings associated with each of the GUI elements, the weightings indicating a likelihood of selection of each element given the last GUI element selected or taking account of a sequence of previous selections. The weightings are preferably based on statistics of previous sequences of user interactions, such that the prediction accuracy improves over time as the weightings change in response to user interactions. This capacity of the software to learn over time and to use weightings based on previous user interaction sequences to predict a next GUI element is a significant improvement over prior art selection mechanisms.
For example, if the valid next GUI elements following a given interaction are a “Proceed” button and a “Cancel” button, then the software refers to its maintained statistics of previous selections and predicts which button will be selected next based on this previous experience. The software then moves a selection pointer into the area of the GUI of the selected button (if the selection mechanism uses a pointer) or adds emphasis such as a highlight colour or animation of the selected button. There may be no input pointer if the selection mechanism is eye tracking. This movement of a pointer or addition of emphasis to a selectable GUI element does not force the user to take that option, it merely assists them with the positioning task.
The invention can provide great assistance where certain sequences of interactions involving selection of GUI elements are predictabl
Bird Colin Leonard
Chapman Sydney George
Cunningham G. F.
Duffield Edward H.
International Business Machines - Corporation
Jankus Almis R.
LandOfFree
Assisting user selection of graphical user interface elements does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Assisting user selection of graphical user interface elements, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Assisting user selection of graphical user interface elements will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2613801