Computer graphics processing and selective visual display system – Image superposition by optical means – Operator body-mounted heads-up display
Reexamination Certificate
1996-06-25
2002-08-20
Hjerpe, Richard (Department: 2674)
Computer graphics processing and selective visual display system
Image superposition by optical means
Operator body-mounted heads-up display
C345S007000, C345S009000
Reexamination Certificate
active
06437758
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates generally to the field of computer-human user interface technology and more particularly to a method, apparatus, system and computer program product for allowing a computer to automatically determine what aspect of the computer's operation has the user's interest and to optimize that aspect.
2. Background
Human/Computer Interaction
An important characteristic of modern computing systems is the interface between the human user and the computer. Early interactive interfaces were text based wherein a user communicated with the computer by typing a sequence of characters on a keyboard and the computer communicated with the user by displaying characters on an output device—commonly a display screen. These input characters specified a command to the computer's operating system or to an application program executing on the computer. This command invoked program logic to perform a given operation. Modem computer systems use a graphical user interface (GUI) to simplify the interaction between a user and a computer. A GUI equipped computer communicates with a user by displaying graphics, including text and icons, on a display screen and the user communicates with the machine both by typing in textual information in response to dialogs and by manipulating the displayed icons with a pointing device, such as a mouse.
Many modem GUIs provide a window environment. In a typical window environment the graphical display portrayed on the display screen is arranged to resemble the surface of an electronic “desktop” and each application program running on the computer is represented as one or more electronic “paper sheets” displayed as rectangular regions on the display screen. These rectangular regions are called “windows”. Each window may include a multitude of panes. Each pane being an area for a particular type of information (textual, still image, moving image, etc.).
Each window displays information generated by an associated application or system program. Further, there may be several windows simultaneously present on the desktop with each containing information generated by a program. A program presents information to the user through each window by drawing or “painting” images, graphics or text within the window. The user can also move a window to a different location on the display screen and change its size and appearance to arrange the desktop in a convenient manner. The user communicates with the program by “pointing at” objects displayed in the window with a cursor controlled by a pointing device and manipulating the objects as desired. In some cases the program requests additional information from the user in response to a manipulation. This request is presented as a “dialog” that allows the user to provide the requested information to the dialog from the keyboard.
Each window typically includes a number of standard graphical objects such as sizing boxes, buttons and scroll bars. These features represent user interface controls that the user can manipulate with the pointing device. When the controls are selected or manipulated, the GUI invokes program logic in the underlying program to effect a corresponding command.
One characteristic of a GUI is that the GUI is only responsive to a user's explicit manipulation of the pointing device or keyboard. In the case of a mouse, the user physically moves the mouse device and a cursor on the display moves accordingly. Some pointing devices actually track the user's gaze and move the cursor to where the user “looks” on the display screen. However, even with the gaze tracking (eye tracking) devices, the GUI only responds to the user's explicit commands whether that command be a button press, a blink, or a shift of view. The computer remains a tool that the user operates by issuing explicit commands.
In contrast, humans have the ability to make inferences by looking at another human's eyes. Pupils dilate when people see something attractive. People look at what they are interested in and stare at things they find interesting. Also, human eye movements reflect thought processes. Thus, humans observe what other persons do with their eyes and make inferences as to what that other person is interested in and/or thinking.
The prior art in computer-human interfaces does not determine the user's immediate interest. Prior art computer-human interfaces simply respond to a user's command, whether input by typing the command at a keyboard, by manipulating a mouse to move a cursor, or by using a gaze tracking device to move a cursor. Thus, the computer is unable to detect or anticipate what characteristic of the computer's operation is of most interest to the user at any given time.
Gaze Tracking Devices
Most gaze tracking devices operate based upon the principal that the direction of a person's gaze is directly related to the relative positions of the pupil and the reflection of an object off the cornea (gaze tracking is often termed eye tracking). These devices often include image processing capabilities that operate on a video image of an eye to determine the gaze direction of the eye. These image processing capabilities are enhanced by using the bright eye affect.
The bright eye affect is a result of the highly reflective nature of the retina. This characteristic of the retina means that a significant amount of the light that enters an eye is reflected back through the pupil. Thus, when light shines into an eye along the axis of a camera lens, the retina reflects a significant portion of the light back to the camera. Hence, the pupil appears as a bright disk to the camera. This affect allows the pupil to be more readily imaged from a video of an eye.
Other methods exist for gaze tracking. Some incorporate having two video cameras, one for tracking head movement and the other for measuring a reflection off of the eyes. Other mechanisms involve measuring electric potential differences between locations on different sides of an eye. High accuracy devices are very intrusive on the user and require that the user's head be held in a fixed position or that the user wear special equipment to track the eye.
Recently, an eyegaze eyetracking system has been developed as described in
The Eyegaze Eyetracking System—Unique Example of a Multiple
-
Use Technology
, 4th Annual 1994 IEEE Dual-Use Technologies and Applications Conference, May, 1994. This system comprises a video camera located below a computer display that monitors one of the user's eyes. The device also contains an infrared light emitting diode (LED) located at the center of the camera's lens to maximize the bright-eye affect. Image processing software on the computer computes the user's gazepoint on the display sixty times a second with an accuracy of about a quarter inch.
Gaze tracking devices have been used for weapon control, operator training, usability analysis, market research, and as an enablement for the disabled. However, gaze tracking devices have not been used to determine what characteristic of a computer's operation interests the computer user at a particular time or to allow the computer to adapt to a user's interest as demonstrated by where on the display screen the user is looking.
Text to Speech
Many modern computers now provide text-to-speech capability. This capability processes text strings and produces understandable audio speech from the computer's audio output device (headphones or speaker). This capability allows a computer to present an audio version of a text string to a computer user.
Problems with Downloading Information
The background of the World Wide Web (WWW) and WWW browsers are well described by reference to the first chapter of
Instant HTML Web Pages
, by Wayne Ause, Ziff-Davis Press, ISBN 1-56276-363-6, Copyright 1995, pages 1-15, hereby incorporated by reference as illustrative of the prior art.
Using the Internet, a computer user has access to an immense amount of information. However, retrieving t
Glass Robert
Nielsen Jakob
Tognazzini Bruce
Hjerpe Richard
Laneau Ronald
McDermott & Will & Emery
Sun Microsystems Inc.
LandOfFree
Method and apparatus for eyetrack—mediated downloading does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for eyetrack—mediated downloading, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for eyetrack—mediated downloading will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2934984