Computer graphics processing and selective visual display system – Display peripheral interface input device – Cursor mark position control device
Reexamination Certificate
1999-08-13
2001-10-02
Liang, Regina (Department: 2674)
Computer graphics processing and selective visual display system
Display peripheral interface input device
Cursor mark position control device
C345S158000
Reexamination Certificate
active
06297804
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to a pointing apparatus, and more particularly to a pointing apparatus which allows users to point to an object and to specify coordinates while moving freely between the virtual space displayed on a display and the real world space in which real objects exist.
First, an overview of the present invention will be described. The pointing apparatus according to the present invention allows the user to point not only to objects in the virtual world displayed on the display but also to objects in the real world, such as documents or three-dimensional objects. That is, the present invention relates to an user interface, called “Augmented Reality”, created by combining (or making continuously) the real world with the virtual world.
BACKGROUND OF THE INVENTION
The description of the related art of the present invention begins with the description of “Augmented Reality”, followed by the description of related known arts.
“Augmented Reality” (hereinafter called “AR”), a concept introduced in 1993, is described, for example, in Document (1), “Communications of the ACM” Vol. 36, No. 7. PP. 24-97.
In contrast to “Virtual Reality” in which all the real world objects are realized virtually, AR augments the real world with computers to realize an easy-to-understand interface.
Among the systems designed based on AR, “Digital Desk” and “InteractiveDESK” are related to the present invention. They are described below.
“Digital Desk” is an interactive desktop system proposed by Pierre Wellner of Rank Xerox corporation. The concept is detailed, for example, in Document (2) “Communications of the ACM” Vol. 36, No. 7, pp. 87-96. As shown in
FIG. 8
, this system has a projector
202
and a camera
201
above a work desk
203
. The projector
202
projects images, generated by a computer
200
, onto the work desk
203
(see a projected image
204
), and the camera
201
takes pictures of hand motions, hand-written characters, hand-written pictures, and documents. The user can perform tasks interactively while conversing with the computer
200
through the work desk
203
on which the images generated by the computer
200
and real documents are mixed. The so-called real world interface is thus realized. Applications are available which illuminate, and display marks on, the numbers in a document pointed by a finger and paste those numbers on a desktop calculator created by the projected light.
Patent documents related to “Digital Desk” of Rank Xerox corporation include Japanese Patent Publication Kokai JP-A No. Hei 7-168949 (Title of invention “Interactive Copier”) and Japanese Patent Publication Kokai JP-A No. Hei 9-237349 (Title of invention “Interactive Desktop System”).
Of these patent publications, Japanese Patent Publication Kokai JP-A No. Hei 7-168949 (Title of invention “Interactive Copier”) discloses a configuration, comprising a a video projector displaying virtual world information onto a work desk and a camera monitoring the work desk. The disclosed configuration is designed to allow the user to interact with information projected by the video projector and to create documents or pictures on actual paper in the manual operation mode. Thus it enables an interaction between virtual world images, projected by the video projector, and real world paper on the work desk to be performed intuitively in the manual operation mode.
Japanese Patent Publication Kokai JP-A No. Hei 9-237349 (Title of invention “Interactive Desktop System”) also discloses a configuration comprising a video projector displaying virtual world information onto a work desk and a camera monitoring the work desk. This configuration allows the user to request the camera to take the picture of real objects, to display via the computer images taken by the camera onto the video projector, and to copy and paste real objects such as documents while sitting at the work desk. In this configuration, a mouse is used to specify an area to be photographed, and the cursor which is a pointing position indicator is projected on the work desk as the display information from the video projector.
The concept and the features of “InteractiveDESK”, a system which uses the entire desk as the interactive environment, are detailed, for example, in Document (3), ITE Technical Report Vol. 19, No. 65, pp. 1-6.
FIG. 9
is a decigram showing an overall configuration of InteractiveDESK. As shown in
FIG. 9
, the InteractiveDESK comprises a work desk
304
whose working surface is a large display through which data may be entered with a pen, an upright display
302
provided in front of the user, a camera
301
monitoring the top of the work desk, and other components such as a keyboard
306
and a mouse
307
. The camera
301
recognizes user actions or the positions of documents on the desk and helps the user with his or her work. For example, it switches input means (keyboard
306
, mouse
307
, or pen) between two types of displays, desk built-in type and the upright type, and extracts electronic information on the paper documents on the desk.
Japanese Patent Publication Kokai JP-A No. Hei 6-266498 (Title of Invention: Data Processing System) is available for a patent application publication relating “InteractiveDESK”. Japanese Patent Publication Kokai JP-A No. Hei 6-266498 mentioned above discloses a method for determining a position at which desired information is to be displayed based on at least one of the three: user status, display surface status, and display status.
For example, when documents or some other objects are placed on the display screen, this system displays information at a position where there is no such objects to prevent real world objects such as documents from interfering with the virtual world such as display information. The publication also discloses a method for switching the display with the above-described input means.
Other prior arts similar to “Digital Desk” or “InteractiveDESK” include Japanese Patent Publication Kokai JP-A No. Hei 9-319556 (Title of Invention: Data Processing Device). This publication discloses a system comprising a work desk, a projector projecting images on the work desk, and a camera monitoring the desk and a technology for interacting with real world objects such as documents on the desk. However, there is no distinct difference between the system disclosed in this publication and those systems described above such as “Digital Desk” or “InteractiveDESK”. Thus, explanation on the system of Japanese Patent Publication Kokai JP-A No. Hei 9-319556 is obtained herein.
The technologies for pointing to real world objects with laser beam, such as those disclosed in Japanese Patent Publication Kokai JP-A No. Hei 9-46776 (Title of Invention: Remote Indication Device and Method) and Japanese Patent Publication Kokai JP-A No. Hei 5-83705 (Title of Invention: Still Picture Input Device) are outlined below.
FIG. 10
shows the configuration of the remote indication device proposed by Japanese Patent Publication Kokai JP-A No. Hei 9-46776. An indication device
400
and a remote unit
406
are connected via a network
405
. The remote unit
406
takes a picture of a document
407
at a remote location and sends the image to the indication unit
400
over the network
405
. The indication device
400
displays the image of the document sent from the remote unit
406
onto a display
401
. The presenter points to a desired position on the remotely-located document on the display
401
with a keyboard
402
or a mouse
404
connected to the indication device
400
. The pointing information is sent to the remote unit
406
via the network
405
. In response to the pointing information sent from the indication device
400
, the remote unit
406
projects laser beam onto the actual position on the document
407
corresponding to the position pointed by the presenter on the display
401
.
FIG. 11
shows the configuration of the still image input device disclosed in Japanese Patent Publication Kokai JP-A No. Hei 5-83705 described above. This device is a pointing apparat
Foley & Lardner
Liang Regina
NEC Corporation
LandOfFree
Pointing apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Pointing apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Pointing apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2567293