Computer graphics processing and selective visual display system – Display driving control circuitry – Controlling the condition of display elements
Reexamination Certificate
1999-04-05
2001-07-24
Huynh, Ba (Department: 2173)
Computer graphics processing and selective visual display system
Display driving control circuitry
Controlling the condition of display elements
C345S215000, C345S215000, C707S793000, C707S793000
Reexamination Certificate
active
06266057
ABSTRACT:
BACKGROUND AND SUMMARY OF THE INVENTION
This invention relates to an information processing system which operators can use to directly handle images displayed on a display unit inserted into a desk and more particularly to an information processing system suited for instinctly handling images using the actual objects placed on the desk. The terms “actual object” and “physical object” have the same meaning in this application unless otherwise indicated.
As written in “(1) Progress in Human Interface Vol. 2, PP. 6572 (1993)”, a data glove used to interact with a computer is available as the conventional technique for improving the computer man-machine interface. Also, in the conventional techniques written in (2) Proceedings of UIST'91, “The Digital Desk Calculator: Tangible Manipulation on a Desk Top Display”, the hand of an operator without a glove is detected by a camera, the movement of hand and sheets of paper are monitored, display objects are moved, and “cut & copy” is performed by selecting a character string on a sheet of paper with a finger. In addition, in the conventional technique written in (3) “
5-19958
Gazette”, the hand of an operator is photographed by a camera and used for pointing.
As a prior art of a method for making information handled in an information processing unit corresponding to an object, there is a system named XAX which is indicated in (4) “Bridging the Paper and Electronic Worlds: The Paper User Interface” (Proceedings of INTERCHI 1993, pp. 507 to 512). The object used in this system is a paper on which a discrimination pattern is printed. A user inputs the content written on such a paper into the system using a scanner or FAX. The system recognizes the pattern printed on the paper and discriminates what kind of paper it is. As a discrimination result, the system understands that the paper is a paper on which the content of a file held by the system is printed or the paper is a written order for instructing file retrieval to the system.
When the XAX system can discriminate the paper as a paper on which a file is printed, the system can indicate the original file to the user. From a viewpoint of the user, the printed paper corresponds to the original file and it can be understood that by indicating the printed paper, the original file is taken out. When the read paper is a written order for instructing file retrieval to the system, the system recognizes the information (for example, the file name and keyword) written on the paper and retrieves the corresponding file. The retrieval result is indicated to the user in a certain form. For example, a retrieval result list or the file content can be printed via FAX or a printer. From a viewpoint of the user, the paper (written order) to be used for retrieval corresponds to the retrieval result and it can be understood that by indicating the paper, the retrieval result is taken out.
In the conventional technique which uses the data glove, the man-machine interface is problematic because the operator feels incompatibility. The operator feels incompatibility because the hand of the operator is covered with the glove. Also, in the other two conventional techniques in which the movement of the hand is monitored by the camera and this is used for the man-machine interface, actual operations cannot be performed instinctly because the correspondence between operations of images and actual objects and the functions of instructions for various operations are not considered concretely.
The prior art mentioned above can retrieve information using a paper or give an instruction to an information processing unit but cannot change dynamically the correspondence between the paper and electronic information and limit the operation to the electronic information because it corresponds to the paper. For example, to increase information which can be taken out by a certain paper, it is necessary to additionally write the attribute (for example, the document name) of new information on the paper. Therefore, by this prior art, it is impossible to perform the aforementioned corresponding operation interactively by viewing the screen.
The prior art mentioned above indicates a concept of erasing information which is displayed on the screen using an object but does not give consideration to map and hold the object to each electronic information. Therefore, it is impossible to take out electronic information using an object or to change the attribute of electronic information via the operation to the object.
The present invention is directed toward providing an improved information processing system which establishes an environment in which the operator can interface with computers in a format similar to actual operation and actual objects can be used as man-machine interface parts.
According to one aspect of the present invention, improved processing is achieved by providing for an operator managed menu driven system for storing the physical object attributes of selected actual physical objects to be used. This feature allows for different operator hands, as well as different physical objects. Since actual physical objects to be used are also used in setting up and storing characteristic features to be compared during information processing, an especially user friendly man-machine interface environment is achieved.
According to preferred embodiments of the present invention, there are provided a method for recognizing the actual object from an image of a camera device; an attribute definition method for associating operations between the recognized actual object and the displayed image; and an attribute dealing method for comparing a characteristic point coordinate of the recognized actual object with the characteristic point coordinate of another actual object or that of the image displayed on the display surface and executing operations defined by the attribute definition method for the associated image when the comparison result is within a given value. This object is achieved in the display unit, the camera device that photographs the image displayed on the display unit and the actual object placed on the display unit or near it, and the information processor with the processor connected to the display unit and camera device.
Advantages of the invention are also achieved in part by recognizing a user gesture for holding or releasing the image or actual object displayed on the above-mentioned display surface in a gesture recognition method. This function is achieved in the information processor which provides the method for recognizing the hand gesture of the operator from the image input from the camera device and operates the image on the display surface according to the result of this gesture recognition method.
Assume that a facsimile is placed on a desk as an actual object and a data file is opened on the display unit as an image. When the operator moves this data file in the facsimile direction to be placed within a given distance, the information processor performs facsimile send processing on the assumption that the data file is sent in the facsimile. This enables not only an actual object to be used in the man-machine interface but also an instinctive operating environment to be established. In addition, the operator can directly handle an object in a sense of holding by recognizing the image or actual object hold and release gestures as the hand gesture of the operator without specifying a menu and an icon. The man-machine interface is thus remarkably improved.
According to certain preferred embodiments of the present invention, a system is provided for implementing the interactive operator use of objects to retrieve, read, change and file electronically stored information. The system includes a display device, a file storage means for storing electronic information, a discrimination means for discriminating an object indicated by a user, and a link storage means for storing link information which designates the correspondence between the aforementioned object and the electronic information stored in the afore
Arai Toshifumi
Kitamura Tadaaki
Kuzunuki Soshiro
Shojima Hiroshi
Crowell & Moring , L.L.P.
Hitachi , Ltd.
Huynh Ba
LandOfFree
Information processing system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Information processing system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Information processing system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2515147