Data processing: speech signal processing – linguistics – language – Linguistics – Natural language
Reexamination Certificate
1998-12-08
2001-05-15
Thomas, Joseph (Department: 2644)
Data processing: speech signal processing, linguistics, language
Linguistics
Natural language
C707S793000, C707S793000
Reexamination Certificate
active
06233547
ABSTRACT:
FIELD OF THE INVENTION
The invention relates generally to the field of retrieval of multi-media objects such as still images, videos, graphics, computer generated graphics, drawings and the like, and specifically, to retrieving multi-media objects using a natural language, such as English, that includes anaphonic phrases or sentences.
BACKGROUND OF THE INVENTION
Multi-media objects carry a great deal of information and as multi-media technology is growing, there has been an increasing demand for a system that allows users to easily describe, archive, search and retrieve these multi-media objects. Some conventional methods and their limitations are described as follows.
In the past, people have used shoe boxes, albums and the like to archive images and then search and retrieval of these images is performed based on the user's memory. Stock agencies have used index cards to keep track of stock images and search and retrieval is done using personnel experiences and preferences. Such methods of archiving and retrieving images are difficult, time-consuming and expensive. These methods are also subjective in nature.
As computers became popular and more images were stored on-line, a keyword based approach was developed. Keyword representations can be created either manually or automatically. In the manual approach, a set of keywords are assigned to each image in the database. The keywords describe the image content of interest (i.e. objects, events, concepts, place, activities, etc.) The KODAK PICTURE EXCHANGE (KPX) uses this approach. A shortcoming of this approach is that a multi-media object, in this instance images, can not always be described by a disjoint set of keywords. This method of image retrieval depends on an exact match of a keyword used in the description and in the search, and the keywords used to describe/retrieve an image may change from user to user. Some incremental improvements can be made to this method by use of a thesaurus.
In the automatic approach, keywords are selected from within the document itself based on statistics pertaining to the relative frequency of word occurrence. This approach is more suitable for document retrieval applications where a large amount of text is available to obtain accurate statistics, such as in the area of newspaper article retrieval. Many text retrieval engines have been developed using this approach. However, in the case of images, the caption will typically be a sentence or two, which is not enough to extract meaningful statistics. Another limitation of the keyword-based technique for image retrieval is that only the words, and not the meaning or context, are taken into account. This makes this technique unsuitable for applications that contain a sparse amount of text to describe an image.
Images also can be searched and retrieved using image content analysis techniques. Image content attributes are defined using color, texture, shape and the like. Some of the existing systems that perform image content analysis are QBIC from IBM, and Virage from Virage Corporation. The drawback of this approach is it only allows for image similarity type search and retrieval, such as responding to queries of the form “Find me images like this one . . . ”.
The University of Buffalo has developed a system, PICTION, which uses natural language captions to label human faces in an accompanying newspaper photograph. A key component of the system is the utilization of spatial and characteristic constraints (derived from captions) in labeling face candidates (generated by a face locator). The system is limited to only identifying faces based upon the spatial constraints defined in the caption, for example “John Doe is to the left of Jane Doe . . . ”.
Anil Chakravarthy at MIT has developed a program as part of his thesis “Information Access and Retrieval with Semantic Background Knowledge” for retrieving captions of pictures and video clips using natural language queries. This thesis presents a limited framework for structured representation through the incorporation of semantic knowledge. However, the program only accepts images accompanied by well-formed single sentence description. Queries also need to be well-formed single sentence descriptions.
U.S. Pat. No. 5,493,677 discloses a natural language archival and retrieval system for images. This patent discloses inputting a search query in a natural language and then searching for archived images. It identifies name, location and noun phrases from the query; other words are eliminated. For example, prepositions are not used for further processing. This eliminates the context of some sentences and may give inaccurate results during retrieval, for example, the difference between the two phrases, “A man on a horse.” and “A man and a horse.” In addition, when inputting information that is to be associated with an image into the database, it has to be specified in a standardized form. The user is involved for part-of-speech disambiguation and word-sense disambiguation. This is time consuming and labor intensive.
Consequently, a need exists for a smart retrieval system to eliminate the above-described drawbacks.
SUMMARY OF THE INVENTION
The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, the invention resides in a computer program product for retrieving multi-media objects using a natural language containing a pronoun, comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) receiving a query in the natural language containing the pronoun; (b) determining the pronoun in the query; (c) determining whether either a phrase or sentence containing the pronoun conforms to a predetermined phrase structure; (d) determining a noun or noun phrase to which the pronoun refers based on step (c); and (e) processing the query based on step (d).
The above and other objects of the present invention will become more apparent when taken in conjunction with the following description and drawings wherein identical reference numerals have been used, where possible, to designate identical elements that are common to the figures.
ADVANTAGEOUS EFFECT OF THE INVENTION
The present invention has the advantage of identifying antecedent basis for pronouns in search queries, such as in image retrieval and image captions.
REFERENCES:
patent: 5265014 (1993-11-01), Haddock et al.
patent: 5386556 (1995-01-01), Hedin et al.
patent: 5493677 (1996-02-01), Balogh et al.
patent: 5895464 (1999-04-01), Bhandari et al.
patent: 5963940 (1999-10-01), Liddy et al.
patent: 6026388 (2000-02-01), Liddy et al.
patent: 6076051 (2000-06-01), Messerly et al.
patent: 6101492 (2000-08-01), Jacquemin et al.
Barnett, et al., “Knowledge and Natural Language Processing”, Communications of the ACM, vol. 33, No. 8, p. 50 (22)—DIALOG File 275, Acc. No. 01372855, Aug. 1990.
Eastman Kodak Company
Thomas Joseph
Woods David M.
LandOfFree
Computer program product for retrieving multi-media objects... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Computer program product for retrieving multi-media objects..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Computer program product for retrieving multi-media objects... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2563053