Data processing: database and file management or data structures – Database design – Data structure types
Reexamination Certificate
1998-12-16
2001-11-13
Lintz, Paul R. (Department: 2771)
Data processing: database and file management or data structures
Database design
Data structure types
C707S793000, C707S793000, C707S793000
Reexamination Certificate
active
06317740
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to the field of assigning keywords to media objects located in files stored in a database.
2. Description of the Related Art
With the explosive growth of information that is available through the World-Wide Web (“WWW”), it is becoming increasingly difficult for a user to find information that is of interest to him/her. Therefore, various search mechanisms that allow a user to retrieve documents of interest are becoming very popular. However, most of the popular search engines today are textual. Given one or more keywords, such search engines can retrieve WWW documents that have those keywords. Although most WWW pages have images, the current image search engines on the WWW are primitive.
There are two major ways to search for an image. First, a user can specify an image and the search engine can retrieve images similar to the specified image. Second, the user can specify keywords and all images relevant to the user specified keywords can be retrieved. The present inventor has been involved in the development of an image search engine called the Advanced Multimedia Oriented Retrieval Engine (AMORE). See S. Mukherjea et al, “Towards a Multimedia World-Wide Web Information Retrieval Engine,” Proceedings of the Sixth International World-Wide Web Conference, pages 177-188, Santa Clara, Calif., April 1997; and http.//www.ccrl.com/amore. AMORE allows the retrieval of WWW images using both of the techniques. In AMORE the user can specify keywords to retrieve relevant images or can specify an image to retrieve similar images.
The similarity of two images can be determined in two ways: visually and semantically. Visual similarity can be determined by image characteristics like shape, color and texture using image processing techniques. In AMORE, Content-Oriented Image Retrieval (COIR) is used for this purpose. See K. Hirata et al., “Media-based Navigation for Hypermedia Systems,” Proceedings of ACM Hypertext '93 Conference, pages 159-173, Seattle, Wash., November 1993. When a user wants to find images similar to a red car, COIR can retrieve pictures of other red cars. However, it may also be possible that the user is not interested in pictures of red cars, but pictures of other cars of similar manufacturer and model. For example, if the specified image is an Acura NSX, the user may be interested in other Acura NSX images. Finding semantically similar images (i.e. other images having the same or similar associated semantics) is useful in this example. Considering another example, a picture of a figure skater may be visually similar to the picture of an ice hockey player (because of the white background and similar shape), but it may not be meaningful for a user searching for images of ice hockey players. Finding semantically similar images will be useful in this example as well.
In order to find images which are semantically similar to a given image, the meaning of the image must be determined. Obviously this is not very easy. The best approach would be to assign several keywords to an image to specify its meaning. Manually assigning keywords to images would give the best result, but is not feasible for a large collection of images. Alternatively, the text associated with images can be used as their keywords. Unfortunately, unlike written material, most HyperText Markup Language (HTML) documents do not have an explicit caption. Therefore, the HTML source file must be parsed and only keywords “near” an image should be assigned to it. However, because the HTML page can be structured in various ways, the “nearness” is not easy to determine. For example, if the images are in a table, the keywords relevant to an image may not be physically near the image in the HTML source file. Thus, several criteria are needed to determine the keywords relevant to an image.
There are many popular WWW search engines, such as Excite (http://www.excite.com) and Infoseek (http://www.infoseek.com). These engines gather textual information about resources on the WWW and build up index databases. The indices allow the retrieval of documents containing user specified keywords. Another method of searching for information on the WWW is manually generated subject-based directories which provide a useful browsable organization of information. The most popular one is Yahoo (http://www.yahoo.com). However, none of these systems allow for image searching.
Image search engines for the WWW are also being developed. Excalibur's Image Surfer (http://isurf.yahoo.com) and WebSEEk (see S. Chang et al., “Visual Information Retrieval From Large Distributed Online Repositories,” Communications of the ACM, 40(12):63-71, December 1997) have built a collection of images that are available on the WWW. The collection is divided into categories (like automotive, sports, etc), allowing a user to browse through the categories for relevant images. Keyword searching and searching for images visually similar to a specified image are also possible. Alta Vista's Photo Finder (http://image.altavista.com) also allows keyword and visually similar image searches. However, semantically similar searching is not possible in any of these systems.
WebSeer is a crawler that combines visual routines with textual heuristics to identify and index images on the WWW. See C. Frankel et al., “WebSeer: An Image Search Engine for the World-Wide Web,” Technical Report 96-14, University of Chicago, Computer Science Department, August 1996. The resulting database is then accessed using a text-based search engine that allows users to describe the image that they want using keywords. The user can also specify whether the desired image is a photograph, animation, etc. However, the user can not specify an image and find similar images.
Finding visually similar images using image processing techniques is a developed research area. Virage (see J. R. Bach et al., “The Virage Image Search Engine: An Open Framework for Image Management,” Proceedings of the SPIE—The International Society for Optical Engineering: Storage and Retrieval for Still Image and Video Databases IV, San Jose, Calif., February 1996) and QBIC (see M. Flickner et al., “Query by Image and Video Content: The QBIC System,” IEEE Computer, 28(9):23-48, September 1995) are systems for image retrieval based on visual features, which consist of image primitives, such as color, shape, or texture and other domain specific features. Although they also allow keyword search, the keywords need to be manually specified and there is no concept of semantically similar images.
Systems for retrieving similar images by semantic content are also being developed. See A. Smeaton et al., “Experiments on using Semantic Distances between Words in Image Caption Retrieval,” Proceedings of the ACM SIGIR '96 Conference on Research and Development in Information Retrieval, pages 174-180, Zurich, Switzerland, August 1996 and Y. Aslandogan et al., “Using Semantic Contents and WordNet in Image Retrieval,” Proceedings of the ACM SIGIR '97 Conference on Research and Development in Information Retrieval, pages 286-295, Philadelphia, Pa., July 1997. However, these systems also require that the semantic content be manually associated with each image. For these techniques to be practical for the WWW, automatic assignment of keywords to the images is essential.
Research looking into the general problem of the relationship between images and captions in a large photographic library like a newspaper archive has been undertaken. See R. Srihari, “Automatic Indexing and Content-based Retrieval of Captioned Images,” IEEE Computer, 28(9):49-56, September 1995 and N. Rowe, “Using Local Optimality Criteria for Efficient Information Retrieval with Redundant Information Filters,” ACM Transactions on Information Systems, 14(2):138-174, March 1996. These systems assume that captions have already been extracted from the pictures, an assumption not easily applicable to the WWW.
Various techniques have been developed for assigning keywords to
Cho Junghoo
Mukherjea Sougata
Lintz Paul R.
NEC USA Inc.
Sughrue. Mion, Zinn, Macpeak & Seas, PLLC
LandOfFree
Method and apparatus for assigning keywords to media objects does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for assigning keywords to media objects, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for assigning keywords to media objects will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2592563