Image analysis – Pattern recognition – Feature extraction
Reexamination Certificate
1999-12-14
2003-12-30
Patel, Jayanti K. (Department: 2621)
Image analysis
Pattern recognition
Feature extraction
C705S001100, C358S527000
Reexamination Certificate
active
06671405
ABSTRACT:
FIELD OF THE INVENTION
The invention relates generally to the field of image processing, and in particular to the field of image assessment and understanding.
BACKGROUND OF THE INVENTION
Image assessment and understanding deal with problems that are easily solved by human beings given their intellectual faculties but are extremely difficult to solve by fully automated computer systems. Image understanding problems that are considered important in photographic applications include main subject detection, scene classification, sky and grass detection, people detection, automatic detection of orientation, etc. In a variety of applications that deal with a group of pictures, it is important to rank the images in terms of a logical order, so that they can be processed or treated according to their order. A photographic application currently of interest is automatic albuming, where a group of digital images are automatically organized into digital photo albums. This involves clustering the images into separate events and then laying out each event in some logical order, if possible. This order implies at least some attention to the relative content of the images, i.e., based on the belief that some images would likely be preferred over others.
A number of known algorithms, such as dud detection, event detection and page layout algorithms, are useful in connection with automatic albuming applications. Dud detection addresses the elimination, or de-emphasis, of duplicate images and poor quality images, while event detection involves the clustering of images into separate events by certain defined criteria, such as date and time. Given a set of images that belong to the same event, the objective of page layout is to layout each event in some logical and pleasing presentation, e.g., to find the most pleasing and space-efficient presentation of the images on each page. It would be desirable to be able to select the most important image in the group of images, e.g., the one that should receive the most attention in a page layout.
Due to the nature of the image assessment problem, i.e., that an automated system is expected to generate results that are representative of high-level cognitive human (understanding) processes, the design of an assessment system is a challenging task. Effort has been devoted to evaluating text and graphical data for its psychological effect, with the aim of creating or editing a document for a particular visual impression (see, e.g., U.S. Pat. Nos. 5,875,265 and 5,424,945). In the '265 patent, a system analyzes an image, in some case with the aid of an operator, to determine correspondence of visual features to sensitive language that is displayed for use by the operator. The difficulty in this system is that the visual features are primarily based on low level features, i.e., color and texture, that are not necessarily related to image content, and a language description is difficult is to use for relative ranking of images. The '945 patent discloses a system for evaluating the psychological effect of text and graphics in a document. The drawback with the '945 patent is that it evaluates the overall visual impression of the document, without regard to its specific content, which reduces its usefulness for developing relative ranking. Besides their complexity and orientation toward discernment of a psychological effect, these systems focus on the analysis and creation of a perceptual impression rather than on the assessment and utilization of an existing image.
SUMMARY OF THE INVENTION
The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, a method is disclosed for assessing an image with respect to certain features, wherein the assessment is a determination of the degree of importance, interest or attractiveness of an image. First, a digital image is obtained corresponding to the image. Then one or more quantities are computed that are related to one or more features in the digital image, including one or more features pertaining to the content of the digital image. The quantities are processed with a reasoning algorithm that is trained on the opinions of one or more human observers, and an output is obtained from the reasoning algorithm that assesses the image. In a dependent aspect of the invention, the features pertaining to the content of the digital image include at least one of people-related features and subject-related features. Moreover, additional quantities may be computed that relate to one or more objective measures of the digital image, such as colorfulness or sharpness.
From another aspect, the invention may be seen as assessing the emphasis or appeal of an image, as hereinafter defined. From this perspective, for both appeal and emphasis assessment, self-salient image features are calculated, such as:
a. People related features: the presence or absence of people, the amount of skin or face area and the extent of close-up based on face size.
b. Objective features: the colorfulness and sharpness of the image.
c. Subject related features: the size of main subject and the goodness of composition based on main subject mapping.
While the above-noted features are adequate for emphasis assessment, it is preferable that certain additional relative-salient image features are considered for appeal assessment, such as:
a. The representative value of each image in terms of color content.
b. The uniqueness of the picture aspect format of each image.
An assessment of an image is obtained with a reasoning engine, such as a Bayesian network, which accepts as input the above-noted features and is trained to generate image assessment values. This assessment may be an intrinsic assessment for individual images, in which case the self-salient features are processed by a Bayesian network trained to generate the image appeal values, or the assessment may be a relative assessment for a group of images, in which case the self-salient and, optionally, the relative-salient features are processed by a Bayesian network trained to generate image emphasis values.
The advantage of the invention lies in its ability to perform an assessment of one or more images without human intervention. In a variety of applications that deal with a group of pictures, such as automatic albuming, such an algorithmic assessment enables the automatic ranking of images in terms of their logical order, so that they can be more efficiently processed or treated according to their order.
These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiments and appended claims, and by reference to the accompanying drawings.
REFERENCES:
patent: 4396903 (1983-08-01), Habicht et al.
patent: 5418895 (1995-05-01), Lee
patent: 5424945 (1995-06-01), Bell
patent: 5855482 (1999-01-01), Remer
patent: 5875265 (1999-02-01), Kasao
patent: 5983218 (1999-11-01), Syeda-Mahmood
patent: 6282317 (2001-08-01), Luo et al.
U.S. patent application Ser. No. 09/112,661, filed Jul. 9, 1998, Hsien-Che Lee et al.
U.S. patent application Ser. No. 09/223,860, filed Dec. 31, 1998, Jiebo Luo et al.
U.S. patent application Ser. No. 09/274,645, filed Mar. 23, 1999, Andreas E. Savakis et al.
“A computational model of print-quality perception” by T. Tehan, R.C. Thompson, A. Manning. Expert Systems with Applications, No. 1999, Elsevier, UK, vol. 17, No. 4, pp. 243-256.
“Indoor-Outdoor Image Classification” by Martin Szummer and Rosalind W. Picard. Content-Based Access of Image and Video Database, 1998. Proceedings, 1998 IEEE International Workshop on Bombay, India, Jan. 3, 1998, Los Alamitos, CA. IEEE Comput. Soc. US, 1998, pp. 42-51.
Etz Stephen
Savakis Andreas E.
Azarian Seyed
Eastman Kodak Company
Patel Jayanti K.
Woods David M.
LandOfFree
Method for automatic assessment of emphasis and appeal in... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for automatic assessment of emphasis and appeal in..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for automatic assessment of emphasis and appeal in... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3140902