Determining orientation of images containing blue sky

Details Determining orientation of images containing blue sky Determining orientation of images containing blue sky

1999-11-29

2003-01-28

Wu, Jingge (Department: 2623)

Image analysis

Color image processing

Pattern recognition or classification using color

C382S164000, C382S295000, C382S225000

Reexamination Certificate

active

06512846

ABSTRACT:

BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates generally to the field of digital image processing and digital image understanding, and more particular to a system for determining the orientation in photographic and other similar images containing sky.
2. Description of the Related Art
Digital images may be acquired through a number of means, including scanning a picture captured on negative films or direct downloading from a digital camera. A photographer can take a picture while holding the camera in any of the possible ways (upright, upside down, rightside up, or leftside up). In addition to that, the film may be loaded from left to right or from right to left, resulting in unpredictable image orientation. It is beneficial to have an automatic way of sensing image orientation to avoid tedious reorientation of an image, which would otherwise need to be performed by an operator. It is also an efficiency issue for mass image processing in commercial systems.
Sky is among the most important subject matters frequently seen in photographic images. Detection of sky can often facilitate a variety of image orientation, understanding, enhancement, and manipulation tasks. Sky is a strong indicator of an outdoor image for scene categorization (e.g., outdoor scenes vs. indoor scenes, picnic scenes vs. meeting scenes, city vs. landscape, etc.). See, for example M. Szummer and R. W. Picard, “Indoor-Outdoor Image Classification,” in
Proc. IEEE Intl. Workshop on Content
-
based Access of Image and Video Database,
1998 and A. Vailaya, A. Jain, and H. J. Zhang, “On Image Classification: City Vs. Landscape,” in
Proc. IEEE Intl. Workshop on Content
-
based Access of Image and Video Database,
1998 (both of which are incorporated herein by reference).
The most prominent characteristic of sky is its color, which is usually light blue when the sky is clear. Such a characteristic has been used to detect sky in images. For example, U.S. Pat. No. 5,889,578, entitled “Method And Apparatus For Using Film Scanning Information To Determine The Type And Category Of An Image” by F. S. Jamzadeh, (which is incorporated herein by reference) mentions the use of color cue (“light blue’”) to detect sky without providing further description.
In a work by Saber et al. (E. Saber, A. M. Tekalp, R. Eschbach, and K. Knox, “Automatic Image Annotation Using Adaptive Color Classification,” CVGIP:
Graphical Models and Image Processing
, vol. 58, pp. 115-126, 1996, incorporated herein by reference), color classification was used to detect sky. The sky pixels are assumed to follow a 2D Gaussian probability density function (PDF). Therefore, a metric similar to the Mahalonobis distance is used, along with an adaptively determined threshold for a given image, to determine sky pixels. Finally, information regarding the presence of sky, grass, and skin, which are extracted from the image based solely on the above-mentioned color classification, are used to determine the categorization and annotation of an image (e.g., “outdoor’”, “people”). Note that Saber does not deal with determination of image orientation, which is assumed known.
U.S. Pat. No. 5,642,443, entitled, “Whole Order Orientation Method And Apparatus” by Robert M. Goodwin, (which is incorporated herein by reference) uses color and (lack of) texture to indicate pixels associated with sky in the image. In particular, partitioning by chromaticity domain into sectors is utilized by Goodwin. Pixels with sampling zones along the two long sides of a non-oriented image are examined. If an asymmetric distribution of sky colors is found, the orientation of the image is estimated. The orientation of a whole order of photos is determined based on estimates for individual images in the order. For the whole order orientation method in Goodwin to be successful, a sufficiently large group of characteristics (so that one with at least an 80% success rate is found in nearly every image), or a smaller group of characteristics (with greater than a 90% success rate—which characteristics can be found in about 40% of all images) is needed. Therefore, with Goodwin, a very robust sky detection method for any given image is not required.
For recognizing the orientation of an image, knowledge of sky and its orientation may indicate the image orientation for outdoor images. Contrary to the common belief, a sky region is not always at the top of an image. In many cases, a sky region may touch more than the top border of an image.
Problems to be Solved by the Invention
A major drawback of conventional techniques is that they utilized the location of objects (e.g., sky) to determine orientation. Those conventional systems which identify sky in images suffer from the disadvantage that the sky may not always appear at the top of the image and may also touch more than the top border of an image. Further, conventional techniques cannot differentiate sky from other similarly colored or textured subject matters, such as a blue wall, a body of water, a blue shirt, and so on. Failure to reliably detect the presence of sky, in particular false positive detection, and an inadequate assumption that the location of the sky indicates the image orientation, may lead to failures in orientation process.
SUMMARY OF THE INVENTION
The invention provides a robust image orientation process that identifies sky based on color hue classification, texture analysis, and physics-motivated sky trace analysis. The invention utilizes hue color information to select bright, sky colored pixels and utilizes connected component analysis to find potential sky regions. The invention also utilizes gradient to confirm that sky regions are low in texture content and segments open space, defined as smooth expanses, to break up adjacent regions with similar sky color beliefs but dissimilar sky colors. The invention further utilizes gradient to determine the zenith-horizon direction and uses a physics-motivated sky trace signature to confirm the determined orientation indeed comes from a valid sky region (as opposed to a merely “sky-colored” region).
More specifically, the invention can take the form of a method, image recognition system, computer program, etc., for determining image orientation. The invention classifies potential sky pixels in the image by color, identifies spatially contiguous regions of the potential sky pixels, identifies actual sky regions by eliminating ones of the spatially contiguous regions that have a texture above a predetermined texture threshold, computes desaturation gradients of the actual sky regions, classifies the image as one of portrait and landscape based on average horizontal and vertical desaturation gradient values of pixels within each of the actual sky regions, determines orientation of the image based on a polarity of the average horizontal and vertical desaturation gradients, and confirms that the actual sky regions are true sky regions by comparing the desaturation gradients with a predetermined desaturation gradient for sky.
The invention also maintains only ones of the spatially contiguous regions as the actual sky regions which have substantially dissimilar average horizontal and vertical gradients, have a color distribution which is consistent with a predetermined color distribution for sky, and contact a border of the image.
The classifying of the image as one of portrait and landscape includes classifying the image as the portrait if an absolute value of the average horizontal desaturation gradient is greater than an absolute value of the average vertical desaturation gradient and classifying the image as the landscape if an absolute value of the average horizontal desaturation gradient is less than an absolute value of the average vertical desaturation gradient. The desaturation gradients comprise desaturation gradients for red, green and blue trace components of the image.
The predetermined desaturation gradient for sky comprises, from horizon to zenith, a decrease in red and green light trace components and a substantially constant blue light trace c

Affiliated with

Also associated with

Rating

Determining orientation of images containing blue sky does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Determining orientation of images containing blue sky, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Determining orientation of images containing blue sky will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3024686