Television – Camera – system and detail – With object or scene illumination
Reexamination Certificate
1999-10-15
2001-10-16
Garber, Wendy R. (Department: 2712)
Television
Camera, system and detail
With object or scene illumination
C382S162000
Reexamination Certificate
active
06304294
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to the field of digital imaging. More particularly, the present invention relates to the field of illumination estimation of digital images.
BACKGROUND OF THE INVENTION
A perceived color of an object is determined by three main factors: the physical properties of the object, the actual illumination of the object by an illumination source, and the sensitivity of the input device. Changing any one of these three factors may possibly change the perceived color of the object. For example, assuming that the input device is a digital camera, the illumination source is natural sunlight, and a white piece of paper is the viewed object, the digital camera will perceive this piece of paper as being white in color. However, if the illumination source is changed from natural sunlight to fluorescent lighting and no other transformations on the raw data are performed, the digital camera will perceive the color of the object as off-white; this piece of paper will be perceived by the camera as being different colors when illuminated by different types of illumination sources.
In contrast to the digital camera, a human observer perceives this white piece of paper as being white whether the natural sunlight or the fluorescent lighting serves as the illumination source. Within certain limits, the human eye compensates for the varying types of illumination sources and perceives an object illuminated by different types of illumination sources as having the same color.
In general, white balancing algorithms aid digital cameras in reproducing color images which more closely mimic the perception of the human eye. In this prior example, white balancing algorithms are utilized in digital cameras to allow the resultant digital image of an object to be perceived as the same color whether illuminated by fluorescent lighting or natural sunlight. White balancing algorithms emphasize color reproduction which is realistic to the human eye. Typically, the first step in a white balancing algorithm is to estimate the scene illumination from the digital image.
There are a variety of different techniques for estimating a scene illumination from an image. These techniques include gray-world algorithm, estimation based on specular reflectance, neural network, statistical approaches, extra sensing devices, and reflectance in two-dimensional space. The gray-world algorithm assumes that the spatial average of surface reflectance within the scene is achromatic. Accordingly, the gray-world algorithm assumes that the average pixel value of a digital image of a scene corresponds to the color of the illumination. Often times this assumption is not true and, as a result, the gray-world algorithm fails. The estimation based on specular reflectance technique is similar to the gray-world algorithm, because this estimation based on reflectance relies on a certain percentage of light reflected in a specular manner. If there are not enough highlights, the estimation based on specular reflectance fails to accurately estimate scene illumination for white balancing.
The neural network technique and the statistical approach require considerable training to accurately estimate the illumination. As the neural network and the statistical approach gain experience by more training, they are able to increase their accuracy of illumination estimation. However, providing this training is expensive during both development and implementation of the neural network and the statistical approach. Even with this extensive training, the illumination of certain images may not be approximated accurately enough by neither the neural network technique nor the statistical approach.
As a way to increase accuracy of illumination estimation, extra sensing devices are sometimes utilized to gather additional information otherwise not captured with a single sensor. However, these extra sensor devices add additional manufacturing costs and mechanical complexity to the camera device.
Additional methods of estimating the illumination of the image are based on linear decomposition of the surface reflectance and the spectral power distribution of the illumination. Under the assumption that the surface reflectance lies in a two-dimensional space, and the illumination lies in a three-dimensional space, on exact solution for the illumination can be calculated from a three channel image. However, utilizing a two-dimensional space for the reflectance does not ensure that the exact solution for the illumination is necessarily the correct solution. For example, the an exact representation of the actual surface reflectance can be outside the boundary of a two-dimensional space. Accordingly, when the correct surface reflectance level cannot be accurately represented in this two-dimensional space, this method will not provide an optimal estimation of the illumination of the scene.
SUMMARY OF THE INVENTION
The apparatus for and method of estimating the illumination closely approximates an illumination level of a corresponding image. This apparatus for and method of estimating the illumination utilizes appropriate linear models to represent the illumination spectral power distribution of the illumination and the spectral surface reflectances. The linear model of the illumination spectral power distribution of the illumination, L(n), is represented by:
L
(
n
)
=
∑
m
=
1
D
1
α
m
B
m
(
n
)
The linear model of the surface reflectance function, R(n), is represented by:
R
k
(
n
)
=
∑
m
=
1
D
2
β
k
,
m
C
m
(
n
)
By utilizing these linear models, the illumination of the image is capable of being closely approximated by the following equation and corresponding constraint:
min
a
m
,
β
k
,
m
∑
k
∑
j
&LeftDoubleBracketingBar;
f
k
,
j
-
[
∑
n
S
j
(
n
)
×
(
∑
m
α
m
B
m
(
n
)
)
×
(
∑
m
β
k
,
m
C
m
(
n
)
)
]
&RightDoubleBracketingBar;
L
(
n
)
≥
0
and
0
≤
R
k
(
n
)
≤
1
the color value at one position in the image. By applying standard minimization techniques to the above equation and corresponding constraint, the unknown spectral illumination weighting coefficients (&agr;
m
) and reflectance weighting coefficients (&bgr;
k,m
) are estimated. These estimated illumination weighting coefficients (&agr;
m
) and reflectance coefficients (&bgr;
k,m
) calculated by the above equation and corresponding constraint are not exact solutions but are approximations. By utilizing the approximations of &agr;
m
and &bgr;
k,m
, the illumination corresponding to the image is also a carefully estimated approximation and obviates the need for an exact solution for the illumination level. By overcoming the constraints, necessary for calculating an exact solution for the illumination level of the scene, the estimated illumination has a wider degree of freedom than an exact solution and is capable of being closer to the real illumination.
REFERENCES:
patent: 4648051 (1987-03-01), Wandell et al.
patent: 4685071 (1987-08-01), Lee
patent: 4992963 (1991-02-01), Funt et al.
patent: 5636143 (1997-06-01), Takahashi
patent: 5907629 (1999-05-01), Funt et al.
patent: 6038339 (2000-03-01), Hubel et al.
Jian Ho, Brian V. Funt, and Mark S. Drew, “Separating a Color Signal into Illumination and surface Reflectance Components: Theory and Applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence. vol. 12. No. 10., pp. 966-977, Oct. 1990.*
G.D. Finlayson, B. V. Funt and K. Barnard, “Color Constancy Under Varying Illumination,” 0-8186-7042-8/95 $4.00 ©to Aug. 1995 IEEE, pp. 720-725.*
Brian Funt, Vlad Cardei and Kobus Barnard, “Learning Color Constancy,” School of Computing Science, Simon Fraser University, Vancouver, Canada, The Fourth Color Imaging Conference: Color Science, Systems and Applications, pp. 58-60.
“Probabilistic Color Constancy”; This document is drawn from a book chapter by D'Zmura, Iverson & Sing
Tao Bo
Tastl Ingeborg
Card Ronald C.
Fujii Harold T.
Garber Wendy R.
Nguyen Luong
Sony Corporation
LandOfFree
Apparatus for and method of estimating the illumination of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Apparatus for and method of estimating the illumination of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Apparatus for and method of estimating the illumination of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2574565