Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
2000-07-12
2003-10-21
Couso, Yon J. (Department: 2625)
Image analysis
Applications
3-d or stereo imaging analysis
C345S426000
Reexamination Certificate
active
06636627
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to a light source direction estimating method and apparatus, wherein a direction of a light source is estimated from an image having been obtained by performing an image recording operation by use of a single light source or a plurality of light sources. This invention also relates to a recording medium, on which a program for causing a computer to execute the light source direction estimating method has been recorded and from which the computer is capable of reading the program.
2. Description of the Related Art
Techniques for combining a real image and a virtual image, such as a computer graphics (CG) image, to obtain a composed image giving no unnatural feeling are referred to as mixed reality techniques and are expected to play an important role in various fields, such as communication, traffic, and amusements. In order for a real image and a virtual image to be combined such that a composed image giving no unnatural feeling can be obtained, it is necessary for the real image and the virtual image to be combined with geometrical, optical, and temporal consistency. In particular, as for the optical consistency, if the direction of a light source and the intensity of light radiated out from the light source with respect to the real image and the direction of a light source and the intensity of light radiated out from the light source with respect to the virtual image do not coincide with each other, a composed image giving an unnatural feeling is obtained. Therefore, the optical consistency is important markedly. Accordingly, a technique for estimating a distribution of intensity of light radiated out from a light source in accordance with an image has been proposed in, for example, “Estimation of illumination distribution by using soft shadows” by I. Sato, et al., Computer Vision And Image Media, 110-3, Mar. 19, 1998, pp. 17-24.
With the technique proposed by Sato, et al., an intensity distribution of environmental illumination is estimated on the basis of a soft shadow of a real object. Specifically, the intensity distribution of environmental illumination is estimated by utilizing a shadow of an object, whose three-dimensional shape is known, the shadow appearing in an image. More specifically, relationship between brightness of a virtual light source and a signal value at a certain measuring point on the object is expressed with a determinant, an inverse matrix of the matrix in the determinant is calculated, and the intensity distribution of light radiated out from the light source is thereby estimated. In cases where the intensity distribution of light radiated out from the light source is estimated from the image with the proposed technique, the intensity distribution of light radiated out from a light source with respect to a virtual image can be set to be identical with the intensity distribution of light radiated out from the light source with respect to the real image, and therefore the real image and the virtual image can be combined to form a composed image giving no unnatural feeling.
However, with the technique proposed by Sato, et al., it is necessary for the position of the measuring point to be set at a position which is mathematically linearly independent and which allows calculation of the inverse matrix. It is necessary for the position of the measuring point to be found with manual operations. Therefore, the operations cannot be kept simple.
SUMMARY OF THE INVENTION
The primary object of the present invention is to provide a light source direction estimating method, with which a direction of a single light source is capable of being estimated easily.
Another object of the present invention is to provide a light source direction estimating method, with which directions of a plurality of light sources are capable of being estimated easily.
A further object of the present invention is to provide an apparatus for carrying out the light source direction estimating method.
A still further object of the present invention is to provide a recording medium, on which a program for causing a computer to execute the light source direction estimating method has been recorded and from which the computer is capable of reading the program.
The present invention provides a first light source direction estimating method, wherein an image is obtained, which image has been formed by performing an image recording operation on an object under a substantially single light source, the object containing an area, which has approximately uniform surface reflectivity and which exhibits diffuse reflection, and wherein a direction of the light source, under which the image recording operation was performed, is estimated in accordance with the image, the method comprising the step of:
estimating the direction of the light source., under which the image recording operation was performed, in accordance with signal values of an object image signal, which represents an object image embedded in the image having been obtained from the image recording operation, and relationship between a position of a three-dimensional body image, which is represented by a position-matched three-dimensional shape signal that has been obtained from position matching performed on the object image signal and a three-dimensional shape signal representing a three-dimensional shape of the object, and positions of virtual light sources having been set in a plurality of arbitrary directions.
The term “performing an image recording operation under a substantially single light source” as used herein means that the image recording operation is performed in a state approximately identical to the state in which it can be regarded that a single light source alone is utilized and the presence of other light sources can be ignored as in the cases where the image recording operation is performed under the sunlight in fine weather.
The term “three-dimensional shape signal representing a three-dimensional shape of an object” as used herein means the signal representing the shape of the object itself.
The term “position matching performed on an object image signal and a three-dimensional shape signal” as used herein means the operation for matching the object image signal and the three-dimensional shape signal such that the orientation, the center position, the size, and the like, of the object in the image, which has been obtained from the image recording operation, and the orientation, the center position, the size, and the like, of the three-dimensional shape of the object may coincide with each other.
The first light source direction estimating method in accordance with the present invention should preferably be modified such that a plurality of measuring points are set at an area of the three-dimensional body image, which is represented by the position-matched three-dimensional shape signal that has been obtained from the position matching, the area of the three-dimensional body image corresponding to the area of the object, which area has approximately uniform surface reflectivity and which exhibits diffuse reflection,
a calculation is made to find a cosine between a direction vector directed from each of virtual light sources, which have been set in a plurality of arbitrary directions, to each of the measuring points and a normal vector at each of the measuring points,
a relationship between the calculated cosines and signal values of the object image signal, which signal values correspond to the plurality of the measuring points, is plotted with respect to each of the directions of the virtual light sources, and
a direction of a virtual light source, which direction is associated with a plotted relationship that forms an approximately straight line, is estimated as the direction of the light source, under which the image recording operation was performed.
The direction vector can be calculated from the spatial coordinates of the virtual light source and the spatial coordinates of the measuring point. The normal vector can be calculated from th
Birch & Stewart Kolasch & Birch, LLP
Couso Yon J.
Fuji Photo Film Co.
LandOfFree
Light source direction estimating method and apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Light source direction estimating method and apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Light source direction estimating method and apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3140619