Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
1998-02-25
2001-01-30
Bella, Matthew (Department: 2721)
Image analysis
Applications
3-d or stereo imaging analysis
C382S154000, C382S201000, C382S291000, C382S285000, C348S094000, C348S139000, C345S419000
Reexamination Certificate
active
06181815
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to a subject image extraction device for extracting an image of a desired subject from an inputted image, and in particular, to a subject image extraction device which can extract an image of a subject which existed in a designated 3-D space domain at the moment of shooting.
DESCRIPTION OF THE PRIOR ART
Various kinds of methods are proposed for extracting an image of a desired subject or object from an inputted image, such as a method utilizing prestored information concerning the subject as disclosed in Japanese Patent Application Laid-Open No.HEI7-220095 (hereafter, referred to as ‘document No.1’), a method utilizing two or more image pickup devices (cameras) as disclosed in Japanese Patent Application Laid-Open No.HEI7-177423 (hereafter, referred to as ‘document No.2’) and Japanese Patent Application No.HEI8-271914 (hereafter, referred to as ‘document No.3’), etc.
In the method utilizing prestored subject information which is disclosed in the document No.1, information concerning the image pickup device such as the focal length of the image pickup device, the size of a pixel of a sensor of the image pickup device, the number of pixels of the sensor, etc. and information concerning the subject such as the size of the subject, the distances between the center point on the subject and parts of the subject, etc. are preliminarily recorded and stored in a storage unit. Then, the size of the subject in the image which has been picked up by the image pickup device is estimated using the information concerning the image pickup device and the subject. Meanwhile, a plurality of parts of the image which are considered to be candidates of the image of the subject are extracted from the image. Then, it is judged whether each of the candidate parts is the genuine subject image or not, by comparing the size of the candidate part with the estimated size of the subject in the image. And one candidate part whose size is the nearest to the estimated size of the subject in the image is extracted as the image of the subject.
In the method utilizing two or more image pickup devices which is disclosed in the document No.2, corresponding points in each image which have been picked up by the two or more image pickup devices are searched for, and disparities between the images with regard to the corresponding points are obtained using the difference of coordinates between the corresponding points in each image. Then, part of an image whose disparity between the images are large (i.e. parts which existed in the foreground near to the image pickup devices at the moment of shooting) are extracted as the image of the subject.
In the method utilizing two or more image pickup devices which is disclosed in the document No.3. The subject is shot in the centers of images by the two or more image pickup devices, and the aforementioned search of point correspondence (hereafter, referred to as ‘point correspondence search’) is executed from the center of the images. Then, outline point judgment (i.e. judgment whether a point in an image is a point on the outline of the subject or not) is executed using 3-D coordinates (corresponding to the point) which are calculated using the corresponding points obtained by the point correspondence search, and the outline points are used for extraction of the subject image. Even when false correspondence occurred in the point correspondence search and parts of the outline points dropped out from the extraction, such parts are restored by means of outline restoration assuming continuity.
However, in the method disclosed in the document No.1, the aforementioned information concerning the subject such as the size of the subject, the distances between the center point on the subject and parts of the subject, etc. and the information concerning the image pickup device such as the focal length of the image pickup device, the size of a pixel, the number of pixels, the distance between the subject and the image pickup device, etc. are needed to be prepared, and thus images of subjects whose information have not been prepared are impossible to be extracted from the image which has been picked up by the image pickup device. Further, according to the method, either the distance between the center point on the subject and part of the subject or the distance between the principle point of the image and part of the image is necessary as the information. In the case where the former distance is used, expensive devices such as an infrared irradiation unit etc. are necessary for obtaining the distance. And in the case where the latter distance is used, methods such as autofocusing, which does not operate stably under insufficient shooting conditions such as poor lighting etc., have to be employed, and thus stable extraction of the subject image is difficult.
In the method utilizing two or more image pickup devices and judging the depth using the disparity, the accuracy of subject image extraction is highly dependent on setting of a threshold value (in disparity) which is used to distinguish between the background and the foreground, and the setting of the disparity threshold value for accurate subject image extraction is very difficult for the operator of the subject image extraction device since the disparity is not a value which can be directly measured. The method of the document No.2 tries to resolve the problem by obtaining 3-D coordinates using the disparities. However, the method needs troublesome control or measurement of relative positions of the two or more image pickup devices, angles between the optical axes of the image pickup devices, etc. Further, according to the method, when false correspondence occurred in the point correspondence search, the part where the false correspondence occurred is extracted as noise, or in other words, parts of the subject image drop out.
In the method disclosed in the document No.3, the aforementioned problem concerning the difficulty of the threshold value determination is resolved by calculating 3-D coordinates without using the disparities, in which parameters which represent relative positions of the image pickup devices and angles between the optical axes of the image pickup devices are calculated using images of a predetermined pattern picked up by the image pickup devices, instead of controlling or measuring the relative positions of the image pickup devices and angles between the optical axes of the image pickup devices. Further, with regard to the aforementioned problem concerning the point correspondence search, error rate in the point correspondence search is reduced in the method of the document No.3, by shooting the subject in the centers of the images by the image pickup devices and executing the point correspondence search starting from the center of the images with high priority. Furthermore, even in the case where the false correspondence occurred in the point correspondence search, parts which have dropped out from the extraction or parts which have been incorrectly extracted are corrected by the outline restoration on the assumption of continuity. However, according to the method, all the image pickup devices have to be correctly controlled to shoot the subject in the centers of images. Further, in the case where the background has a complex scene, error rate in the point correspondence search increases and thus the outline restoration is necessitated to be executed considerably oftener, causing difficulty in the subject image extraction especially when the subject has a complex outline.
SUMMARY OF THE INVENTION
It is therefore the primary object of the present invention to provide a subject image extraction device by which an image of a desired subject can be extracted from an inputted image, without needing preparation of the information concerning the subject or the image pickup device.
Another object of the present invention is to provide a subject image extraction device by which an image of a desired subject can be extracted from an inputted image, with easy setting
Bella Matthew
Chawan Sheela
NEC Corporation
Ostrolenk Faber Gerb & Soffen, LLP
LandOfFree
Subject image extraction device does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Subject image extraction device, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Subject image extraction device will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2493943