Image analysis – Applications – Range or distance measuring
Reexamination Certificate
2002-12-16
2003-09-23
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Applications
Range or distance measuring
C356S003000
Reexamination Certificate
active
06625301
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus, an image processing method and a transmission medium, and more particularly to an image processing apparatus, an image processing method and a transmission medium which are used for calculating a distance to a target point with a stereo method.
The so-called stereo method is a technique to measure a distance to a target point based on the principle of triangular surveying by using images picked up from a plurality of visual points.
FIG. 1
shows an example of configuration of a conventional image processing apparatus for measuring a distance to a target point with the stereo method.
In
FIG. 1
, memories
901
-
1
to
901
-
5
store one-frame images picked up by cameras
1
to
5
, respectively, and image data are read out of the memories to be output in predetermined sequence. Of the cameras
1
to
5
, the camera
1
, for example, is set as a reference camera. An image of the reference camera is compared with each of the other cameras (compared cameras) to perform a matching process, and a parallax is calculated in accordance with a result of the matching process.
An SAD (Sum of Absolute Difference) circuit
902
calculates an absolute value of difference for each pixel value between the reference image output from the memory
901
-
1
and each of the images output from the other memories
901
-
2
to
901
-
5
. For example, the SAD circuit
902
calculates absolute values of differences for each pixel value between the memories
901
-
1
and
901
-
2
, between the memories
901
-
1
and
901
-
3
, between the memories
901
-
1
and
901
-
4
, and between the memories
901
-
1
and
901
-
5
, and then outputs the absolute values.
An SSAD (Sum of SAD) circuit
903
outputs results of an inter-camera block matching process performed based on the absolute values of differences in pixel value between the cameras.
A minimum value detecting portion
904
detects a minimum value from among the results output from the SSAD circuit
903
.
A second-order portion
905
performs approximation (interpolation) for the minimum value output from the minimum value detecting portion
904
and other values thereabout by using a second-order function to calculate a minimum value with higher accuracy.
A memory
906
stores the minimum value output from the second-order portion
905
.
The operation of the conventional image processing apparatus thus constructed will be described below.
Assume now that the camera
1
is disposed at the center and the other cameras
2
to
5
are disposed so as to surround the camera
1
. An image output from the camera
1
is stored as a reference image in the memory
901
-
1
. Images output from the other cameras
2
to
5
are stored respectively in the memories
901
-
2
to
901
-
5
.
The SAD circuit
902
creates a pair of the reference image output from the memory
901
-
1
and each of compared images output from the other memories
901
-
2
to
901
-
5
, and then calculates and outputs an absolute value of difference for each pixel value between both the images of each pair.
The SSAD circuit
903
performs a matching process for a predetermined pixel block (e.g., a 5×5 block) based on the absolute values of differences in pixel value for each pair of the images output from the SAD circuit
902
. In other words, the SSAD circuit
903
calculates a difference value in units of a pixel block, and supplies calculated results to the minimum value detecting portion
904
.
The minimum value detecting portion
904
detects a minimum value from among the difference values output from the SSAD circuit
903
, and supplies the detected minimum value to the second-order portion
905
.
The second-order portion
905
performs approximation (interpolation) for the minimum value output from the minimum value detecting portion
904
and other data of two points thereabout by using a second-order function to create and output a minimum value with higher accuracy.
The memory
906
stores data output from the second-order portion
905
.
In processing based on the stereo method, it is required to solve the problem of corresponding points, i.e., determining which pixel of a compared image each pixel of a reference image corresponds. The problem of corresponding points is generally solved by a matching process. Stated otherwise, by comparing the reference image and the compared image, portions showing a maximum matching (similarity) therebetween are determined as corresponding points and correlated to each other.
When searching the corresponding points based on the matching process, a search area is decided beforehand by previous calibration. Theoretically the search area should exist along a straight line called an Epipolar line, but practically it exists along a curved line due to lens aberrations, etc. in many cases.
To overcome such a problem, it is thought to list all search points and store them in a lookup table beforehand. Even in the case of using the lookup table thus created, an intensity value of each pixel may differ between cameras of differences in the kinds of lenses and diaphragms used, resulting in a problem that the matching process cannot be precisely performed.
With a view of overcoming the above problem, processing to compress an input image into 4 bits and to emphasize a portion, where an intensity value changes to a large extent, for absorbing a difference of the intensity value has been proposed (see “A Stereo Machine for Video-rate Dense Depth Mapping and Its New Applications”, Takeo Kanade, Atsushi Yoshida, Kazuo Oda, Horoshi Kano and Masaya Tanaka, Proceedings of 15th Computer Vision and Pattern Recognition Conference (CVPR), Jun. 18-20, 1996, San Francisco).
With this proposed method, however, a fine intensity difference of an original image is ignored and cannot be reflected on a result of the matching process. Hence there has been a problem that a correct result of the matching process is obtained in a portion where changes of the intensity value are large, while a correct result of the matching process cannot be obtained in a portion where changes of the intensity value are small.
Further, because it is desired that distance data resulting from the matching process be obtained with as high an accuracy as possible, the conventional image processing apparatus mentioned above is designed to perform interpolation for minimum value data, which results from searching the corresponding point, by using a second-order curve and obtain data with higher accuracy.
Executing the interpolation however requires a division process which entails a relatively high computing cost. To reduce a circuit scale needed, therefore, the interpolation has been hitherto executed only upon a minimum value among results obtained from the block matching process (i.e., output of the minimum value detecting portion
904
). This method can reduce a hardware scale, but causes a delay in the overall process because the interpolation is required to be executed after the minimum value has been output. Accordingly, this method has caused difficulty in speeding up the processing.
Further, it is important in the stereo method to improve accuracy of a distance to the target point. From this point of view, a stereo processing using a plurality of cameras and a plurality of base line lengths (distances between the cameras) has been proposed, by way of example (see “Stereo Matching Utilizing a Plurality of Base Line Lengths”, Journal of The Institute of Electronics, Information and Communication Engineers (Japan), D-II, Vol. J75-D-II No. 8, pp. 1317-1327, August 1992).
With this proposed method, an error in matching can be reduced and distance accuracy can be improved as a result of using a plurality of cameras in a combined manner.
In the proposed method, however, the stereo processing is executed in accordance with one combination of the plural cameras (i.e., by using all the cameras in one combination). Therefore, the proposed method has had difficulty in dealing with the problem of occlusion an
Choobin Barry
Lerner David Littenberg Krumholz & Mentlik LLP
Mehta Bhavesh M.
Sony Corporation
LandOfFree
Image processing apparatus, image processing method and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing apparatus, image processing method and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing apparatus, image processing method and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3067617