Image analysis – Image transformation or preprocessing – Correlation
Reexamination Certificate
1999-07-30
2002-11-19
Boudreau, Leo (Department: 2621)
Image analysis
Image transformation or preprocessing
Correlation
C106S219000, C106S154110, C348S042000, C348S048000, C396S089000
Reexamination Certificate
active
06483949
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image processing apparatus and method, and to a medium therefor and, more particularly, relates to an image processing apparatus and method for measuring a distance to an object by using a plurality of stereo image pairs, and to a medium therefor.
2. Description of the Related Art
As shown in, for example,
FIG. 1
, a stereo image distance measurement system is known in which the same object
5
is photographed by a standard camera
1
and a reference camera
2
, and in an image processing apparatus
31
, corresponding pixels of a pair of photographed images (a standard image and a reference image A) are specified, and a distance from the standard camera
1
to the object
5
is determined by using parallax information.
The operation of this stereo image distance measurement system is described below.
FIG. 2
shows the relationship between a reference image A and a standard image in which an object point P in a three-dimensional space is photographed by the standard camera
1
or the reference camera
2
. That is, if a pixel corresponding to the object point P within the standard image is denoted as a pixel m
b
and a pixel corresponding to the object point P within the reference image A is denoted as a pixel m
i
, the coordinates of the pixel m
i
are as described below by using a projection transformation matrix H of 3 rows and 3 columns:
m
i
=H·m
b
Therefore, the projection transformation matrix H can be computed if the coordinates of the pixels m
b
and m
i
are determined.
Accordingly, in a system in which the positional relationship between the standard camera
1
and the reference camera
2
, and a distance Z from the standard camera
1
to the object point P are measured in advance, the coordinates of the pixels m
b
and m
i
are specified, and the projection transformation matrix H corresponding to the distance Z is computed in advance by using these coordinates. The computed projection transformation matrix H is stored in the image processing apparatus
31
.
As shown in, for example,
FIG. 1
, when the distance from the standard camera
1
to the object point P is to be determined, the image processing apparatus
31
, as shown in
FIG. 3
, assumes the distance to be Z
1
from among the distances Z
1
to Z
n
which are discretely set in a distance measurement range, and determines a reference image m
i1
by using the projection transformation matrix H, corresponding to the distance Z
1
, in which the corresponding pixel (standard pixel) m
b
on the standard image is computed in advance and stored such that the object point P is photographed by the standard camera
1
.
Thereafter, the image processing apparatus
31
computes the mutual correlation value S
1
between the determined reference image m
i1
and the standard pixel m
b
. The mutual correlation value S
1
is computed by using the normalized mutual correlation between a predetermined small area with the pixel m
b
on the standard image being the center and a similar small area with the pixel m
i1
in the reference image A being the center, and an evaluation function, such as the sum of the squares of a luminance difference.
Next, assuming the distance from the standard camera
1
to the object point P to be Z
2
, the image processing apparatus
31
determines a reference pixel m
i2
by using a projection transformation matrix H corresponding to the distance Z
2
which is computed in advance and stored. Thereafter, the image processing apparatus
31
computes a mutual correlation value S
2
between the determined pixel m
i2
and the pixel m
b
.
Hereafter, in a similar manner, assuming a distance from the standard camera
1
to the object point P to be Z
3
to Z
n
in sequence, the image processing apparatus
31
determines the reference pixels m
i3
to m
in
by using a projection transformation matrix corresponding to the assumed distance and computes the mutual correlation values S
3
to S
n
.
It is known that pixels m
il
to m
in
, in which the pixel m
b
on the standard image is projected within the reference image A, are present on one straight line called an “epipolar line”, as shown in FIG.
3
.
FIG. 4
shows the relationship between the distances Z
1
to Z
n
and the mutual correlation values S
1
to S
n
. In the figure, it is shown that the smaller the value S of the mutual correlation value, the higher the correlation between pixels (the compared pixels are more similar).
Next, the image processing apparatus
31
creates a curve (for example, a second-order curve) passing through the smallest (highest correlation) one (a mutual correlation value S
3
in the example of
FIG. 4
) from among the mutual correlation values S
1
to S
n
and the values (the mutual correlation values S
2
and S
4
in the example of
FIG. 4
) before and after it, and outputs a distance Z
e
corresponding to the smallest value S
e
thereof as a distance of the standard pixel m
b
.
FIG. 5
shows the construction of a stereo image distance measurement system which measures the distance from the standard camera
1
to the object
5
by using two sets of stereo image pairs (a standard image and a reference image A, and a standard image and a reference image B). For the operation thereof, a process similar to the construction shown in
FIG. 1
is performed on two sets of stereo image pairs. That is, as shown in
FIG. 6
, an image processing apparatus
41
determines the pixels m
i1
to m
in
in the reference image A corresponding to the pixel m
b
on the standard image and computes the mutual correlation values S
1
to S
n
with respect to the pixel m
b
. Also, the image processing apparatus
41
determines pixels m
i1
to M
in
on the reference image B corresponding to the pixel m
b
in the standard image, and computes the mutual correlation values S
1′
to S
n′
with respect to the pixel m
b
.
Thereafter, the image processing apparatus
41
adds together the mutual correlation values S
1
to S
n
and the corresponding mutual correlation values S
1′
to S
n′
.
FIG. 7
shows that the mutual correlation values S
1
to S
n
are indicated by &Circlesolid; marks, the mutual correlation values S
1
to S
n′
are indicated by ▴ marks, and the mutual correlation values SS
1
to SS
n
are indicated by □ marks.
Next, the image processing apparatus
41
selects the smallest (highest correlation) one (the addition value SS
4
in the example of
FIG. 7
) from among the addition values SS
1
to SS
n
, and creates a curve (for example, a second-order curve) passing through the selected value and the values (the addition values SS
3
and SS
5
in the example of
FIG. 7
) before and after it, and outputs a distance Z
e
corresponding to the smallest value SS
e
as a distance of the standard pixel m
b
.
In a manner as described above, use of two sets of stereo image pairs makes it possible to obtain a result with higher accuracy than when one set of stereo image pairs is used.
In the above-described stereo image distance measurement system, in order to determine the pixel on the standard image corresponding to the pixel m
b
on the standard image plane, a search is made (mutual correlation values are compared) discretely on the epipolar line. In order to measure a distance with a higher accuracy, it is preferable that a search be made on the epipolar line at shorter intervals.
However, on the epipolar line, if the number of searches is increased, the number of computations is increased correspondingly, resulting in problems in that the scale of the system is increased and the processing time is increased.
SUMMARY OF THE INVENTION
The present invention has been achieved in view of such circumstances. An object of the present invention is to make it possible to measure a distance with a higher accuracy without increasing the number of searches which must be performed.
To achieve the above-mentioned object, according to one aspect of the present invention, there is provided an image processing apparatus for measuri
Miwa Yoko
Yokoyama Atsushi
Bell Boyd & Lloyd LLC
Boudreau Leo
Patel Kanji
LandOfFree
Image processing apparatus and method, and medium therefor does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing apparatus and method, and medium therefor, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing apparatus and method, and medium therefor will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2990075