Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
2000-07-27
2004-10-05
Dastouri, Mehrdad (Department: 2623)
Image analysis
Applications
3-d or stereo imaging analysis
C382S218000, C382S278000
Reexamination Certificate
active
06801653
ABSTRACT:
BACKGROUND OF THE INVENTION
This invention relates to an information processing apparatus and method as well as a medium, and more particularly to an information processing apparatus and method as well as a medium wherein matching between images is performed using a template in measuring a distance according to the stereo method.
Similarly to the principle used by a human being to sense a shape of an object or a distance from an object to a body, a stereo method is generally known as a method of measuring a distance to an object. According to the stereo method, a shape of or a distance to an object can be measured in accordance with the principle of triangulation using images observed by a plurality of cameras having different visual points from each other.
FIG. 1
illustrates the principle of the stereo method. Referring to
FIG. 1
, two cameras of a base camera
1
and a reference camera
2
are disposed at visual points different from each other so that a position of an object point Q to be measured in a three-dimensional space can be determined from the two cameras. In particular, an observation point n
b
at which the object point Q is observed on an image plane
1
A of the base camera
1
and another observation point n
r
at which the object point Q is observed on an image plane
2
A of the reference camera
2
are determined. Then, the position of the object point Q in the three-dimensional space can be determined from the two observation points n
b
and n
r
.
As a technique for detecting the observation point n
r
corresponding to the observation point n
b
, a method of searching for a corresponding point on an epipolar line has been proposed. For example, the observation point n
r
of the reference camera
2
which is a corresponding point to the observation point n
b
which is on the image plane
1
A observed by the base camera
1
(in the following description, an image on the image plane
1
A observed by the base camera
1
is referred to simply as base image
1
a
as seen from
FIG. 2A
) is present on a straight line LP along which a plane (image plane) which is defined by the optical center (optical axis) of the reference camera
2
and the observation point n
b
of the base camera
1
and the image plane
2
A observed by the reference camera
2
(in the following description, an image on the image plane
2
A observed by the reference camera
2
is referred to simply as reference image
2
a
as seen from
FIG. 2B
) intersect each other. The straight line LP is called epipolar line. Then, if the positional relationship between the base camera
1
and the reference camera
2
is known, then since the same object which is at different projection points from each other can be found, a desired corresponding point can be detected for each observation point of the base camera
1
by searching for the corresponding point on the epipolar line (straight line LP) on the reference image
2
a.
As a technique for searching for a corresponding point, “pixel-based matching”, “feature-based matching” and “area-based” matching are known. They have the following characteristics.
The pixel-based matching searches for a corresponding point using concentration values of individual pixels. Therefore, it is high in speed of arithmetic operation, but is low in matching accuracy.
The feature-based matching extracts a characteristic such as a concentration edge from an image and searches for a corresponding point using only the characteristic between images. Therefore, information of a distance image obtained is rough.
The area-based matching involves a kind of correlation arithmetic operation. Therefore, a high arithmetic operation cost is required. However, since a corresponding point to an object can be searched out with a high degree of accuracy and distance values of all pixels can be calculated, the area-based matching is generally used frequently.
FIGS. 2A and 2B
illustrate the principle of the area-based matching. Referring to
FIGS. 2A and 2B
, a local window W (area) is set around a noticed point (noticed pixel)
11
set arbitrarily on an image (base image
1
a
) observed by the base camera
1
, and the window W is set as a template
12
. In
FIG. 2A
, the template
12
is formed from 25 pixels arranged in 5 rows×5 columns.
Then, as seen in
FIG. 2B
, the template
12
is disposed as a template
12
A on an epipolar line
13
of an image (reference image
2
a
) observed by the reference camera
2
, and matching is performed within the set search range and a coincidence degree R(x, y) is arithmetically operated in accordance with the following expression (1):
R
(
x
,
y
)
=
∑
(
x
,
y
)
∈
w
(
I
m1
(
x
,
y
)
-
I
m2
(
x
+
Δ
x
,
y
+
Δ
y
)
)
2
(
1
)
where Im
1
(x, y) is a pixel of the base image
1
a
, Im
2
(x+&Dgr;x, y+&Dgr;y) is a pixel of the reference image
2
a
, and &Dgr;x and &Dgr;y represent an amount of movement of the template
12
on the epipolar line
13
. Thereafter, the template
12
is moved along the epipolar line
13
and is disposed as a template
12
B. Then, similarly as for the template
12
A, a coincidence degree R(x, y) is arithmetically operated in accordance with the expression (1). The template
12
is further moved along the epipolar line
13
and is disposed as a template
12
C. Then, similarly as for the templates
12
A and
12
B, a coincidence degree R(x, y) is arithmetically operated in accordance with the expression (1).
One of the three coincidence degrees R(x, y) determined in accordance with the expression (1) above which exhibits the lowest value exhibits the highest coincidence degree (similarity degree) between the base image a
1
a
and the reference image
2
a
. Accordingly, the movement amount &Dgr;x, &Dgr;y of the template
12
when the coincidence degree R(x, y) exhibits the lowest value is determined as a parallax of the noticed point
11
, and a shape or a depth of the noticed point
11
in the three-dimensional space can be calculated in accordance with the principle of triangulation using the parallax of the noticed point
11
.
In this manner, in the area-based matching, three-dimensional shape data corresponding to all pixels can be obtained by repeating the matching (matching) processing for each pixel. It is to be noted that, while the coincidence degree R(x, y) of the three template
12
A to template
12
C in
FIG. 2B
are arithmetically operated in accordance with the expression (1) above, actually the template
12
is successively moved by a predetermined value within a preset search range on the epipolar line
13
, and the coincidence degree R(x, y) at each of such positions is arithmetically operated.
However, whichever one of the techniques described above is used, it is difficult to accurately determine all corresponding points on an image because some “ambiguity” is involved in matching between images.
For example, if it is tried to use the area-based matching to perform matching of a texture pattern
22
on a plane
21
disposed obliquely in a three-dimensional space as shown in
FIG. 3
, then the texture pattern
22
observed by the two cameras of the base camera
1
and the reference camera
2
is such as shown in
FIGS. 4B and 4C
, respectively. In particular,
FIG. 4A
shows the plane
21
of FIG.
3
and the texture pattern
22
disposed on the plane
21
, and
FIG. 4B
shows an observed image (base image
1
a
) when the plane
21
is observed from the base camera
1
while
FIG. 4C
shows an observed image (reference image
2
a
) when the plane
21
is observed from the reference camera
2
. As can be seen from
FIGS. 4A
to
4
C, although the left and right cameras (base camera
1
and reference camera
2
) observe the same object pattern (texture pattern
22
), a geometrical distortion appears between the images of the texture pattern
22
and the same object pattern is recognized as different objects. This gives rise to a problem that matching is difficult.
In order to determine a cor
Ushiro Teruyuki
Wu Weiguo
Yokoyama Atsushi
Yoshigahara Takayuki
Bell Boyd & Lloyd LLC
Dastouri Mehrdad
Sony Corporation
LandOfFree
Information processing apparatus and method as well as medium does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Information processing apparatus and method as well as medium, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Information processing apparatus and method as well as medium will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3321919