Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
1998-10-23
2002-06-11
Au, Amelia M. (Department: 2623)
Image analysis
Applications
3-d or stereo imaging analysis
C382S284000, C382S295000, C345S422000
Reexamination Certificate
active
06404913
ABSTRACT:
TECHNICAL FIELD
The present invention relates to image synthesizing apparatus and method, position detecting apparatus and method, and a supply medium, particularly to image synthesizing apparatus and method, position detecting apparatus and method, and a supply medium for making it possible that the projected image of a predetermined object seems to be natural in continuous photographic images by synthesizing the projected image of the predetermined object with a plurality of continuous photographic images.
BACKGROUND ART
Because the computer graphics (CG) art has been advanced in recent years, it is possible to form a real image almost same as an actually photographed image (taken on the spot) though;the real image is artificially formed. In this case, the processing is frequently performed in which a predetermined object is continuously photographed (as a dynamic image) by a video camera and then, the CG image of the predetermined object is synthesized every image taken on the spot.
FIG. 1
shows a conventional image synthesizing method. In this case, it is assumed that an existent plate
1
is picked up by an image pickup section
2
and balls
5
and
6
formed through CG are synthesized with the image of the plate
1
so that the balls
5
and
6
seem to be stationary. In this case, the plate
1
on which patterns are drawn is first actually picked up by an image pickup section
2
. In this case, the i-th pattern formed on the plate
1
is assumed as ai. In this case, symbol i denotes an integer and also an identification number attached to each pattern ai. The patterns ai are different from each other in color and shape and therefore, they can be distinguished between them. The image pickup section
2
uses a stereophonic video camera constituted with a main video camera and a sub-video camera and is constituted so that three-dimensional information can be obtained from the parallax between an image picked up by the main video camera and an image picked up by the sub-video camera. The plate
1
is picked up while moving the image pickup section
2
.
In this case, the time for the image pickup section
2
to start pickup is assumed as a first time and a predetermined time after the first time in the pickup time is assumed as a second time.
FIG. 1
shows the image pickup section
2
at the first time and the second time.
As shown in
FIG. 2
, a frame image
1
A
1
obtained by picking up the plate
1
from a predetermined angle is obtained from the image pickup section
2
at the first time. Similarly, as shown in
FIG. 3
, a frame image
1
A
2
different from the frame image
1
A
1
obtained at the first time is obtained at the second time.
Then, as shown in
FIG. 4
, projected images
5
A
1
and
6
B
1
of the balls
5
and
6
are synthesized with the image (image in
FIG. 2
)
1
A
1
picked up by the image pickup section
2
at the first time. Then, as shown in
FIG. 5
, projected images
5
A
2
and
6
B
2
of the balls
5
and
6
are synthesized with the image (image in
FIG. 3
)
1
A
2
picked up by the image pickup section
2
at (the second time at positions where the balls
5
and
6
seem to be stationary on the plate
1
.
In this case, a method of synthesizing the balls
5
and
6
formed through CG so that they seem to be stationary on the plate
1
is described below by referring to
FIGS. 6 and 7
.
Though positions for finally synthesizing projected images
5
A and
6
B are on the two-dimensional images
1
A
1
and
1
A
2
, the synthesis positions are easily understood by considering them in a three-dimensional space. Therefore, three-dimensional data (coordinates) is restored from a two-dimensional frame image. Because the frame image is obtained by the stereophonic video camera, the above restoration is realized.
That is, as shown in
FIG. 6
, three-dimensional coordinates are assumed in which the position of the image pickup section
2
at the first time is an origin O
1
and the horizontal, vertical, and depth directions of the image pickup section
2
are X-axis, Y-axis, and Z-axis. Then, the three-dimensional position (X
1
i
, Y
1
i
, Z
1
i
) of every pattern ai of the plate
1
is first obtained from the two-dimensional frame image
1
A
1
. Then, a user designates a three-dimensional position (three-dimensional synthesis position) (X
1
A, Y
1
A, Z
1
A) on which the ball
5
is put and a three-dimensional position (three-dimensional synthesis position)(X
1
B, Y
1
B, Z
1
B) on which the ball
6
is put.
Then, a synthesis position (two-dimensional position) on the image
1
A
1
corresponding to a three-dimensional synthesis position is obtained. The two-dimensional position is obtained as a position obtained through perspective projection transform of the three-dimensional position. That is, when assuming the focal distance of the image pickup section
2
as f, the two-dimensional synthesis position of a projected image
5
A
1
on the image
1
A
1
at the first time corresponding to the three-dimensional synthesis position (X
1
A, Y
1
A, Z
1
A) is obtained as (X
1
A×F/Z
1
A, Y
1
A×f/Z
1
A). Similarly, the two-dimensional synthesis position of a projected image
6
B
1
on the image
1
A
1
at the first time corresponding to the three-dimensional synthesis position (X
1
B, Y
1
B, Z
1
B) is obtained as (X
1
B×f/Z
1
B, Y
1
B×f/Z
1
B).
Similarly, as shown in
FIG. 7
, the position of the image pickup section
2
at the second time is assumed as an origin O
2
and the horizontal, vertical, and depth directions of the image pickup section
2
are assumed as X-axis, Y-axis, and Z-axis. Then, the three-dimensional position (X
2
i
, Y
2
i
, Z
2
i
) of every pattern ai of the plate
1
is obtained from the two-dimensional image
1
A
2
.
As described above, because the patterns ai on the plate
1
can be distinguished between them, it is possible to identify that the i-th pattern ai in the first three-dimensional coordinate system restored from the image
1
A
1
at the first time corresponds to which pattern ai in the second three-dimensional system restored from the image
1
A
2
at the second time. Therefore, it is possible to make the coordinates (X
1
i
, Y
1
i
, Z
1
i
) of the pattern ai at the first time correspond to the coordinates (X
2
i
, Y
2
i
, Z
2
i
) of the pattern ai at the second time. Because these two coordinate systems view the same pattern ai from different angles, the second three-dimensional coordinate system can be obtained by applying predetermined rotational transform (hereafter, the function of the transform is assumed as R
1
) and predetermined rectilinear transform (hereafter, the function of the transform is assumed as S
1
) to the first three-dimensional coordinate system (though the function of the rectilinear transform is normally shown by T, it is shown by S because T is used as a variable showing time in this specification). Therefore, the relation shown by the following Equation is, effected for each pattern ai.
(
X
2
i,Y
2
i,Z
2
i
)=(
X
1
i,Y
1
i,Z
1
i
)·
R
1
+S
1
(Though the function of the rectilinear transform is normally shown by T, it is shown by S because T is used as a variable showing time in this specification.)
Therefore, the coordinate transform functions R
1
and S
1
can be obtained by substituting (X
1
i
, Y
1
i
, Z
1
i
) and (Z
2
i
, Y
2
i
, Z
2
i
) of each pattern ai for the above Equation.
However, when restoring the position (three-dimensional coordinates) of the pattern ai in a three-dimensional space from image data (image data for the images
1
A
1
and
1
A
2
) in a two-dimensional space picked up at a certain time, the restored position includes any error. Therefore, when using, for example, the first three-dimensional coordinate system as a criterion, the position of the i-th pattern ai is not exactly present at (X
1
i
, Y
1
i
, Z
1
i
). Similarly, when using the second three-dimensional coordinate system A criterion, the position of the i-th pattern ai is not exactly present at (X
2
i
, Y
2
i
, Z
2
i
).
Therefore, values obtained by squaring the magnitude of the three
Au Amelia M.
Bell Boyd & Lloyd LLC
Dastouri Mehrdad
Sony Corporation
LandOfFree
Image synthesizing apparatus and method, position detecting... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image synthesizing apparatus and method, position detecting..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image synthesizing apparatus and method, position detecting... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2950328