Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
2000-05-10
2003-10-14
Ahmed, Samir (Department: 2623)
Image analysis
Applications
3-d or stereo imaging analysis
C345S419000, C348S208100
Reexamination Certificate
active
06633664
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a three-dimensional structure acquisition method, an apparatus and a computer readable medium for generating a realistic image from an original image.
In recent years, three-dimensional computer graphics is in increasing demand in various fields such as games, movies, commercials and the like. However, high labor costs are incurred to create a three-dimensional model for three-dimensional computer graphics. Thus, a method for generating the three-dimensional model automatically or semiautomatically from a real object or a real landscape is required.
2. Description of the Related Art
Conventionally, various inputting methods are proposed for inputting a three-dimensional structure such as an object or a landscape. For example, in a document, R. C. Bolles, H. H. Baker and D. H. Marimont, “Epipolar-Plane Image Analysis: An Approach to Determining Structure from Motion” IJCV, vol.1, No.1, 1987, pp.7-55 (which will be referred to as document
1
in the following), a method called epipolar plane image analysis is disclosed. In the epipolar plane image analysis, many images of a landscape taken by a video camera are input, spatio-temporal images are generated, and a three-dimensional structure is obtained by analyzing linear patterns shown in the x-t plane image. This method is simple and easy. But, it is necessary to accurately move the camera with uniform linear motion. When the uniform linear motion is accompanied by unknown motion due to hand shake, vibration and the like, the patterns shown in the x-t plane image do not become linear. Therefore, accuracy for extracting the three-dimensional structure significantly decreases.
In Japanese laid-open patent application No.
11-339043
“METHOD FOR INPUTTING THREE DIMENSIONAL SHAPE AND RECORDING MEDIUM FOR RECORDING THREE DIMENSIONAL SHAPE INPUT PROGRAM” (which will be referred to as document
2
in the following), a method for calibrating an image as preprocessing of the method in the document
1
is proposed. In the method, camera fluctuation due to camera shake is estimated from paths of feature points. The estimated camera motion is used for calibrating the image as if the camera keeps uniform linear motion. According to this method, accurate uniform linear motion of the camera becomes unnecessary and the epipolar plane image analysis becomes easy. Similarly, in a method proposed in a document Z. Zhu, G. Xu and X. Lin, “Constructing 3D Natural Scene from Video Sequences with Vibrated Motions,” Proc. IEEE VRAIS '98, 1998, pp. 105-112, camera fluctuation is estimated from optical flow. Then, the image calibrated as if the camera keeps uniform linear motion by using the estimation data.
However, according to the method in the document
2
, there is a problem in that accuracy of three-dimensional structure calculation deteriorates when the depth dimension of the three-dimensional structure is large.
In addition, in a method shown in documents, C. Tomasi and T. Kanade, “Shape and Motion from Image Streams : a Factorization Method—Full Report on the Orthographic Case,” Computer Science Technical Report, CMU-CS-104, Carnegie Mellon Univ., 1992 (which will be referred to as document
3
in the following) and C. J. Poelman and T. Kanade, “A Paraperspective factorization method for Shape and Motion Recovery,” IEEE PAMI, vol. 19, no. 3, 1997, pp. 206-218, three-dimensional coordinates of the image feature points are obtained by inputting a plurality of images of a landscape, determining correspondence of feature points and using a factorization method. This method can be easily performed because a special apparatus or special consideration is unnecessary for inputting an image. However, there is a defect in that error becomes large when the depth dimension of the three-dimensional object is large since the camera model used in the method is not the perspective transform model.
Further, in Japanese laid-open patent application No. 7-146121 “RECOGNITION METHOD AND DEVICE FOR THREE-DIMENSIONAL POSITION AND ATTITUDE BASED ON VISION”, a camera calibration method is disclosed. In the method, a camera position and attitude are estimated from an image of a three-dimensional object taken by the camera, in which the position and the size of the three-dimensional object is known. However, according to this method, a three-dimensional structure of an unknown object or landscape can not be obtained.
In Japanese laid-open patent application No. 8-181903“IMAGE PICKUP APPARATUS”, an apparatus is disclosed in which a plurality of images are input, translation and rotation deviations between the images are estimated and a merged image is synthesized. However, according to this method, the three-dimensional structure of an object or a landscape can not be obtained.
In an apparatus disclosed in Japanese laid-open patent application No. 11-183139 “SECTIONAL AND THREE-DIMENSIONAL SHAPE MEASURING APPARATUS”, three-dimensional structure of an object is measured from a plurality of images taken by projecting slit light to the object. However, there is a defect that it is expensive to realize the apparatus since a camera needs to operate together with an apparatus for projecting the slit light.
SUMMARY OF THE INVENTION
It is an object of the present invention to provide a three-dimensional structure acquisition method, an apparatus and a computer readable medium for acquiring a three-dimensional structure of an object or a landscape accurately even when the depth dimension of the object or the landscape is large.
The above object of the present invention is achieved by a three-dimensional structure acquisition method comprising:
an input step for inputting images from a camera to a computer;
a first extraction step for extracting a three-dimensional structure from the images by a predetermined method;
a generation step for generating depth maps from the three-dimensional structure;
a calibration step for correcting image deviation due to camera fluctuation by using the depth maps;
a second extraction step for extracting a three-dimensional structure from calibrated images by the predetermined method; and
an outputting step for outputting the three-dimensional structure to a storage device.
According to the above-mentioned invention, the three dimensional structure can be acquired accurately since images which are accurately calibrated by using the depth maps are used.
In the three-dimensional structure acquisition method, the predetermined method may be based on epipolar plane image analysis.
According to this invention, the three dimensional structure can be acquired accurately by using the epipolar image analysis.
The three-dimensional structure acquisition method may further include the steps of:
generating synthesized images after extracting the three-dimensional structure;
calculating differences between the synthesized images and the calibrated images;
wherein the generation step, the calibration step and the second extraction step are performed repeatedly until the differences become smaller than or equal to a predetermined value.
According to the above-mentioned invention, accuracy of the three-dimensional structure is further improved. In addition, the process end can be judged by comparing the synthesized images and the calibrated images.
In addition, according to the present invention, a three-dimensional structure acquisition apparatus and a computer readable medium storing a three-dimensional structure acquisition program are provided.
REFERENCES:
patent: 5796427 (1998-08-01), Suzuki
patent: 6342918 (2002-01-01), Inou et al.
patent: 7146121 (1995-06-01), None
patent: 818848 (1996-01-01), None
patent: 8181903 (1996-07-01), None
patent: 10116341 (1998-05-01), None
patent: 1137721 (1999-02-01), None
patent: 11183139 (1999-07-01), None
patent: 11339021 (1999-12-01), None
patent: 11339043 (1999-12-01), None
R.C. Bolles, H.H. Baker and D.H. Marimont, “Epipolar-Plane Image Analysis: An Approach to Determining Structure from Motion”, IJCV, vol. 1, No. 1, 1987, pp. 7-55.
Matsuda Tatsuki
Minamida Kouki
Notomi Mikito
Shinya Mikio
Ahmed Samir
Bali Vikkram
Nippon Telegraph and Telephone Corporation
LandOfFree
Three-dimensional structure acquisition method, apparatus... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Three-dimensional structure acquisition method, apparatus..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Three-dimensional structure acquisition method, apparatus... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3164566