Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
1998-06-01
2001-03-06
Au, Amelia (Department: 2623)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S103000, C382S154000, C345S419000
Reexamination Certificate
active
06198852
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Technical Field
The present invention relates to an image processing method and apparatus employing multi-view parallax geometry. In one application, a previously unknown view from a new perspective of 3-dimensional objects is rendered from 2-dimensional images taken from different perspectives.
2. Background Art
The analysis of 3D scenes from multiple perspective images has been a topic of considerable interest in the vision literature. For the sake of clarity, this specification refers to published techniques described in this literature where appropriate, and for the sake of brevity such references are made to the following publications by the numbers given below in brackets (e.g., “[3]”):
References
[1] G. Adiv. Inherent ambiguities in recovering 3-d motion and structure from a noisey flow field.
IEEE Trans. on Pattern Analysis and Machine Intelligence
, pages 477-489, May 1989.
[2] S. Avidan and A. Shashua. Novel view synthesis in tensor space. In
IEEE Conference on Computer Vision and Pattern Recognition
, pages 1034-1040, San-Juan, June 1997.
[3] S. Carlsson. Duality of reconstruction and positioning from projective views. In
Workshop on Representations of Visual Scenes
, 1995.
[4] S. Carlsson and D. Weinshall. Dual Computation of Projective Shape and Camera Positions from Multiple Images. In
International Journal of Computer Vision, in press.
[5] H. S. M Coxeter, editor.
Projective Geometry
. Springer Verlag, 1987.
[6] O. D. Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig? In
European Conference on Computer Vision
, pages 563-578, Santa Margarita Ligure, May 1992.
[7] O. D. Faugeras and B. Mourrain. On the geometry and algebra of the point and line correspondences between n images. In
International Conference on Computer Vision
, pages 951-956, Cambridge, Mass., June 1995.
[8] Olivier Faugeras.
Three-Dimensional Computer Vision—A Geometric View-point
. MIT Press, Cambridge, Mass., 1996.
[9] Richard Hartley. Lines and poins in three views—a unified approach. In
DARPA Image Understanding Workshop Proceedings
, 1994.
[10] Richard Hartley. Euclidean Reconstruction from Uncalibrated Views. In
Applications of Invariance in Computer Vision
, J. L. Mundy, D. Forsyth, and A. Zisserman (Eds.), Springer-Verlag, 1993.
[11] M. Irani and P. Anandan. Parallax geometry of pairs of points for 3d scene analysis. In
European Conference on Computer Vision
, Cambridge, UK, April 1996.
[12] M. Irani, B. Rousso, and S. Peleg. Computing occluding and transparent motions.
International Journal of Computer Vision
, 12(1):5-16, January 1994.
[13] M. Irani, B. Rousso, and P. peleg. Recovery of ego-motion using region alignment.
IEEE Trans. on Pattern Analysis and Machine Intelligence
, 19(3):268-272, March 1997.
[14] R. Kumar, P. Anandan, and K. Hanna. Direct recovery of shape from multiple views: a parallax based approach. In
Proc
12
th ICPR
, 1994.
[15] H. C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections.
Nature
, 293:133-135, 1981.
[16] R. Mohr. Accurate Projective Reconstruction In
Applications of Invariance in Computer Vision
, J. L. Mundy, D. Forsyth, and A. Zisserman, (Eds.), Springer-Verlag, 1993.
[17] A. Shashua. Algebraic functions for recognition.
IEEE Transactions on Pattern Analysis and Machine Intelligence
, 17:779-789, 1995.
[18] A. Shashua and N. Navab. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In
IEEE Conference on Computer Vision and Pattern Recognition
, pages 483-489, Seattle, Wash., June 1994.
[19] A. Shashua and P. Ananadan. Trilinear Constraints revisited: generalized trilinear constraints and the tensor brightness constraint. IUW, February 1996.
[20] P.H.S. Torr. Motion Segmentation and Outlier Detection. PhD Thesis:
Report No. OUEL
1987/93, Univ. of Oxford, UK, 1993.
[21] M. Spetsakis and J. Aloimonos. A unified theory of structure from motion.
DARPA Image Understanding Workshop
, pp.271-283, Pittsburgh, Pa., 1990.
[22] D. Weinshall, M.Werman, and A. Shashua. Shape descriptors: Bilinear, tri-linear and quadlinear relations for multi-point geometry, and linear projective reconstruction algorithms. In
Workshop on Representations of Visual Scenes
, 1995.
As set forth in the foregoing literature, given two calibrated cameras, their relative orientations can be determined by applying the epipolar constraint to the observed image points, and the 3D structure of the scene can be recovered relative to the coordinate frame of a reference camera (referred to here as the reference frame—e.g., see [15, 8]). This is done by using the epipolar constraint and recovering the “Essential Matrix” E which depends on the rotation R and translation T between the two cameras. Constraints directly involving the image positions of a point in three calibrated views of a point have also been derived [21].
If the calibration of the cameras is unavailable, then it is known that reconstruction is still possible from two views, but only up to a 3D projective transformation [6]. In this case the epipolar constraint still holds, but the Essential Matrix is replaced by the “Fundamental Matrix”, which also incorporates the unknown camera calibration information. The 3D scene points, the camera centers and their image positions are represented in 3D and 2D projective spaces (using homogeneous projective coordinates). In this case, the “reference frame” reconstruction may either be a reference camera coordinate frame [10], or as defined by a set of 5 basis points in the 3D world [16]. A complete set of constraints relating the image positions of multiple points in multiple views have been derived [7, 17]. Alternatively, given a projective coordinate system specified by 5 basis points, the set of constraints directly relating the projective coordinates of the camera centers to the image measurements (in 2D projective coordinates) and their dual constraints relating to the projective coordinates of the 3D scene points have also been derived [3, 22].
Alternatively, multiple uncalibrated images can be handled using the “plane+parallax” (P+P) approach, which analyzes the parallax displacements of a point between two views relative to a (real or virtual) physical planar surface II in the scene [18, 14, 13]. The magnitude of the parallax displacement is called the “relative-affine structure” in [18]. [14] shows that this quantity depends both on the “Height” H of P from II and its depth Z relative to the reference camera. Since the relative-affine-structure measure is relative to both the reference frame (through Z) and the reference plane (through H), this specification refers to the P+P framework also as the “reference-frame+reference-plane” formulation.
Using the P+P formulation, [17] derived “tri-linear” constraints involving image positions of a point in three uncalibrated views. The P+P has the practical advantage that it avoids the inherent ambiguities associated with estimating the relative orientation (rotation+translation) between the cameras; this is because it requires only estimating the nomography induced by the reference plane between the two views, which folds together the rotation and translation. Also, when the scene is “flat”, the F matrix estimation is unstable, whereas the planar homography can be reliably recovered [20].
SUMMARY OF THE INVENTION
The invention provides a new geometrical framework for processing multiple 3D scene points from multiple uncalibrated images, based on decomposing the projection of these points on the images into two stages: (i) the projection of the scene points onto a (real or virtual) physical reference p
Anandan Padmananbhan
Irani Michal
Weinshall Daphna
Au Amelia
Michaelson Peter L.
Michaelson & Wallace
Wallace Robert M.
Wu Jingge
LandOfFree
View synthesis from plural images using a trifocal tensor... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with View synthesis from plural images using a trifocal tensor..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and View synthesis from plural images using a trifocal tensor... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2482619