Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
1998-10-02
2001-03-13
Powell, Mark R. (Department: 2772)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C382S154000, C382S284000
Reexamination Certificate
active
06201541
ABSTRACT:
INCORPORATION BY REFERENCE
U.S. patent application Ser. No. 08/497,224, filed Jun. 30, 1995, in the name of Amnon Shashua, and entitled “Apparatus And Method For Recreating And Manipulating A 3D Object Based On A 2D Projection Thereof,” assigned to the assignee of this application, incorporated by reference.
U.S. patent application Ser. No. 08/989,047, filed Dec. 11, 1997, in the names of Dan Albeck, et al., and entitled “Apparatus And Method For 3-Dimensional Surface Geometry Reconstruction,” assigned to the assignee of this application, incorporated by reference.
FIELD OF THE INVENTION
This application relates generally to the field of reconstructing and/or manipulating surface geometries of one or more three dimensional objects in a scene, from a plurality of two dimensional images of the scene. The application more particularly relates to reconstruction and manipulation of such surface geometries of the object(s) as determined from a plurality of sets of two-dimensional images, each image set facilitating the reconstruction and/or manipulation of a predetermined portion of the surface geometries of the object(s) in the scene, with the plurality of image sets facilitating the reconstruction and/or manipulation of respective portions of the surface geometries of the object(s) in the scene.
BACKGROUND OF THE INVENTION
Reconstruction and/or manipulation (generally, “reconstruction”) of surface features of three-dimensional object(s) in a scene, from a plurality of two-dimensional images of the object(s), is useful in a number of applications. U.S. patent application Ser. No. 08/989,047, filed Dec. 11, 1997, in the names of Dan Albeck, et al., and entitled “Apparatus And Method For 3-Dimensional Surface Geometry Reconstruction” (hereinafter referred to as the “Albeck application”) describes an apparatus for performing such reconstruction using a rig of three cameras, using a tensor arrangement described in U.S. patent application Ser. No. 08/497,224, filed Jun. 30, 1995, in the name of Amnon Shashua, and entitled “Apparatus And Method For Recreating And Manipulating A 3D Object Based On A 2D Projection Thereof” (hereinafter referred to as the “Shashua application”) to generate information regarding reconstruction for the features of the object(s) from three images generated by the cameras. In the arrangement described in the Shashua application, the surface features that are reconstructed are defined by points that have coordinates in a coordinate system relative to one of the cameras in the rig. A problem arises in reconstruction if the surface features that are to be reconstructed cannot all be recorded by all of the cameras with the rig in one position. The apparatus described in the Albeck application provides a mechanism for moving the rig so as to allow the cameras to record sets of images of various portions of the surface(s) of the object(s). However, when the rig moves from one location, in which the cameras record a set of images of one portion of the object(s), to another location, in which the cameras record another set of images of another portion of the object(s), the coordinate system for the points defining the surface features of the various portions of the object(s) also changes.
In order to utilize the reconstruction information generated in the two coordinate systems in a unitary manner in connection with the object(s) in the scene, it is necessary to relate the coordinate systems to a common coordinate system, which will allow all of the points of the various portions of the reconstructed object(s) to be related to the common coordinate system, effectively “stitching” the reconstructions together. The common coordinate system can conveniently be one of the two coordinate systems used in reconstruction, or it can be a third coordinate system, but in any case all of the points for the various portions of the reconstructed object need to be related to a common coordinate system. When a rig, such as the rig described in the Albeck application is moved from one position to another, to facilitate recording of sets of images of different portions of the object(s), the movement comprises one or both of a translation and a rotation of the rig, both in three dimensions. If the translational and rotational movement of the rig can be controlled sufficiently precise, the relation of the coordinate system after movement to the coordinate system before movement can easily be determined. However, if, for example, the mass of the rig is sufficiently large, in a number of applications the movement of the rig cannot readily be controlled sufficiently precisely to allow sufficiently precise reconstruction.
SUMMARY OF THE INVENTION
The invention provides new and improved systems and methods for “stitching” a plurality of reconstructions of three-dimensional surface features of object(s) in a scene, the reconstructions being defined relative to respective coordinate systems, thereby to relate them to a common coordinate system.
In brief summary, the invention provides a system disclosed for stitching a plurality of reconstructions of three-dimensional surface features of at least one object in a scene to relate the reconstructions to a common coordinate system. Each reconstruction is generated from a plurality of two-dimensional images of the scene, and each reconstruction is defined relative to a respective one of a plurality of reconstruction coordinate systems defined relative to one of the images in the scene. The system generates, from at least some of the images, values for translational and rotational components relating a respective reconstruction coordinate system to the common coordinate system. Thereafter, the system uses the values of the translational and rotational components to convert coordinates of the surface features in the respective reconstruction coordinate systems to the common coordinate system.
In embodiments of the system, the common coordinate system for simplicity is selected to be one of the reconstruction coordinate systems, and the values for translational and rotational components relating the other reconstruction coordinate system(s) to the common coordinate system uses overlapping regions of the images which define the respective reconstruction coordinate systems. In those embodiments, the system first determines the dominate translation that best aligns the largest portions of the image which defines the common coordinate system and the other images, shifts the other images by the amount of the dominate translation and then generates a pixel-to-pixel correspondence to generate a displacement field. From the dominate translation and the displacement field, the system generates the values for the translational and rotational components.
In embodiments of the system, for multiple reconstructions, instead of processing successive pairs of sets of images to determine translational and rotational components associated with respective reconstruction coordinate systems, the system makes use of a statistical methodology, identified as the “E-M” (estimation and maximization) methodology, to generate maximum likelihood estimates of the translational and rotational components associated with the relations between the reconstructin coordinate systems and the common coordinate system.
REFERENCES:
patent: 5598515 (1997-01-01), Shashua
patent: 5818959 (1998-10-01), Webb et al.
patent: 5821943 (1998-10-01), Shashua
patent: 98/27514 (1998-07-01), None
patent: 96/34365 (1998-07-01), None
Goldberger Jacob
Shalom Tamir
Zelnik Ilan
CogniTens Ltd.
Jordan Richard A.
Powell Mark R.
Santiago Enrique L.
LandOfFree
System and method for “Stitching” a plurality of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for “Stitching” a plurality of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for “Stitching” a plurality of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2438068