Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
2000-03-27
2003-11-04
Mehta, Bhavesh M (Department: 2625)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S154000, C382S294000, C345S629000
Reexamination Certificate
active
06643413
ABSTRACT:
FIELD OF THE INVENTION
The invention relates generally to image-based rendering, and more specifically to using manifold mosaics to accomplish image-base rendering.
BACKGROUND OF THE INVENTION
Image-based rendering (IBR) in general simulates a continuous range of virtual camera viewpoints of a scene or object from a discrete set of input images of the scene or object. IBR can be used for a variety of different applications. For example, a currently popular application is to use IBR in conjunction with electronic commerce-type applications on the Internet. A user interested in seeing a car's interior or exterior from different viewpoints, for example, may be able to rotate inside a virtual camera inside the car's interior, and zoom in and out to see more or less of the interior, and may also be able to rotate a virtual camera around the car's exterior. Thus, using IBR in such instances enables consumers to better see physical real-world objects that they may desire to purchase, allowing those consumers to make better-informed decisions.
Within the prior art, IBR has been accomplished in large part by using what is know as view interpolation. View interpolation generally involves using a number of known views to generate a desired view, using interpolation of the known views. View interpolation is, however, a difficult task to accomplish. For example, feature correspondence—the corresponding of features, such as pixels or objects, within one of the known views to one or more of the other of the known views—usually needs to be accurately established. Correspondence is particularly difficult when the two known views, or images, have been taken from distant viewpoints.
However, when accurate depth information of a to-be-rendered image is available—that is, information regarding the depth of the desired view relative to a subject object or scene—the image can be rendered from a close known view by accomplishing what is known as warping. That is, the close known view is modified according to its depth. However, providing precise depth information regarding a to-be-rendered image is generally difficult to accomplish.
Many prior art IBR techniques solve the difficulties of correspondence and precise depth information by using a substantial number of images. To render an image at a new viewpoint, these techniques generally select nearby rays of known images, and interpolate them. Constant depth assumptions are made about the depth of the scene in order to locate optimal close rays, for optimal rendering quality. However, the large number of images needed for these techniques means that a large amount of data is required, which is undesirable for Internet and other applications, where the bandwidth over which the images will be transmitted may be low, resulting in inconveniently long transmission times, and/or where the storage space available for the images may be relatively small, such as in specialized devices like Internet appliances that may not have large hard disk drives or similar types of storage devices.
To solve this problem, the prior art typically compromises the extent to which the subject scene can be viewed by the user by constraining the motion of the virtual rendering camera. For example, some systems only allow users to explore large environments at pre-specified locations. However, even though a continuous change in viewing directions at each location is allowed, such systems can only jump among the pre-specified locations when a different viewing location is desired. This causes visual discontinuity and discomfort for the user. Thus, while the amount of data necessary for such IBR techniques can be reduced, the compromise is that perceived continuous virtual camera movement can be lost. Unfortunately, perceived continuous virtual camera movement is generally considered to be important for a user to smoothly navigated within a virtual environment.
For this and other reasons, therefore, there is a need for the present invention.
SUMMARY OF THE INVENTION
The invention relates to manifold mosaic hopping for image-base rendering (IBR). In one embodiment, a set of manifold mosaics, such as concentric mosaics, is constructed. Each mosaic represents a number of unique viewpoints that different from one another in only one direction. For example, in the case of concentric mosaics, the viewpoints in a given mosaic differ from one another only in the angular direction, in the r-theta domain. From a current virtual camera or viewing position within the set of mosaics, a new position can be moved to, such that the movement is continuous over any one mosaic, and is discrete across the mosaics. For example, in the case of concentric mosaics, movement is continuous over a given mosaic's circle, and is discrete when moving in the radial direction across mosaics. An image is rendered by local warping, without interpolation of more than one mosaic.
Embodiments of the invention provide for a reduction in the amount of data—that is, the number of images—needed to render a new image based on a new viewpoint of a scene or object, without unduly constraining the locations at which the new image can be rendered. The number and location of the manifold mosaics can be selected such that the discrete movement across mosaics perceptually appears to the user as continuous movement. That is, the number and location of the mosaics can be selected to take advantage of insensitive characteristics of the human eye, such that perceived smooth movement is still achieved even though discrete movement is in fact taking place.
REFERENCES:
patent: 6097854 (2000-08-01), Szeliski et al.
patent: 6393163 (2002-05-01), Burt et al.
Chen and Williams, View Interpolation for image synthesis, Computer Graphics (SIGGRAPH'93), pp. 279-288, Aug. 1993.
Chen, QuickTime VR—an image-based approach to virtual environment navigation, Computer Graphics (SIGGRAPH'95), pp. 29-38, Aug. 1995.
Gortler, Grzeszczuk, Szeliski, Cohen, The lumigraph, in Computer Graphics Proceedings, Annual Conference Series, pp. 43-54, Proc. ISGGRAPH'96 (New Orleans), Aug. 1996, ACM SIGGRAPH.
Levoy, Hanrahan, Light-field rendering, in Computer Graphics Proceedings, Annual Conference Series, pp. 31-42, Proc. SIGGRAPH'96 (New Orleans), Aug. 96, ACM SIGGRAPH.
Mark, McMillan, Bishop, Post-rendering 3d warping, in Proc. Symposium on 13D Graphics, pp. 7-16, 1997.
McMillan, An image-based approach to three-dimensional computer graphics, technical report, ph.d. dissertation, UNC Computer Science TR97-013, 1999.
Peleg, Herman, Panoramic mosaics by manifold projection, in IEEE Computer Soc'y Conference on Computer Vision and Pattern Recognition (CVPR'97), pp. 338-343, San Juan, Puerto Rico, Jun. 1997.
M. Ramasubramanian, S. Pattanaik, D. Greenberg, A perceptually based physical error metric for realistic image synthesis, Computer Graphics (SIGGRAPH'99), pp. 73-82, Aug. 1999.
Shum, He, Rendering with concentric mosaics, in Proc. SIGGRAPH 99, pp. 299-306, Aug. 1999.
Zorin, Barr, Correction of geometric perceptual distortions in pictures, Computer Graphicws (SIGGRAPH'95), pp. 257-264, Aug. 1995.
A. Lippman, Movie Maps, an application of the optical videodisc to computer graphics, Computer Graphics (SIGGRAPH'80), 14(3):32-43, Jul. 1980.
Chai Jin-Xiang
Shum Heung-Yeung
Tong Xin
Wang Li Feng
Bayat Ali
Fischer Craig S.
Lyon & Harr L.L.P.
Mehta Bhavesh M
Microsoft Corporation
LandOfFree
Manifold mosaic hopping for image-based rendering does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Manifold mosaic hopping for image-based rendering, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Manifold mosaic hopping for image-based rendering will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3118367