Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
1998-11-17
2002-11-05
Mancuso, Joseph (Department: 2623)
Image analysis
Applications
3-d or stereo imaging analysis
C382S293000
Reexamination Certificate
active
06477268
ABSTRACT:
BACKGROUND
The invention relates to the field of panoramic image based virtual reality.
In a virtual reality setting, a user can interact with objects within an image-based virtual world. In one approach, the objects in the virtual world can be rendered based on a mathematical description of the objects, such as wire-frame models. The rendering work depends on the scene complexity, as does the number of pixels in an image. A powerful graphics computer interface is typically required to render the images in real time.
In an alternate approach, the virtual world can be rendered in the form of panoramic images. Panoramic images are images that are “stitched” from several individual images. Multiple images can be acquired of an object from different viewpoints which can then enable a user to view the scene from different viewing angles and to interact with objects within the panoramic image. A hybrid approach that superimposes 3D geometry-based interactive objects onto a panoramic scenery image background, can also be used. The above two methods enhance to some extent the interactivity for the panoramic image-based virtual worlds.
In the following, the following terminology will be used: a view image is an image projected on a planar view plane, such as the film plane of a camera; a vista image is an image that is projected on a geometrical surface other than a plane, such as a cylinder or a sphere; a panoramic image (or vista) is an image (or a vista) produced by “stitching” multiple images (or vistas).
To navigate freely between a panoramic image vista composed of multiple vista images, these vista images must be linked. However, smooth transitions are difficult to attain. One solution would be to continuously zoom between the vista images until the source vista approximates the destination vista, and then directly switch the image to the destination vista. Many users, however, find the quality of the visual effects of zoomed vista transitions still unacceptable.
Image morphing provides another solution to smooth abrupt changes between vistas. Typically, two corresponding transition windows with a number of corresponding points are located on the source and destination vistas. Scenes with larger disparity (depth) differences among the objects, however, are often difficult to align due to effects from motion parallax. Another problem can occur with singular views where the optical center of one vista is within the field of view of the other vista. Singular views are common in vista transitions, because the direction of the camera movement during a transition is usually parallel to the viewing direction.
SUMMARY OF THE INVENTION
The method of the invention provides smooth vista transitions in panoramic image-based virtual worlds. In general, the method aligns two panoramic vistas with unknown camera axes for smooth transitions by locating epipoles on the corresponding panoramic images. The method combines epipolar geometry analysis and image morphing techniques based on control lines to produce in-between frames which simulate moving a video camera a the source vista to a destination vista. Epipolar geometry analysis is related to the relative alignment of the camera axes between images and will be discussed below.
In a first aspect, the method of the invention locates an epipole on the source vista and an epipole on the destination vista and aligns the source vista and the destination vista based on the located epipoles.
In another aspect, the method determines the alignment between the panoramic vistas from the epipole of each vista and an image flow between corresponding image features of the aligned panoramic vistas. The method also forms at predetermined times and based on the image flow, intermediate forward resampled images of one of the vistas and corresponding backward resampled images of another one of the vistas and merges at each predetermined time the forward resampled image and the backward resampled image to form a sequence of in-between images. The image sequence can be displayed as a video movie.
The invention may include one or more of the following features:
For example, the method selects a control line on the source vista and a corresponding control line on the destination vista and computes the image flow between pixels on the source vista and the destination vista based on the control lines.
The method forms at predetermined times and based on the computed image flow, intermediate forward resampled images of one of the vistas and corresponding backward resampled images of another one of the vistas, and merges the forward and backward resampled images to form a sequence of in-between images.
The corresponding control lines selected on the images completely surround the respective epipoles. The image flow of each pixel on the images can then be inferred from the image flow of pixels located on the control lines.
Locating the epipoles includes selecting corresponding pairs of epipolar lines on the source vista and on the destination vista and minimizing by an iterative process the sum of squared differences of a projected coordinate between an image pixel located on one vista and the image pixels located on the corresponding epipolar line on the other vista. Preferably, locating the epipoles includes reprojecting the source vista and the destination vista to produce respective source and destination view images and determining the epipoles from the reprojected view images.
The forward-resampled and backward-resampled image pixels are added as a weighted function of time to produce a sequence of in-between images, much like a video movie.
Forward-resampled and backward-resampled destination pixels that have either no source pixel (“hole problem”) or more than one source pixel (“visibility problem”) or that are closer to a set of control lines than a predetermined distance (“high-disparity pixels”) are treated special.
Other advantages and features full become apparent from the following description and from the claims.
REFERENCES:
patent: 5644651 (1997-07-01), Cox et al.
patent: 5655033 (1997-08-01), Inoguchi et al.
patent: 5703961 (1997-12-01), Rogina et al.
patent: 6078701 (2000-06-01), Hsu et al.
PanoVR SDK—A software development kit for integrating photo-realistic panoramic images and 3-D graphical objects into virtual worlds, by Chiang et al., ACM VRST, 1997, pp. 147-154.*
Chiang, C., “A Method of Smooth Node Transition in Panoramic Image-Based Virtual Worlds”, Advanced Tech. Ctr., Comp. and Comm. Res. Labs., ITRI, Taiwan.
Shenchang Eric Chen, “QuickTime VR-An Image-Based Approach to Virtual Environment Navigation” In Proc. SIGGRAPH 95, 1995.
Seitz et al., “View Morphing”, Department of Computer Sciences, University of Wisconsin-Madison, 1996.
Cheng Tse
Chiang Cheng-Chin
Hsieh Jun-Wei
Bali Vikkram
Fish & Richardson P.C.
Industrial Technology Research Institute
Mancuso Joseph
LandOfFree
Producing transitions between vistas does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Producing transitions between vistas, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Producing transitions between vistas will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2977223