Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
2000-07-07
2004-03-02
Dastouri, Mehrdad (Department: 2623)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S294000, C382S304000, C348S036000, C352S070000
Reexamination Certificate
active
06701030
ABSTRACT:
BACKGROUND
1. Technical Field
The invention is related to deghosting mosaiced images, and more particularly to a system and process for deghosting a mosaiced image of a scene produced by stitching together overlapping images of the scene, each of which was captured by a different camera from a different viewpoint.
2. Background Art
Typically, mosaiced images, such as a panorama mosaic, are created by stitching together images taken by a single camera at a fixed viewpoint that is rotated to capture images of the surrounding scene. However, more recently, such mosaics, and particularly panoramic mosaics, have been created using a series of cameras each capturing a different part of the surrounding scene from a different viewpoint. This type of system for capturing images and creating mosaiced images by stitching the corresponding images captured by the respective cameras has been employed in the authoring of panoramic videos. In such a system, the cameras are video cameras and the images captured by the cameras at the same moment in time are stitched together to form a frame of the panoramic video.
A particular problem associated with stitching images together that were captured by different cameras at different viewpoints is localized double images of objects in the scene, which is often referred to as parallax distortion or ghosting. While it is irrelevant for objects in the scene that are far away from the cameras, objects that are closer in can create a double image in the mosaiced image. It is akin to a person bringing an object closer and closer to their eyes. At some point the object appears as two instead of one because the person's eyes are viewing it from different viewpoints. In the context of a mosaiced image, a close-in object depicted in two of the images captured at different viewpoints will result in a double image in the overlap region of these images in mosaiced image. This problem can be particularly acute in the creation of a panoramic video as objects will typically move in and out of the scene and may come close enough to the camera rig to cause the aforementioned localized double image. Up to this time no provision has been made to compensate for this double imaging problem, with the exception of eliminating any close in objects in the scene. The present deghosting technique addresses this problem without necessitating the elimination of close in objects.
SUMMARY
The present invention is directed toward a system and process for deghosting mosaiced images created by stitching together images of a scene captured from different viewpoints. As discussed above, when images are stitched together (i.e., mosaiced) that were captured by different cameras at different viewpoints, the possibility of localized double images of objects exists. Essentially, this double imaging or ghosting will occur if an object in the scene is close in to the cameras capturing the images. However, in accordance with the present invention this localized ghosting can be compensated for by estimating the amount of local mis-registration and then locally warping each image in the mosaiced image to reduce any ghosting. Preferably, this is accomplished using the procedures described in U.S. Pat. No. 5,986,668 entitled “Deghosting Method and Apparatus for Construction of Image Mosaics”, the disclosure of which is hereby incorporated by reference. This method was developed to reduce distortions caused by the fact that typical mosaicing processes assume an idealized camera model. However, in actuality un-modeled radial distortion (i.e., that component of the radial distortion that cannot be adequately modeled in the mosaicing process), tangential distortion, and non-square pixel distortion, among others can cause the local mis-registrations which can be corrected for using the aforementioned method. The same procedures employed to reduce these types of distortion can be adapted to reduce ghosting as well. Essentially, the deghosting process involves computing the optical flow between all pairs of images, and inferring the desired local warps from these computations.
The foregoing deghosting technique is particularly useful in the context of generating panoramic video using a non-catadioptric, multi-camera panoramic video authoring system, when employed on a frame by frame basis. As mentioned previously, these types of authoring systems use a series of video cameras to capture different parts of the surrounding scene from different viewpoints. In such a system, the images captured by the cameras at the same moment in time are stitched together to form a frame of the panoramic video. However, objects will typically move in and out of the scene and may come close enough to the camera rig to cause the aforementioned localized double image. Given this, each mosaiced frame of the panoramic video would be deghosted in accordance with the present technique to reduce the localized mis-alignments that are responsible for the ghosting.
In addition to the just described benefits, other advantages of the present invention will become apparent from the detailed description which follows hereinafter when taken in conjunction with the drawing figures which accompany it.
REFERENCES:
patent: 4463380 (1984-07-01), Hooks, Jr.
patent: 5649032 (1997-07-01), Burt et al.
patent: 5963664 (1999-10-01), Kumar et al.
patent: 5986668 (1999-11-01), Szeliski et al.
patent: 5987164 (1999-11-01), Szeliski et al.
patent: 6018349 (2000-01-01), Szeliski et al.
patent: 6389179 (2002-05-01), Katayama et al.
patent: 6522787 (2003-02-01), Kumar et al.
Shum et al, Construction and Refinement of Panoramic Mosaics with Global and Local Alignment, Jan. 1998, IEEE, Sixth International Conference on Computer Vision, pp. 953-958.*
Richard Szeliski, Image Mosaicing for Tele-Reality Applications, 1994, IEEE, ISBN: 0-8186-6410, pp. 44-53.
Szeliski Richard S.
Uyttendaele Matthew T.
Dastouri Mehrdad
Lyon Richard L.
Lyon & Harr LLP
Microsoft Corporation
LandOfFree
Deghosting panoramic video does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Deghosting panoramic video, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Deghosting panoramic video will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3266544