Generalized panoramic mosaic

Details Generalized panoramic mosaic Generalized panoramic mosaic

1999-09-15

2003-03-11

Kelley, Chris (Department: 2613)

Television

Panoramic

Reexamination Certificate

active

06532036

ABSTRACT:

FIELD OF THE INVENTION
This invention relates to video image mosaicing for obtaining panoramic mosaics of a scene.
PRIOR ART
Prior art references considered to be relevant as a background to the invention are listed below. Acknowledgement of the references herein shall not be inferred as meaning that these are in any way relevant to the patentability of the invention disclosed herein. Each reference by a number enclosed in square brackets and accordingly the prior art will be referred to throughout the specification by numbers enclosed in square brackets.
[1
] ARPA Image Understanding Workshop,
Monterey, Calif., November 1994, Morgan Kaufmann.
[2
] Fifth International Conference on Computer Vision,
Cambridge, Mass., June 1995, IEEE-CS.
[3
] IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, Calif., June
1996.
[4] P. J. Burt and E. H. Adelson. A multiresolution spline with application to image mosaics.
ACM Trans. on Graphics,
2(4), pages 217-236, October 1983.
[5] P. J. Burt and P. Anandan. Image stabilization by registration to a reference mosaic. In
ARPA Image Understanding Workshop [
1], pages 425-434.
[6] S. E. Chen and L. Williams. View interpolation for image synthesis. In
SIGGRAPH,
pages 279-288, Anaheim, Calif., August 1993, ACM.
[7] T. R. Halfhill. See you around.
Byte Magazine,
pages 85-90, May 1995.
[8] M. Hansen, P. Anandan, K. Dana, G. van der Wal, and P. J. Burt. Real-time scene stabilization and mosaic construction. In
ARPA Image Understanding Workshop [
1], pages 457-465.
[9] M. Irani, P. Anandan, and S. Hsu. Mosaic based representations of video sequences and their applications. In
Fifth International Conference on Computer Vision [
2] page 605-611.
[10] P. Jaillon and A. Montanvert. Image mosaicing applied to three-dimensional surfaces. In 12
International Conference on Pattern Recognition,
pages 253-257, Jerusalem, Israel, October 1994, IEEE-CS.
[11] A. Krishnan and N. Ahuja. Panoramic image acquisition. In
IEEE Conference on Computer Vision and Pattern Recognition [
3], pages 379-384.
[12] S. Mann and R. Picard. Virtual bellows: Constructing high quality stills from video. In
First IEEE International Conference on Image Processing,
Austin, Tex., November 1994.
[13] L. McMillan and G. Bishop. Plenoptic modeling: An image-based rendering system. In
SIGGRAPH,
Los Angeles, Calif., August 1995, ACM.
[14] D. L. Milgram. Computer methods for creating photomosaics.
IEEE Trans. on Computers,
C-24, pages 1113-1119, 1975.
[15] D. L. Milgram. Adaptive techniques for photomosaicing.
IEEE Trans. on Computers,
C-26, pages 1175-1180, 1977.
[16] S. Peleg, Elimination of seams from photomosaics.
Computer Graphics and Image Processing,
16, pages 90-94, May 1981.
[17] B. Rousso, S. Avidan, A. Shashua, and S. Peleg. Robust recovery of camera rotation from three frames. In
IEEE Conference on Computer Vision and Pattern Recognition [
3], pages 796-802.
[18] H. S. Sawhney, S. Ayer, and M. Gorkani. Model-based 2D and 3D dominant motion estimation for mosaicing and video representation. In
Fifth International Conference on Computer Vision [
2], pages 583-590.
[19] S. Seitz and C. Dyer. Physically valid view synthesis by image interpolation. In
Proc. IEEE Workshop on Representation of Visual Scenes,
Cambridge, Mass., June 1995, IEEE-CS.
[20] R. Hartley and R. Gupta. Linear pushbroom cameras. In J. O. Eklundh, editor,
Third European Conference on Computer Vision,
pages 555-566, Stockholm, Sweden, May 1994, Springer.
[21] M. Irani, B. Rousso, and S. Peleg. Detecting and tracking multiple moving objects using temporal integration. In G. Sandini, editor,
Second European Conference on Computer Vision,
pages 282-287, Santa Margherita, Italy, May 1992, Springer.
[22] S. Peleg, J. Herman, D. Dixon, P. J. Burt, and J. R. Bergen. U.S. Patent Application—
Improved methods for mosaic image construction.
[23] R. Szeliski. Video mosaics for virtual environments.
IEEE Computer Graphics and Applications,
pages 22-30, March 1996.
[24] R. Szeliski and S. B. Kang. Direct methods for visual scene reconstruction. In
Proc. IEEE Workshop on Representation of Visual Scenes,
Cambridge, Mass., June 1995, IEEE-CS, pages 26-33.
[25] J. Y. Zheng and S. Tsuji. Panoramic representation for route recognition by a mobile robot International Journal of Computer Vision, Vol. 9, pages 55-76, 1992.
BACKGROUND OF THE INVENTION
The need to combine pictures into panoramic mosaics existed since the beginning of photography, since the camera's field of view is always smaller than the human field of view. Also, very often large objects cannot be captured in a single picture, and only photo-mosaicing enables a more complete view. Digital photography created new applications for mosaicing [14, 15, 16, 4, 24, 23], which were first implemented for aerial and satellite images.
Three major issues are important in traditional image mosaicing:
(i) Image alignment, which determines the transformation that aligns the images to be combined into a mosaic. Paper photo-mosaicing uses rigid transformations for alignment: picture translations (shifts) and rotations. Digital processing enables more general transformations, like affine or planar-projective.
(ii) Image cut and paste is necessary since most regions in the panoramic mosaic are overlapping, and are covered by more than one picture. The cut and paste process involves either a selection of a single image for each overlapping region, or some kind of a combination of all overlapping images.
(iii) Image blending is necessary to overcome the intensity difference between images, differences that are present even when images are perfectly aligned. Such differences are created by a dynamically changing camera gain.
The simplest mosaics are created from a set of images whose mutual displacements are pure image-plane translations. This is approximately the case with some satellite images. Such translations can either be computed by manually pointing to corresponding points, or by image correlation methods. Other simple mosaics are created by rotating the camera around its optical center using a special device, and creating a panoramic image which represents the projection of the scene onto a cylinder [7, 11, 12, 13] or a sphere. Since it is not simple to ensure a pure rotation around the optical center, such mosaics can be used only in limited cases.
In more general camera motions, which may include both camera translations and camera rotations, more general transformations for image alignment are used [5, 8, 9, 10, 18]. In most cases images are aligned pairwise, using a parametric transformation like an affine transformation or planar-projective transformation. These transformations include an intrinsic assumption regarding the structure of the scene, such as being planar. A reference frame is selected, and all images are aligned with this reference frame and combined to create the panoramic mosaic. These methods are therefore referred to as reference frame based methods.
Aligning all frames to a single reference frame is reasonable when the camera is far away and its motion is mainly a sideways translation and a rotation around the optical axis. Significant distortions are created when camera motions include other rotations.
FIG. 1
shows the effects of large rotations on reference frame based methods. The objects a, b, x, y, c, d, w, z are viewed from two cameras C
1
and C
2
. The image I
1
is selected to be a reference frame and image I
2
is projected onto that reference frame. Large rotations generate distortions when projecting on the reference frame, and the information derived from frames with such rotations is blurred, and almost useless. Moreover, in long sequences in which the

Affiliated with

Also associated with

Rating

Generalized panoramic mosaic does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Generalized panoramic mosaic, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Generalized panoramic mosaic will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3055742