Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
1999-12-02
2002-05-21
Au, Amelia M. (Department: 2623)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S284000
Reexamination Certificate
active
06393163
ABSTRACT:
BACKGROUND OF THE DISCLOSURE
1. Field of the Invention
The invention relate systems that process images and, more particularly, to systems that automatically generate a mosaic of individual images and process the mosaic.
2. Description of the Prior Art
Until recently, image processing systems have generally processed images, such as frames of video, still photographs, and the like, in an individual manner. Each individual frame or photograph is typically processed by filtering, warping, and applying various parametric transformations. After processing, the individual images are combined to form a mosaic, i.e., an image that contains a plurality of individual images. Additional image processing is performed on the mosaic to ensure that the seams between the images are invisible such that the mosaic looks like a single large image. The alignment of the images and the additional processing to remove seams is typically accomplished manually by a technician using a computer workstation, i.e., the image alignment and combination processes are computer aided. In such computer aided image processing systems, the technician manually selects processed images, manually aligns those images, and a computer applies various images combining processes to the images to remove any seams or gaps between the images. Manipulation of the images is typically accomplished using various computer input devices such as a mouse, trackball, keyboard and the like. Unfortunately, such manual mosaic construction is time consuming and costly. Furthermore, manual mosaic construction cannot be accomplished in real-time, i.e., the mosaic cannot be constructed as the images are generated by an image source such as a video camera. Consequently, the images in a real-time image generation system are stored for subsequent computer aided processing at a later time.
Since manually generated mosaics are costly, such mosaics do not find much practical use except in publishing applications and image retouching systems. Although mosaics hold much promise, the lack of an automatic mosaic construction system has limited their use.
Therefore, a need exists in the art for a system that automatically generates a mosaic from either pre-existing images or in real-time as the images are generated by an image source. Furthermore, a need exists for systems that utilize the automatically generated mosaic in various practical applications.
SUMMARY OF THE INVENTION
The present invention overcomes the disadvantages heretofore associated with the prior art by automatically generating a mosaic from a plurality of input images. Specifically, the invention is a mosaic construction system that sequentially executes an image alignment process and a mosaic composition process such that, from a sequence of images, the system automatically produces a mosaic for utilization by various applications. The invention is capable of constructing both dynamic and static mosaics. A dynamic mosaic includes imagery that is time variant, e.g., the mosaic is updated with new content over time, while the content of a static mosaic is time invariant.
More specifically, the image alignment process automatically aligns one input image to another input image, an input image to an existing mosaic (generated from previously occurring input images) such that the input image can be added to the mosaic, or an existing mosaic to an input image. In each of these instances, the coordinate system within with the aligned images is either the coordinate system of the input image, the coordinate system of the mosaic or an arbitrary reference coordinate system. The arbitrary reference coordinate system can be either time invariant or time variant.
Furthermore, the input image and mosaic can be aligned to one another within an image pyramid framework. As such, the system converts both the input image and the mosaic into Laplacian image pyramids and the alignment process is applied to the levels within the respective pyramids. Consequently, the system uses an accurate coarse-to-fine image alignment approach that results in sub-pixel alignment accuracy. The outcome of the alignment process is alignment information that defines the transformations required to achieve alignment, a dynamic mosaic, between the input image and the mosaic such that the mosaic can be updated with the image information contained in the input image and, in a static mosaic, between the images comprising the mosaic.
Once the alignment process is complete, the invention utilizes a mosaic composition process to construct (or update) a mosaic. The mosaic composition process contains two processes: a selection process and a combination process. The selection process automatically selects images for incorporation into the mosaic and may include masking and cropping functions. Once the selection process selects which image(s), or portions thereof, are to be included in the mosaic, the combination process combines the various images to form the mosaic. The combination process applies various image processing techniques, such as merging, fusing, filtering, image enhancement, and the like, to achieve a seamless combination of the images. The resulting mosaic is a smooth image that combines the constituent images such that temporal and spatial information redundancy are minimized in the mosaic.
The automatically generated mosaic finds many practical uses. Illustrative uses include: (1) a mosaic based display system including an image printing system, (2) a surveillance system and (3) a mosaic based compression system. The mosaic based display system permits a system user to display, manipulate, search and alter a mosaic. The mosaic based compression system exploits the temporal and spatial redundancy in image sequences and efficiently compresses the image information. The compression system can be used for compressing image information for storage in a storage device or can be used for compressing image information for transmission through a band-limited transmission channel. The surveillance system combines the mosaic construction system, the compression system and the display system to provide a comprehensive system for remote motion detection.
REFERENCES:
patent: 4835532 (1989-05-01), Fant
patent: 5034986 (1991-07-01), Karmann et al.
patent: 5040067 (1991-08-01), Yamazaki
patent: 5140416 (1992-08-01), Tinkler
patent: 5140647 (1992-08-01), Ise et al.
patent: 5187754 (1993-02-01), Currin et al.
patent: 5258837 (1993-11-01), Gormley
patent: 5262856 (1993-11-01), Lippman et al.
patent: 5325449 (1994-06-01), Burt et al.
patent: 5339104 (1994-08-01), Hong
patent: 5416606 (1995-05-01), Katayama et al.
patent: 5629988 (1997-05-01), Burt et al.
patent: 5657402 (1997-08-01), Bender et al.
patent: 5721624 (1998-02-01), Kumashiro et al.
Brown, L.G., A survey of image registration techniques, ACM computing surveys, vol. 24, No. 4, Dec. 1992.*
Hartley, Richard I., “Self-Calibration from Multiple Views with a Rotating Camera”, Lecture Notes in Computer Science, vol. 800 pp. 471-478, 1994.
Sun, Huifang and Kwok, Wilson, “Concealment of Damaged Block Transform Coded Images Using Projections onto Convex Sets”, IEEE Transactions on Image Processing, vol. 4, No. 4, Apr. 1995.
McLean, “Structured Video Coding”, Massachusetts Institute of Technology, 1991.
Bergen et al., “Hierarchical Model-Based Motion Estimation”, Proceedings of European Conference on Computer Vision, Mar. 23, 1992.
Michal Irani, “Isolating Multiple 2D Image Motions for Image Enhancement and for 3D Motion Analysis”, Hebrew University of Jerusalem, 1993.
Burt et al., “Enhanced Image Capture Through Fusion”, International Conference on Computer Vision Proceedings, 1993.
Burt et al., “Merging Images Through Pattern Decomposition”, Applications of Digital Processing VIII, SPIE, vol. 575, pp. 173-181, 1985.
Burt et al., “A Multiresolution Spline With Application to Image Mosaics”, ACM Transactions on Graphics, vol. 2, No. 4, pp. 217-236, Oct. 1983.
Bove et al., “Real-Time Decoding and Display of Structured Video”, IEEE ICMCS, May 1994.
Lundmark et al., “Image Sequence
Anandan Padmanabhan
Burt Peter J.
Hansen Michael W.
Hsu Stephen Charles
Irani Michal
Au Amelia M.
Burke William J.
Miller Martin
Sarnoff Corporation
LandOfFree
Mosaic based image processing system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Mosaic based image processing system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Mosaic based image processing system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2834887