Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
1998-04-30
2001-04-17
Couso, Jose L. (Department: 2621)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S278000
Reexamination Certificate
active
06219462
ABSTRACT:
The present invention generally relates to image processing systems and, more particularly, to a method and apparatus for aligning images within a image processing system.
BACKGROUND OF THE DISCLOSURE
In many image processing systems it is necessary that images be aligned with one another to perform image merging or analysis. The phrase image processing, as used herein, is intended to encompass the processing of all forms of images including temporally unrelated images as well as images (frames) of a video signal, i.e., a sequence of temporally related images. Image alignment in an image processing system is necessary to create mosaics of multiple images, perform some forms of image compression, perform motion estimation and/or tracking and the like. Alignment (also known as registration) of images begins with determining a displacement field that represents the offset between the images and then warping one image to the other to remove or minimize the offset. The images may be taken from the same sensor or from two entirely different sensors, possibly of different modality (e.g., an infrared sensor and a visible sensor). Often, the displacement field that defines the offset between the images can be described as a global parametric transformation between the two images, such as an affine, quadratic, or a projective transformation. Many techniques have been developed for the parametric transformation of a pair of images.
Most flow-based techniques divide the registration process into two steps: first a flow-field is estimated, then, using regression, the global parametric transformation which best describes the flow field is found. However, often the local flow estimates are noisy and unreliable, resulting in poor registration accuracy and a lack of robustness.
To overcome this problem, direct gradient-based techniques have been developed. These techniques estimate the global transformation parameters by directly using local image intensity information without first computing a local flow-field. They achieve high registration accuracy since they avoid the noisy step of computing a local flow-estimation. However, these techniques assume that the intensity values of corresponding pixels in the two images are the same, which is known as the “brightness constancy assumption”. As a result, the applicability of direct gradient-based techniques is limited to situations when the images to be registered are substantially similar in appearance. Consequently, the direct gradient-based techniques cannot handle large changes in illumination and contrast between the images. Because the images need to be substantially similar to be registered using direct gradient-based techniques, images produced by sensors having different modality and/or containing a substantial range of motion cannot be accurately registered by direct gradient-based techniques.
Therefore, a need exists in the art for a method and apparatus that aligns images having substantial illumination differences between the images and/or a substantial amount of motion and/or other image differences that would otherwise make registration difficult.
SUMMARY OF THE INVENTION
The disadvantages of the prior art are overcome by the present invention of a method, which when given any local match measure, applies global estimation directly to the local match measure data, without first performing an intermediate step of local flow estimation. Any local match measure can be used as part of the inventive method, such as correlation, normalized-correlation, squared or absolute brightness difference, statistical measures such as mutual information, and the like. Global estimation constrains the analysis of the local match measure, thereby avoiding noisy local motion estimates, while still providing an accurate result for image registration.
In one embodiment of the invention, the inventive generalized global alignment method is used with a normalized-correlation match-measure to result in a global correlation-based alignment method which combines the robustness and accuracy of global alignment with the broad applicability of the normalized-correlation match measure. The inventive method overcomes many of the limitations of existing gradient-based and flow-based techniques. In particular, the novel method can handle large appearance differences between the images and large image motion within the image scene. Also, in contrast to the flow-based methods, the invention accurately registers imagery with sparse texture or feature content.
REFERENCES:
patent: 4797942 (1989-01-01), Burt
patent: 5202928 (1993-04-01), Tomita et al.
patent: 5220614 (1993-06-01), Crain
patent: 5259040 (1993-11-01), Hanna
patent: 5488674 (1996-01-01), Burt et al.
patent: 5557684 (1996-09-01), Wang et al.
patent: 5611000 (1997-03-01), Szeliski et al.
patent: 5629988 (1997-05-01), Burt et al.
patent: 5872630 (1999-02-01), Johs et al.
K.J. Hanna, “Direct multi-resolution estimation of ego-motion and structure from motion”, Proceedings of IEEE Workshop on Visual Motion, Princeton, NJ, Oct. 7-9, 1991.
J. Bergen, P. Anandan, K. Hanna, R. Hingorani, “Hierarchical Model-Based Motion Estimation”, Proc. of European Conference on Computer Vision-92, Mar. 23, 1992.
Anandan Padmanabhan
Irani Michal
Burke William J.
Couso Jose L.
Patel Kanji
Sarnoff Corporation
Silverio John V.
LandOfFree
Method and apparatus for performing global image alignment... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for performing global image alignment..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for performing global image alignment... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2555722