Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
2000-08-30
2004-05-18
Au, Amelia M. (Department: 2623)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S284000, C382S289000, C345S634000
Reexamination Certificate
active
06738532
ABSTRACT:
TECHNICAL FIELD
The present invention generally relates to image registration and, more particularly, to image registration using a two stage registration process.
DISCUSSION
Two different images of a common scene are often obtained so that different categories of information within the scene can be captured. For instance, the images may represent reflectances in different spectra, such as a first image in the visible wavelengths and the other image in the infrared wavelengths. One image may be a synthetic image, such as from synthetic aperture radar, and the other image may be an electro-optical image, such as from a visible or infrared system. On the other hand, both images may be synthetic, such as from a synthetic aperture radar and a digital graphic map.
To understand the images captured, image registration is required. Image registration involves the computation of a mathematical transformation which maps or transforms pixel coordinates of one image into the associated pixel coordinates of the other image. Digital images often need to be registered so that pixels associated with specific locations in one image can be associated with pixels of exactly the same locations in a second image.
Image registration can be quite difficult because of the number of degrees of freedom involved in the registration process. The parameters involved include translation in two dimensions, rotation around a point in the image plane, and perspective of the scene from the viewpoint (which involves the relative displacement between the observation point and the central point of view, the orientation of the imaging device at that point, and the imaging characteristics of the imaging device, such as focal length, etc.).
In order to reduce image registration difficulty, registration is typically performed between images that have the same viewing geometry of the same scene. In this way, the images have large differences in only two parameters, that is, translation in the coordinate axes of the images, e.g., X and Y. Although other parameters may vary, those differences constrain the applicability and usability of the registration process by introducing error that can defeat or mislead the registration operation.
Numerous techniques, primarily based on statistical correlation or information measures, have been developed and improved to solve offset differences with optimal performance. Such techniques affect a rapid and efficient search through the range of possible offsets and generate a “goodness of fit” measure at each hypothesized offset. The location offering the best fit is taken to derive the offset mapping.
On the other hand, relative rotation between images is extremely expensive to compute. Techniques comparable to correlation or information measures have not yet been developed to rapidly and efficiently search through the space of possible rotations. In addition, rotational registration couples with the offset since rotation of one image is seen as being around some point in its plane. As a result, searching is typically done using a brute force technique. For example, one image is rotated, then a rapid offset registration process is performed, and the sequence is repeated until a best goodness of fit measure is obtained. The rotation at which that best fit is obtained is then taken to indicate the relative rotation between the images.
Although many approaches have been employed to improve rotational registration, they all suffer from certain deficiencies. Such deficiencies include multiple reduced resolution stages which require extra processing, rotational matching impracticality due to computational requirements, unsuitability due to expensive Fourier methods, and lack of quality assurance to evaluate the registration mapping. In view of the foregoing, an improved registration process is needed.
SUMMARY OF THE INVENTION
In view of the foregoing, it is in object of the present invention to provide a two-stage process with a reduced resolution stage first serving to give approximate parameters that are refined in a subsequent stage in a computationally efficient manner.
It is another object of the present invention to use a reduced resolution stage so that brute force matching over a range of rotational differences can be effectively applied.
It is yet another object of the present invention to obtain both angle and offset approximations during the reduced resolution stage to make the subsequent stage more efficient.
It is still yet another object of the present invention to register images of vastly different resolutions.
It is another object of the present invention to use small patch registration in a partially registered image to make control point matching more efficient.
It is yet another object of the present invention to use patch location adjustment to make control point matching more robust.
It is still yet another object of the present invention to use patch layout and patch matching evaluation to measure control point match quality.
It is another object of the present invention to use geometric figure mapping from a fiducial image space to the registered image space to determine registration quality.
With the foregoing objects in mind, the present invention includes capturing a first original image of a scene under surveillance, capturing a second original image of the scene under surveillance, coarsely matching the first original image to the second original image to give approximate angle and offset parameters of the first original image to the second original image, and finely matching the first original image to the second original image based on the approximate angle and offset parameters of the coarsely matching step. The coarsely matching step involves applying a plurality of rotations to a reduced version of the first original image to form a rotated and reduced first original image and performing a correlation match between the rotated and reduced first original image and a reduced version of the second original image. The finely matching step employs control point matching between the first original and the second original image using small image patches at the scale of the lower resolution of the two images.
During coarse matching, the reduced first original image is rotated by a sequence of angle increments that in combination encompass a range of possible rotational mismatches between the first original image and the second original image. The rotated and reduced first original image and the reduced second original image are transformed into the Fourier domain by subjecting each to a Fourier transform to produce Fourier images including an array of spatial frequency coefficients. A complex conjugate transform is then applied to one of the Fourier images. Thereafter, the Fourier images are correlated by pixel multiplication in the Fourier domain to yield a product image.
An inverse Fourier transform is applied to the product image to yield a correlation image having pixel values representing correlation coefficients corresponding to offsets given by row and column coordinates of a correlation coefficient pixel. A peak correlation coefficient is determined for the correlation image by examining all the correlation coefficients and identifying the largest one, and its location by row and column offset in the correlation image. A set of peak correlation coefficients is collected, one for each different rotation of the rotated and reduced first original image, along with a correlation peak-to-side-lobe (PSR) value, correlation histogram, and the peak location. A rotational difference is then identified between the first and second original images by examining the set of peak correlation coefficients for a maximum correlation coefficient. The PSR and correlation histogram are evaluated to determine if the maximum peak is of sufficient quality to proceed. If it is not, a secondary peak having the best PSR and correlation histogram is selected as the maximum peak. An approximate shift offset is identified between the first and second original images by determining ro
Au Amelia M.
Harness & Dickey & Pierce P.L.C.
LaRose Colin
LandOfFree
Image registration using reduced resolution transform space does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image registration using reduced resolution transform space, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image registration using reduced resolution transform space will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3243177