Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
2000-11-20
2004-02-17
Patel, Jayanti K. (Department: 2625)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S294000, C348S144000
Reexamination Certificate
active
06694064
ABSTRACT:
TECHNICAL FIELD
This invention relates to methods and systems for geometric alignment of overlapping digital images and, specifically, to a computer-implemented method of creating, from a set of component images, a single, seamless composite image of the entire area covered by the set of component images. The method of the present invention has particular applicability to vertical-viewing aerial imagery, but can also be applied to other types of imagery.
BACKGROUND OF THE INVENTION
In earth imaging operations, a set of images is typically acquired from an airplane or satellite in earth orbit using an image acquisition system. The image acquisition system may include a digital camera capable of capturing multispectral or hyperspectral single frame images. Alternatively, the images may be captured by other methods, for example, by conventional film aerial photography, later scanned to create digital images. Multispectral images comprise pixel intensity data in up to 10 spectral bands (e.g., red, green, blue, and near-IR) with relatively broad bandwidth (25 to 150 nm), while hyperspectral images comprise data for a larger number of spectral bands (typically numbering in the hundreds) with a narrow bandwidth (typically 1 to 10 nm).
Each image is defined by an image data record comprising a three-dimensional pixel data array, with X columns of pixels and Y rows of pixels for each of n spectral bands. The image data record is captured by the digital camera and stored on a computer-readable storage device, such as a disk drive or memory card. Typical digital images may be 1000 pixels×1000 pixels in 7 spectral bands, or 4000 pixels×4000 pixels in 4 spectral bands, or even 9000 pixels×9000 pixels in a single spectral band (“black and white” or panchromatic).
For example,
FIG. 1
shows an aircraft
2
carrying a prior art image acquisition system such as the ADAR System 5500 sold by the assignee of the present invention, Positive Systems, Inc., Whitefish, Mont. A GPS receiver of the image acquisition system (not shown) utilizes signals
4
from GPS satellites
6
in earth orbit to accurately determine the position and altitude of the aircraft
2
. The angular orientation of the aircraft
2
(and, consequently, the image acquisition system) may also be measured by a gyroscope or accelerometer subsystem, which is integrated with the GPS receiver in an inertial measurement unit (“IMU”). Orientation is typically indicated by three angles measured by the IMU, namely, Phi (&phgr;), Omega (&ohgr;), and Kappa (K) which represent angular displacement about the respective X, Y, and Z axes, where X is parallel to the aircraft wings, Y is parallel to the aircraft body, and Z runs vertically through the aircraft. Although not typical, the position and orientation data could easily be recorded in a coordinate reference frame other than the Cartesian coordinate system. For example, position and orientation data can be recorded in a polar coordinate frame of reference. Digital image data is often acquired by time-interval photography and stored by the image acquisition system in association with contemporaneous position, orientation, and timing data. To increase accuracy, the position and orientation data is collected from the GPS receiver (or an alternative source such as a GLONASS receiver, a LORAN receiver, manual data entry, etc.) at the same moment when the image is captured. GPS-sensed position, altitude, and orientation data is not required, but can aid in automation of the mosaicking process, as described below.
FIG. 1
illustrates the sequential acquisition of a set of digital images
10
such that the images
10
overlap to ensure complete coverage of the area being imaged and to provide a basis for alignment of the images
10
relative to each other.
FIG. 2A
depicts four adjacent images A, B, C and D that include overlapping regions
20
.
FIG. 2B
depicts a composite image called a mosaic
30
that depicts the surface area covered by the set of adjacent images A-D (FIG.
2
A). Commercially available GPS equipment and orientation sensors are not capable of measuring position, altitude, and orientation with sufficient accuracy for the creation of the mosaic
30
so that no visible image alignment errors are present. Therefore, to provide a geometrically seamless mosaic
30
, the adjacent images A, B, C and D must be manipulated to reduce misalignment. Further, a truly seamless mosaic must be adjusted radiometrically to ensure a uniform image brightness at the overlapping boundaries of the image frames.
Automated prior art methods of facilitating the alignment of overlapping images to form a mosaic are computationally intensive because they require correlation calculations to be made at a large number of locations in the subject images. Conversely, the amount of computation required can be reduced by limiting the number of locations where the correlation coefficients are calculated, but not without affecting the quality of the resulting image alignment.
For example, U.S. Pat. No. 5,649,032 of Burt et al. describes a method of automatically creating a mosaic image from multiple, overlapping still images captured by a video camera. Image alignment may be performed in batch, recursive, or hierarchical modes involving generation of the mosaic from a “pyramid image” by first tiling a subset of the component images in coarse alignment, then progressively improving alignment of the tiled images by calculating affine transformations at progressively greater resolution. This method does not involve selection of possible tie point locations based on whether the image data at a location on a subject image is likely to have a matching location on a target image. Rather, the method involves searching for matching locations on the target image regardless of the quality of the image data at the corresponding location of the subject image.
U.S. Pat. No. 5,187,754 of Currin et al. describes a method of forming a composite, image mosaic from aerial photographs with the aid of an overview image obtained, e.g., by satellite. Tie points or ground control points are painstakingly identified manually by an operator using a computer mouse. Overlapping images are then automatically aligned by a tie point correlation method. The method does not involve automated selection of possible tie point locations.
U.S. Pat. Nos. 5,528,290 of Saund et al. and 5,581,637 of Cass et al. describe a system for creating a composite, mosaic image of a classroom whiteboard using a motorized camera pivotally mounted at a fixed location relative to the whiteboard. The camera pivots to capture multiple overlapping image frames, which are transmitted for reassembly at a viewing location. Landmarks projected or marked on the whiteboard in locations where the image frames overlap are selected using a gradient analysis applied at all pixel locations in the overlapping region of the images. Images are then aligned on a “board coordinate system” frame of reference by applying a weighted perspective distortion correction based upon a significance factor of the landmarks identified. This method is computationally expensive because it requires each pixel location in the overlapping area to be analyzed for the presence of a landmark. It would not be suitable for aerial imaging applications in which greater numbers of images at a much higher resolution than a video camera must be aligned accurately relative to geographic coordinates to form seamless mosaic images.
Thus, a need exists for a more efficient method and system of selecting tie point pairs in overlapping images for use in aligning the overlapping images to form a mosaic image. Methods suitable for use in aerial image processing applications are also needed.
SUMMARY OF THE INVENTION
In accordance with the present invention, a method of automatically creating mosaic images is implemented in a computer usable medium. The method involves obtaining a plurality of images including overlapping areas, identifying one or more a search site points (SSPs) in the overlapping areas
Patel Jayanti K.
Positive Systems, Inc.
Stoel Rives LLP
Tabatabai Abolfazl
LandOfFree
Digital aerial image mosaic method and apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Digital aerial image mosaic method and apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Digital aerial image mosaic method and apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3340187