Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
2001-03-09
2003-07-22
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S284000, C382S305000, C707S793000
Reexamination Certificate
active
06597818
ABSTRACT:
The invention is generally related to image processing systems and, more specifically, to a method and apparatus for performing geo-spatial registration within an image processing system.
BACKGROUND OF THE INVENTION
The ability to locate scenes and/or objects visible in a video/image frame with respect to their corresponding locations and coordinates in a reference coordinate system is important in visually-guided navigation, surveillance and monitoring systems. Aerial video is rapidly emerging as a low cost, widely used source of imagery for mapping, surveillance and monitoring applications. The individual images from an aerial video can be aligned with one another and merged to form an image mosaic that can form a video map or provide the basis for estimating motion of objects within a scene. One technique for forming a mosaic from a plurality of images is disclosed in U.S. Pat. No. 5,649,032, issued Jul. 15,1992, which is hereby incorporated herein by reference.
To form a “video map”, a mosaic (or mosaics) of images may be used as a database of reference imagery and associated “geo-coordinates” (e.g., latitude/longitude within a reference coordinate system) are assigned to positions within the imagery. The geo-coordinates (or other image or scene attributes) can be used to recall a mosaic or portion of a mosaic from the database and display the recalled imagery to a user. Such a searchable image database, e.g., a video map, is disclosed in U.S. patent application Ser. No. 08/970,889, filed Nov. 14, 1997, and hereby incorporated herein by reference.
A system that images a scene that has been previously stored in the reference database and recalls the reference information in response to the current images to provide a user with information concerning the scene would have applicability in many applications. For example, a camera on a moving platform imaging a previously imaged scene contained in a database may access the database using the coordinates of the platform. The system provides scene information to a user. However, a key technical problem of locating objects and scenes in a reference mosaic with respect to their geo-coordinates seen from the camera platform's current location. Current systems for geo-location, the mapping of camera coordinates to the geo-coordinates, use position and attitude information for a moving camera platform within some fixed world coordinates to locate the video frames in the reference mosaic database. However, the accuracy achieved is only on the order of tens to hundreds of pixels. This inaccuracy is not acceptable for high resolution mapping.
Therefore, there is a need in the art for a method and apparatus that identifies a location within an imaged scene with a sub-pixel accuracy directly from the imagery within the scene itself. There is also a need to extend current geo-registration systems to handle other types of video, e.g., (1) zoomed in video where any one video frame may not have enough information for accurate geo-registration, (2) highly oblique video and video that is drastically different in photometric appearance compared to a reference image.
SUMMARY OF THE INVENTION
The present invention provides a system and method for accurately mapping between camera coordinates and geo-coordinates, called geo-spatial registration. The present invention utilizes the imagery and terrain information contained in the geo-spatial database to precisely align the reference imagery with input imagery, such as dynamically generated video images or video mosaics, and thus achieve a high accuracy identification of locations within the scene. The geo-spatial reference database generally contains a substantial amount of reference imagery as well as scene annotation information and object identification information. When a sensor, such as a video camera, images a scene contained in the geo-spatial database, the system recalls a reference image pertaining to the imaged scene. This reference image is aligned very accurately with the sensor's images using a parametric transformation. Thereafter, other information (annotation, sound, and the like) that is associated with the reference image can easily be overlaid upon or otherwise associated with the sensor imagery. Applications of geo-spatial registration include text/graphical/audio annotations of objects of interest in the current video using the stored annotations in the reference database to augment and add meaning to the current video. These applications extend beyond the use of aerial videos into the challenging domain of video/image-based map and database indexing of arbitrary locales, like cities and urban areas.
Alternative embodiments of the invention utilize one or more of the following techniques in performing geo-registration include iterative image alignment bundle estimation processes, multiframe processing, orientation filtering, outlier rejection, local-to-global image alignment, three-dimensional reference image rendering, and iterative image processing. Such processing enables geo-registration of oblique video imagery and to minimize occlusion effects in the geo-registered imagery. Another alternative embodiment utilizes continuous image processing to yield a continuous stream of geo-registered imagery.
REFERENCES:
patent: 3712563 (1973-01-01), Alpers
patent: 4179824 (1979-12-01), Marsh
patent: 4232313 (1980-11-01), Fleishman
patent: 5124915 (1992-06-01), Krenzel
patent: 5592573 (1997-01-01), Eisenbarth et al.
patent: 5596494 (1997-01-01), Kuo
patent: 5633946 (1997-05-01), Lachinski et al.
patent: 5649032 (1997-07-01), Burt et al.
patent: 5657402 (1997-08-01), Bender et al.
patent: 5784431 (1998-07-01), Kalend et al.
patent: 5835386 (1998-11-01), Orr et al.
patent: 5894323 (1999-04-01), Kain et al.
patent: 5956427 (1999-09-01), Greenspan et al.
patent: 5963664 (1999-10-01), Kumar et al.
patent: 5968109 (1999-10-01), Israni et al.
patent: 6023665 (2000-02-01), Millgard
patent: 6084989 (2000-07-01), Eppler
patent: WO 95 32483 (1995-11-01), None
“Registration of Video to Geo-Referenced Imagery”, R. Kumar et al., Proceedings of the Int. Conf. on Pattern Recognition, Aug. 1998, vol. 2, pp. 1393-1400.
PCT International Search Report, international application No. PCT/US01/07770, date of mailing Mar. 11, 2001.
Hong Jiang Zhang, et al., “Automatic Partitioning of Full-Motion Video”, Multimedia Systems (1993), pp. 10-28.
Hanna Keith
Hirvonen David James
Hsu Stephen Charles
Klinedinst Thomas Edward
Kumar Rakesh
Burke William J.
Mehta Bhavesh M.
Patel Kanji
Sarnoff Corporation
LandOfFree
Method and apparatus for performing geo-spatial registration... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for performing geo-spatial registration..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for performing geo-spatial registration... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3085303