Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
2000-06-28
2003-07-01
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C707S793000, C382S154000
Reexamination Certificate
active
06587601
ABSTRACT:
BACKGROUND OF THE INVENTION
The ability to locate scenes and/or objects visible in a video/image frame with respect to their corresponding locations and coordinates in a reference coordinate system is important in visually-guided navigation, surveillance and monitoring systems. Aerial video is rapidly emerging as a low cost, widely used source of imagery for mapping, surveillance and monitoring applications. The individual images from an aerial video can be aligned with one another and merged to form an image mosaic that can form a video map or provide the basis for estimating motion of objects within a scene. One technique for forming a mosaic from a plurality of images is disclosed in U.S. Pat. No. 5,649,032, issued Jul. 15, 1997, which is hereby incorporated herein by reference.
To form a “video map”, a mosaic (or mosaics) of images may be used as a database of reference imagery and associated “geo-coordinates” (e.g., latitude/longitude within a reference coordinate system) are assigned to positions within the imagery. The geo-coordinates (or other image or scene attributes) can be used to recall a mosaic or portion of a mosaic from the database and display the recalled imagery to a user. Such a searchable image database, e.g., a video map, is disclosed in U.S. patent application Ser. No. 08/970,889, filed Nov. 14, 1997, and hereby incorporated herein by reference.
A system that images a scene that has been previously stored in the reference database and recalls the reference information in response to the current images to provide a user with information concerning the scene would have applicability in many applications. For example, a camera on a moving platform imaging a previously imaged scene contained in a database may access the database using the coordinates of the platform. The system provides scene information to a user. However, a key technical problem of locating objects and scenes in a reference mosaic with respect to their geo-coordinates needs to be solved in order to ascertain the geo-location of objects seen from the camera platform's current location. In current systems for geo-location, the mapping of camera coordinates to the geo-coordinates, use position and attitude information for a moving camera platform within some fixed world coordinates to locate the video frames in the reference mosaic database. However, the accuracy achieved is only on the order of tens to hundreds of pixels. This inaccuracy is not acceptable for high resolution mapping.
Therefore, there is a need in the art for a method and apparatus that identifies a location within an imaged scene with a sub-pixel accuracy directly from the imagery within the scene itself.
SUMMARY OF THE INVENTION
The disadvantages of the prior art are overcome by the present invention of a system and method for accurately mapping between camera coordinates and geo-coordinates, called geo-spatial registration, using a Euclidean model to align and combine images. The present invention utilizes the imagery and terrain information contained in the geo-spatial database to precisely align the reference imagery with input imagery, such as dynamically generated video images or video mosaics, and thus achieve a high accuracy identification of locations within the scene. The geo-spatial reference database generally contains a substantial amount of reference imagery as well as scene annotation information, digital elevation maps (DEM, and object identification information. When a sensor, such as a video camera, images a scene contained in the geo-spatial database, the system recalls a reference image and DEM pertaining to the imaged scene. This reference image is aligned very accurately with the sensor's images using a parametric transformation derived from a Euclidean model. Thereafter, other information (annotation, sound, and the like) that is associated with the reference image can easily be overlaid upon or otherwise associated with the sensor imagery. Applications of geo-spatial registration include text/graphical/audio annotations of objects of interest in the current video using the stored annotations in the reference database to augment and add meaning to the current video. These applications extend beyond the use of aerial videos into the challenging domain of video/image-based map and database indexing of arbitrary locales, like cities and urban areas.
REFERENCES:
patent: 4179824 (1979-12-01), Marsh
patent: 4232313 (1980-11-01), Fleishman
patent: 5124915 (1992-06-01), Krenzel
patent: 5546529 (1996-08-01), Bowers et al.
patent: 5633946 (1997-05-01), Lachinski et al.
patent: 5649032 (1997-07-01), Burt et al.
patent: 5657402 (1997-08-01), Bender et al.
patent: 5724129 (1998-03-01), Matteucci
patent: 5764871 (1998-06-01), Fogel
patent: 5784431 (1998-07-01), Kalend et al.
patent: 5878356 (1999-03-01), Garrot, Jr. et al.
patent: 5920394 (1999-07-01), Gelbart et al.
patent: 5963664 (1999-10-01), Kumar et al.
patent: 5968109 (1999-10-01), Israni et al.
patent: 5995681 (1999-11-01), Lee et al.
patent: 6137491 (2000-10-01), Szeliski
patent: 6173067 (2001-01-01), Payton et al.
patent: 6396960 (2002-05-01), Yoshimura
Kumar, et al., “Registration of Video to Geo-referenced imagery”, Proc. 14thInt. Conf. Pattern Rec., Brisbane, QLD, Australia, IEEE Compu Soc, 2, 1998, p. 1393-1400.
HongJiang Zhang, Atreyi Kankanhalli, Stephen W. Smoliar, “Automatic Partitioning of Full-Motion Video”, Multimedia Systems, pp. 10-28, 1993.
Kumar et al. “Registration of Video to Geo-Referenced Imagery” IEEE, pp. 1393-1400, 1998.
Hsu et al. “Influence of Global Constraints and Lens Distortion of Pose and Appearance Recovery from a Purely Rotating Camera” IEEE, pp. 154-159, 1998.
Behringer, Reinhold “Registration for Outdoor Augmented Reality Applications Using Computer Vision Techniques and Hybrid Sensors”, IEEE, pp. 244-251, 1999.
Tarel et al. “A Coarse to Fine 3D Registration Method Based on Robust Fuzzy Clustering” Computer Vision and Image Understanding, vol. 73, No. 1, pp. 14-28, Jan. 1999.
Hsu Stephen Charles
Samarasekera Supun
Burke William J.
Mehta Bhavesh M.
Patel Kanji
Sarnoff Corporation
LandOfFree
Method and apparatus for performing geo-spatial registration... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for performing geo-spatial registration..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for performing geo-spatial registration... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3045780