Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
1998-08-31
2002-05-28
Boudreau, Leo (Department: 2721)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S154000, C382S282000, C382S284000
Reexamination Certificate
active
06396961
ABSTRACT:
TECHNICAL FIELD
The present invention relates, in general, to image processing systems and, more specifically, to a method and apparatus for fixating a camera on a target point using image alignment.
BACKGROUND OF THE INVENTION
During video surveillance or filming of a scene from a moving vehicle, such as an aircraft or car, a common problem is how to keep the camera centered on a stationary target in the scene. Such centering is desirable, because it results in the video looking stable and, therefore, easy to watch; video that does not stay locked on a point contains significant motions due to the moving platform. The motion makes the video look “jittery” and may even cause most of the desired scene to move out of the image sporadically. This problem is especially acute in situations where the field of view is narrow and the camera platform is subject to sudden changes in orientation, such as airborne surveillance.
The task of keeping a camera centered on a point is commonly referred to as “fixating” a point. Traditional approaches to fixating a point have primarily relied on mechanical stabilization to hold the camera steady. Typically, these approaches use gyroscopes to sense rotation of the camera platform, and motors to compensate for that rotation. However, the cost of such systems increases rapidly depending on the number of rotational axes, and the desired precision of the stabilization.
Recent approaches have added an electronic image matching device to supplement mechanical stabilization. The device operates on a small image region (typically of size 32×32 pixels) around the image center, and uses correlation matching to estimate the translational image motion of this region from one video frame to the next. With correlation matching, the amount of “image slip” may be measured. Image slip is the distance through which the center of the current image moves from the desired fixation point. This image slip may be converted to an angular correction and applied to a motorized pan/tilt camera mount in order to re-center the camera on the desired fixation point. In this manner, fixation may be achieved by a combination of mechanical stabilization and closed-loop pan/tilt control that uses correlation matching to measure the image slip. An implicit assumption of this correlation matching approach is that the patch motion in the image can be represented using a rigid <x,y> translation. If the patch motion differs from a pure translation, or if the patch contains insufficient texture, the motion estimates produced using this technique will contain errors. As these errors accumulate over time, the patch and the fixated point will drift off the target.
As is well known, there are video techniques for indicating moving objects from a stationary platform that operate by simply subtracting from the amplitude of each pixel of a current video image frame, the amplitude of that pixel in a stored preceding video image frame. This results in the substantial removal of those pixels that define stationary objects. Digital image processing techniques which may be employed in this frame difference approach are disclosed in U.S. Pat. No. 4,692,806, which issued on Sep. 8, 1987, and to Anderson et al., and is incorporated herein by reference.
Also incorporated herein by reference is an article by Burt et al. “Object tracking with a moving camera: an Application of Dynamic Motion Analysis,” IEEE Workshop on Visual Motion, Irvine Calif., March 1989. Burt et al. teaches that if the camera is moving, it is often possible to compensate for resulting image motion by electronically shifting and rotating successive image frames to achieve alignment prior to computing a frame difference. Electronic alignment is based on a simple image warp (e.g., based on an affine transform) that is effective when scene motion contains relatively little parallax, such as when the camera is rotating, but not translating, or when objects within the region of the camera's field of view occur in a relatively narrow range of depths.
Further incorporated herein by reference is U.S. Pat. No. 5,629,988 issued to Burt and Hanna on May 13, 1997. This patent discloses a method for removing unwanted camera motion from processed and displayed video. It matches image frames between successive camera images, or between images and a stored reference image to determine camera-induced motion.
None of these disclosures, however, address the problem of fixation in which a selected target point is to be maintained centered in the camera's field of view. The image alignment technique of the present invention is directed to overcoming the problem of fixation, especially in the presence of motor latencies.
SUMMARY OF THE INVENTION
To meet this and other needs, and in view of its purposes, the present invention provides a method and apparatus for fixating a camera on a target point using image alignment. In one embodiment, the method includes the following steps:
a) receiving a plurality of images representative of a scene; b) selecting, within the plurality of images, a first display reference image containing the target point at a first coordinate location; c) estimating, in a current image within the plurality of images, a transformation that maps the current image to the first display reference image; d) estimating a second coordinate location of the target point in the current image using the transformation; e) computing an image slip between the target point in the current image and the target point in the first display reference image; f) converting the image slip into an angular correction for fixating the camera; g) modifying the angular correction by using closed-loop control with the motorized mount; and h) warping the current image using the transformation to align the current image with the first display reference image to obtain a stable view.
It is understood that the foregoing general description and the following detailed description are exemplary, but are not restrictive, of the invention.
REFERENCES:
patent: 4692806 (1987-09-01), Anderson et al.
patent: 5557684 (1996-09-01), Wang et al.
patent: 5561617 (1996-10-01), Van der Wal et al.
patent: 5611000 (1997-03-01), Szeliski et al.
patent: 5629988 (1997-05-01), Burt et al.
patent: 5764786 (1998-06-01), Kuwashima et al.
patent: 5793901 (1998-08-01), Matsutake et al.
patent: 5852672 (1998-12-01), Lu
patent: 5963613 (1999-10-01), Navab
patent: 5963664 (1999-10-01), Kumar et al.
Burt et al. “Object tracking with a moving camera: an Application of Dynamic Motion Analysis,” IEEE Workshop on Visual Motion, Irvine, California (Mar. 1989).
J.R. Bergen et al., Hierarchical Model-Based Motion Estimation, Proceedings of the European Conference on Computer Vision (1992).
M. Hansen et al., Real-time Scene Stabilization and Mosaic Construction, Workshop on Applications of Computer Vision (1994).
L. Wixon et al. “Image Alignment for Precise Camera Fixation and Aim,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Jun. 1998).
B.K.P. Horn, “Closed-Form Solution of Absolute Orientation Using Unit Quaternions,” Journal of the Optical Society of America A (Apr. 1997).
Eledath Jayakrishnan
Hansen Michael Wade
Mandelbaum Robert
Mishra Deepam
Wixson Lambert Ernest
Burke William J.
Patel Kanji
Sarnoff Corporation
LandOfFree
Method and apparatus for fixating a camera on a target point... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for fixating a camera on a target point..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for fixating a camera on a target point... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2910606