Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
2000-07-13
2002-03-05
Mancuso, Joseph (Department: 2623)
Image analysis
Applications
3-d or stereo imaging analysis
C382S284000
Reexamination Certificate
active
06353678
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to the detection of independently moving objects in a sequence of two-dimensional video images representing a three-dimensional (3D) video scene and in particular, to a method that uses a multi-view camera motion constraint and a shape constancy constraint.
BACKGROUND OF THE INVENTION
Automatic methods for processing images of a 3D scene to detect motions that are independent of camera motion are used in applications such as aerial video surveillance and monitoring, rapid model building under uncontrolled scenarios and moving object tracking. The 2D image motion in the scenarios under consideration can be attributed to the camera motion, the shape of the 3D scene and objects, and independent object motion. Automatic methods for solving the problem need to deal with the confounding effects of the various causes of image motion. It may be difficult, for example, to detect a moving object in a scene imaged by a moving camera if the object moves in the same direction as the camera motion.
A particularly difficult case of 3D scenes are sparse 3D scenes in which the “3Dness” of the scene is sparsely distributed and the image parallax for the fixed scene and the independent motions may be equally dominant.
Previous attempts to automatically detect independent motion in 3D scenes have either employed only the epipolar constraints or have assumed that frame correspondences and/or image flows are available or can be reliably computed. One such system, described in an article by G. Adiv entitled “Determining 3D Motion and Structure from Optical Flows Generated by Several Moving Objects,”
IEEE Transactions on Pattern Analysis and Machine Intelligence
, vol. 7, no. 4, pp. 384-401, 1985. The system disclosed in this article assumes that the optical flow for the sequence of images is available and uses this flow to label points that belong to planes. Subsequently, the planar hypotheses are grouped on the basis of a rigidity constraint over two frames. In essence an epipolar constraint is applied to groups of planes.
Epipolar constraints may produce erroneous results, however, when independent object motion is in the same direction as camera motion. In this instance, the epipolar constraints may erroneously be calculated based on the independent object motion instead of the underlying scene. Image flows are time consuming to calculate and are subject to error, for example, if items in one frame are erroneously classified as matching objects in another frame.
SUMMARY OF THE INVENTION
The subject invention is embodied in a system and method that detects independently moving objects in 3D scenes that are viewed under camera motion. The subject invention first calculates 2D view geometry constraints for a set of images. These constraints are tested to determine if the imaged scene exhibits significant 3D characteristics. If it does, then 3D shape constraints, are applied to the set of images. The 3D shape constraints are themselves constrained by the 2D view geometry constraints. The set of images is then tested to identify areas that are inconsistent with the constraints. These areas correspond to the moving objects.
According to one aspect of the invention, the 2D view geometry constraints are calculated by computing a dominant image alignment for successive pairs of images and then computing constrained epipolar transformations for the two image pairs.
According to another aspect of the invention, the 2D view geometry is calculated based on a plurality of target point correspondences among the plurality of frames. The geometry corresponding to a minimum median error is selected as the 2D view geometry of the scene.
According to yet another aspect of the invention, the 3D shape constraint is a parallax geometry that is calculated by iteratively minimizing errors in a parametric transformation using an estimated parallax geometry, over a plurality of images.
REFERENCES:
patent: 4590607 (1986-05-01), Kauth
patent: 5259040 (1993-11-01), Hanna
patent: 5644654 (1997-07-01), Onokera
patent: 5821943 (1998-10-01), Shashua
patent: 5859922 (1999-01-01), Hoffmann
patent: 6049619 (2000-04-01), Anandan et al.
patent: 6137491 (2000-10-01), Szeliski
patent: 6192145 (2001-02-01), Anandan et al.
patent: 6198852 (2001-03-01), Anandan et al.
patent: WO 97/35161 (1997-09-01), None
PCT International Search Report, May 21, 2001.
J. R. Bergen et al., “Hierarchical Model-Based Motion Estimation,” Proceedings of European Conference on Computer Vision, 1992.
R. Cipolla et al., “Robust structure from motion using motion parallax,” Proceedings of the Fourth International Conference on Computer Vision, pp. 374-382, Apr. 1993.
H.S. Sawhney, “Simplifying multiple motion and structure analysis using planar parallax and image Warping,” Proceedings of the 1994 IEEE Workshop on Motion of Non-Rigid and Articulated Objects, pp. 104-109.
G. Adiv, “Determining 3D Motion and Structure from Optical Flows Generated by Several Moving Objects,” IEEE PAMI, pp. 384-401, 1985.
P. H.S. Torr et al., “Robust Prameterization and Computation of the Trifocal Tensor,” Image and Vision Computing, vol. 15, pp. 591-605, 1997.
M. Irani et al., “Computing Occluding and Transparent Motions,” International Journal of Computer Vision, vol. 12, pp. 5-16, 1994.
Z. Zhang, et al., “A Robust Technique for Matching Two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry,” Institut National de Recherche en Informatique et en Automatique, No. 2273, May 1994.
Kumar, et al., “Direct recovery of shape from multiple views: a parallax based approach,” Proc 12thIAPR International Conf on Pattern Recognition, Jerusalem, Israel, Oct. 1994, p. 685-688.
Guo Yanlin
Kumar Rakesh
Sawhney Harpreet
Bali Vikkram
Burke William J.
Mancuso Joseph
Sarnoff Corporation
LandOfFree
Method and apparatus for detecting independent motion in... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for detecting independent motion in..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for detecting independent motion in... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2833998