Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
2000-07-13
2001-10-23
Au, Amelia M. (Department: 2621)
Image analysis
Applications
3-d or stereo imaging analysis
C382S104000, C382S153000, C382S278000
Reexamination Certificate
active
06307959
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention concerns machine vision and in particular, a correlation-based method and apparatus for processing a group of images of a scene to concurrently and iteratively obtain estimates of ego-motion and scene structure.
In machine vision applications, the motion of the camera rig moving through an environment (ego-motion) provides useful information for tasks such as navigation and self-localization within a map. Similarly, recovering the structure of the environment may be useful for tasks such as obstacle avoidance, terrain-based velocity control, and three-dimensional (3D) map construction. In general, the problems of estimating ego-motion and structure from a sequence of two-dimensional (2D) images are mutually dependent. Prior accurate knowledge of ego-motion allows structure to be computed by triangulation from corresponding image points. This is the principle behind standard parallel-axis stereo algorithms, where the baseline is known accurately from calibration. In this case, knowledge of the epipolar geometry provides an efficient mechanism for determining scene structure by searching for corresponding points.
If, on the other hand, prior information is available regarding the structure of the scene, then the ego-motion can be computed directly. Essentially, the space of all possible poses of the camera is searched for the pose for which the perspective projection of the environment onto the image plane most closely matches each image in the sequence of images. The ego-motion is the path from one pose to the next.
It is more difficult to obtain accurate estimates of both ego-motion and structure by analyzing a sequence of 2D images in which neither is known. Generally, algorithms that attempt to perform this function fall into two classes: (i) those that use the epipolar constraint and assume that the motion field is available, and (ii) those that utilize the “positive depth” constraint, also known as “direct” algorithms. A correlation-based approach is described in an article by M. Irani et al. entitled “Robust multi-sensor image alignment,”
Proceedings of the Sixth International Conference on Computer Vision
(
ICCV'
98), pp. 959-965, January 1998. This system uses correlation to align images obtained from multiple modalities (e.g. from visible and IR cameras). The described method, however uses a 2D motion model. In addition, Beaudet masks are employed to estimate first and second derivatives of the correlation surface.
Another system is described in an article by K. J. Hanna et al. entitled “Combining Stereo and Motion Analysis for Direct Estimation of Scene Structure,”
Proceedings of the International Conference on Computer Vision,
pp. 357-365, 1993. The system described in this paper relies on the image-brightness constraint and stereo image processing techniques to align images and determine scene structure.
SUMMARY OF THE INVENTION
The present invention is embodied in a correlation-based, iterative, multi-resolution algorithm that estimates both scene structure and the motion of the camera rig through an environment from a stream of images captured while the camera is moving through the environment.
The exemplary method uses a global ego-motion constraint to refine estimates of inter-frame camera rotation and translation. It also uses local window-based correlation to refine the current estimate of scene structure.
According to one aspect of the invention, each inspection image in the stream of images is aligned to a reference image by a warping transformation. The correlation is determined by analyzing respective Gaussian and Laplacian decompositions of the reference image and warped inspection images.
According to another aspect of the invention, the rotation and translation parameters of the warping transformation are determined by correlating regions in the respective inspection images to corresponding regions in the reference image.
According to yet another aspect of the invention, scene structure is determined on a pixel-by-pixel basis by correlating multiple pixels in a support region
According to another aspect of the invention, the correlation surfaces are modeled as parametric surfaces to allow easy recognition and rejection of outliers and to simplify computation of incremental refinements for ego-motion and structure.
According to another aspect of the invention, the method employs information from other sensors to provide an initial estimate of ego-motion and/or scene structure.
According to yet another aspect of the invention, the method operates using images captured by either single-camera rigs or multiple-camera rigs.
REFERENCES:
patent: 4797942 (1989-01-01), Burt
patent: 4799267 (1989-01-01), Kamejima et al.
patent: 5202928 (1993-04-01), Tomita et al.
patent: 5259040 (1993-11-01), Hanna
patent: 5818959 (1998-10-01), Webb et al.
patent: 6047078 (2000-04-01), Kang
patent: 0 786 739 A2 (1997-07-01), None
patent: 0 898 234 A1 (1999-02-01), None
Stefan Baten et al., Techniques for Autonomous, Off-Road Navigation, IEEE Intelligent Systems, Nov./Dec. 1998, pp. 57-65.*
Sull, et al. “Estimation of Motion and structure of planar surfaces from a sequence of monocular images,” Proc Comp Soc Conf on Comp Vision and Pattern Recognition, US, Los Alamitos, IEEE, Comp. Soc. Press, Jun. 1, 1991 p. 732-733.
K. J. Hanna et al. “Combining Stereo and Motion Analysis for Direct Estimation of Scene Structure,” Proceedings of the International Conference on Computer Vision, pp. 357-365, 1993.
K.J. Hanna, “Direct multi-resolution of ego-motion and structure from motion,” Proceedings of the IEEE Workshop on Visual Motion, pp. 156-162, 1991.
J. R. Bergen et al., “Hierarchical Model-Based Motion Estimation,” Proceedings of European Conference Computer Vision, 1992.
R. Cipolla et al., “Robust structure from motion using motion parallax,” Proceedings of the Fourth International Conference on Computer Vision, pp. 374-382, Apr. 1993.
M. Irani, “Isolating Multiple 2D Image Motions for Image Enhancement and for 3D Motion Analysis,” Scientific Council of the Hebrew University of Jerusalem.
H.S. Sawhney, “Simplifying multiple motion and structure analysis using planar parallax and image Warping,” Proceedings of the 1994 IEEE Workshop on Motion of Non-Rigid and Articulated Objects, pp. 104-109.
R. Kumar et al., “Sensitivity of the pose refinement problem to accurate estimation of camera parameters,” Proceedings in the Third International Conference on Computer Vision, pp. 365-369, 1990.
Mandelbaum, et al., “Terrain Reconstruction for Ground and Underwater Robots,” Proc. IEEE Intl Conf. On Robotics and Automation, San Francisco, CA Apr. 22-28, 2000.
Mandelbaum, et al., “Correlation-Based Estimation of Ego-motion and Structure from Motion and Stereo,” Proc. ICCV, Kerkyra, Greece, Sep. 1999.
Salgian, “Extended Terrain Reconstruction for Autonomous Vehicles,” SPIE Proc., v 4023, Enchanced and Synthetic Vision 2000, Oralndo, Florida, Apr. 24-28, 2000.
Mandelbaum Robert
Salgian Garbis
Sawhney Harpreet Singh
Au Amelia M.
Burke William J.
Sarnoff Corporation
Werner Brian P.
LandOfFree
Method and apparatus for estimating scene structure and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for estimating scene structure and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for estimating scene structure and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2617596