Image analysis – Image segmentation
Reexamination Certificate
1998-11-30
2001-07-17
Patel, Jayanti K. (Department: 2779)
Image analysis
Image segmentation
C345S440000, C348S700000, C382S156000, C382S294000
Reexamination Certificate
active
06263103
ABSTRACT:
FIELD OF THE INVENTION
This invention relates generally to computer vision, and more particularly, to estimating characteristics of scenes represented by images.
BACKGROUND OF THE INVENTION
One general problem in computer vision is how to determine the characteristics of a scene from images representing the underlying scene. Following are some specific problems. For motion estimation, the input is usually a temporally ordered sequence of images, e.g., a “video.” The problem is how to estimate the projected velocities of various things—people, cars, balls, background moving in the video. Another problem deals with recovering real-world three-dimensional (3D) structure from a 2D image. For example, how to recover the shape of an object from a line drawing, a photograph, or a stereo pair of photographs. Yet another problem is how to recover high-resolution scene details from a low-resolution image.
Humans make these types of estimates all the time, frequently sub-consciously. There are many applications for machines to be able to do this also. These problems have been studied by many workers with different approaches and varying success for many years. The problem with most known approaches is that they lack machine learning methods that can exploit the power of modern processors within a general framework.
In the prior art, methods have been developed for interpreting blocks world images. Other prior art work, using hand-labeled scenes, has analyzed local features of aerial images based on vector codes, and has developed rules to propagate scene interpretations. However, these solutions are for specific one-step classifications, and therefore, cannot be used for solving a general class of low-level vision problems. Methods to propagate probabilities have been used, but these methods have not been put in a general framework for solving vision problems.
Alternatively, optical flow can be estimated from images by using a quad-tree to propagate motion information across scale. There, a brightness constancy assumption is used, and beliefs about the velocity of the optical flow is presented as a gaussian probability distribution.
SUMMARY OF THE INVENTION
The present invention analyzes statistical properties of a labeled visual world in order to estimate a visual scene from corresponding image data. The image data might be single or multiple frames; the scene characteristics to be estimated could be projected object velocities, surface shapes, reflectance patterns, or colors. The invention uses statistical properties gathered from labeled training data to form “best-guess” estimates or optimal interpretations of underlying scenes.
Accordingly, training data for typical images and scenes are synthetically generated. A parametric vocabulary for both images and scenes is generated. The probability of image parameters, conditioned on scene parameters (the likelihood function), is modeled, as is the probability of scene parameters, conditioned on neighboring scene parameters. These relationships are modeled with a Markov network where local evidence is propagated to neighboring nodes to determine the maximum a posteriori probability of the scene estimate.
Humans perform scene interpretations in ways that are largely unknown and certainly mathematically undeterminable. We describe a visual system that interprets a visual scene by determining the probability of every possible scene interpretation for all local images, and by determining the probability of any two local scenes neighboring each other. The first probability allows the visual system to make scene estimates from local image data, and the second probability allows the these local estimates to propagate. One embodiment uses a Bayesian method constrained by Markov assumptions.
The method according to the invention can be applied to various low-level vision problems, for example, estimating high-resolution scene detail from a low-resolution version of the image, and estimating the shape of an object from a line drawing. In these applications, the spatially local statistical information, without domain knowledge, is sufficient to arrive at a reasonable global scene interpretation.
Specifically, the invention provides a method for estimating scenes from images. A plurality of scenes are generated and an image is rendered for each scene. These form the training data. The scenes and corresponding images are partitioned into patches. Each patch is quantified as a vector, and the vectors are modeled as a probability density, for example, a mixture of gaussian distributions. The statistical relationship between patches are modeled as a Markov network. Local probability information is iteratively propagated to neighboring nodes of the network, and the resulting probability density at each node, a “belief,” is read to estimate the scene.
In one application of our invention, it is possible to estimate high-resolution details from a blurred, or lower-resolution image. A low-resolution image is the input “image” data, and the “scene” data are the image intensities of the high-resolution details. The invention can also be used to estimate scene motion from a sequence of images. In this application, the image data are the image intensities from two successive images of the sequence, and the scene data are successive velocity maps indicating projected velocities of the visible objects at each pixel position. Another application for our invention is shading and reflectance disambiguation.
REFERENCES:
patent: 5341142 (1994-08-01), Reis et al.
patent: 6141019 (2000-10-01), Roseborough et al.
patent: 6151424 (2000-11-01), Hsu
patent: 6199030 (2001-03-01), Stone
patent: 6208295 (2001-03-01), Dogan et al.
Regazzoni et al., “Multilevel GMRF-based Segmentation of Image Sequences”, University of Genova, Genova, Italy, IEEE, May 1992, pp. 713-716.*
Michael I. Jordan, et al.; “An Introduction to Variational Methods for Graphical Modes”;Learning in Graphical Models; Kluwer Academic Publishers.
Yair Weiss; “Belief Propagation and Revision in Networks with Loops”; Center for Biological and Computational Learning, Department of Brain and Cognitive Sciences, Paper No. 155; Massachusetts Institute of Technology, Nov., 1997.
Yair Weiss; “Interpreting Images by Propagating Bayesian Beliefs”;Advances in Neural Information Processing Systems, M.C. Mozer, M.I. Jordan and T. Petsche, editors; 1997.
Freeman William T.
Pasztor Egon C.
Brinkman Dirk
Mitsubishi Electric Research Laboratories Inc.
Patel Jayanti K.
LandOfFree
Estimating scenes using statistical properties of images and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Estimating scenes using statistical properties of images and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Estimating scenes using statistical properties of images and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2540011