Example-based image synthesis suitable for articulated figures

Details Example-based image synthesis suitable for articulated figures Example-based image synthesis suitable for articulated figures

1999-05-26

2002-03-12

Mehta, Bhavesh (Department: 2621)

Image analysis

Image transformation or preprocessing

Changing the image coordinates

C382S284000, C382S287000, C382S300000, C345S629000, C358S450000

Reexamination Certificate

active

06356669

ABSTRACT:

FIELD OF THE INVENTION
The present invention is generally directed to computer graphics, and more particularly to image-based rendering of figures, by interpolating exemplary images of those figures.
BACKGROUND OF THE INVENTION
In the field of computer vision, two basic approaches have been employed to render, or synthesize, an image of an object or figure. One approach employs a geometric model of the object, which defines the underlying structure and/or behavior of the object. Once the model is known, it can be used to render an image of the object from any desired perspective. In many cases, however, a geometric model is not available for the desired object. In these cases, image-based rendering may be employed. Generally, image-based rendering involves the creation of a new image from a set of example images. In one approach, a number of exemplary images are obtained of the object in different poses or configurations, and point correspondence is established between the various images. Based upon this information, the data within the exemplary images is interpolated to produce a new image which illustrates the object in a designated pose or configuration.
The types of objects that can be synthesized by means of image-based rendering can be classified as either rigid or non-rigid. To date, much of the attention that has been given to the problem of image synthesis has been directed to rigid object transformation. Examples of the various solutions that have been proposed for this category of image synthesis include view morphing, plenoptic modeling with depth recovery, lightfields, and the use of trifocal tensors. In general, these techniques function to extrapolate the perspective geometry of a rigid object, to present a desired view.
Such techniques are not readily applicable to non-rigid objects which are capable of different configurations, such as a face which can have different expressions or a human body which can assume a variety of different configurations. For these types of subjects, more general image interpolation techniques have been employed. For example, techniques which are based on Radial Basis Functions or on Principle Components Analysis have been used to interpolate face images under varying pose, expression and identity conditions. Further information regarding these techniques can be found in the following publications: Beymer et al, “Example Based Image Analysis and Synthesis,”
MIT AI Lab Memo No
. 1431, MIT, 1993; Beymer et al,
Science
, 272:1905-1909, 1996; Jones et al, “Multi-Dimensional Morphable Models,”
Proc. ICCV
:98, Bombay, India, pp. 683-688, 1998; and Lanitis et al, “A Unified Approach To Coding and Interpreting Face Images,”
Proc. ICCV
-95, pp. 368-373, Cambridge, Mass., 1995, the disclosures of which are incorporated herein by reference. Unfortunately, these methods are limited in the types of object appearance that can be accurately portrayed. For instance, face analysis which employs Principle Components Analysis techniques typically assumes that images of face shape and texture fall within a linear subspace, and does not work in complex situations such as the different possible configurations for a human limb, or the like. Radial Basis Functions operate on the assumption that a smooth function is being modeled. They are not applicable to situations in which the data does not conform to a function, such as views in which a single control point location might be applicable to multiple different views.
It is desirable to be able to extend non-rigid image synthesis to those situations where the appearance of the object to be rendered is not a linear manifold nor a smooth function. One example of an object which falls into this category is an articulated body. The mapping from a control parameter to an associated appearance for articulated bodies is often one-to-many, due to the multiple configurations that are possible for a particular location for an end point of the body. Furthermore, the mapping will be discontinuous when constraints are present that require different solutions across a boundary in parameter space.
SUMMARY OF THE INVENTION
In accordance with the present invention, image-based synthesis for nonrigid bodies whose appearances do not form a linear manifold is carried out by representing mappings from control parameters to appearances as subsets of piecewise smooth functions. Each subset contains example images which are well approximated by particular examples which lie on the convex hull of the subset's parameter values. Once the subsets of examples are defined, interpolation is performed by using only the examples in a single subset.
By means of this approach, inconsistent example images are not combined during interpolation. Furthermore, the number of examples that are needed to fully interpolate a function is reduced, since only those examples which lie on the convex hull of an example subset need to be used. Any new example that falls within, and is well approximated by, the convex hull of an existing subset can be ignored during the interpolation process.
To provide for efficient operation of an image-based rendering technique, a method for automatically estimating the correspondence between images is preferably employed. Typically, optic-flow based methods are used for this purpose. However, articulated bodies are particularly difficult to analyze with such techniques. For instance, images of limbs with uniform clothing or skin color can be difficult to track by means of methods which require contrast to be present in the foreground of an image feature. In one embodiment of the invention, the contour of an articulated body can be tracked by means of an image transform which employs radial cumulative similarities, of the type disclosed in copending application Ser. No. 199,799, to find correspondences for rendering human figures in realistic imagery.
The features of the invention, and the advantages offered thereby, are explained in greater detail hereinafter with reference to particular examples that are described with the aid of the accompanying figures.


REFERENCES:
patent: 5325475 (1994-06-01), Poggio et al.
patent: 5416899 (1995-05-01), Poggio et al.
patent: 5745668 (1998-04-01), Poggio et al.
patent: 5774129 (1998-06-01), Poggio et al.

Affiliated with

Also associated with

Rating

Example-based image synthesis suitable for articulated figures does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Example-based image synthesis suitable for articulated figures, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Example-based image synthesis suitable for articulated figures will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2883246