Wavelet-based facial motion capture for avatar animation

Details Wavelet-based facial motion capture for avatar animation Wavelet-based facial motion capture for avatar animation

2001-05-31

2003-06-17

Patel, Jayanti K. (Department: 2621)

Image analysis

Applications

Target tracking or detecting

C348S169000

Reexamination Certificate

active

06580811

ABSTRACT:

FIELD OF THE INVENTION
The present invention relates to dynamic facial feature sensing, and more particularly, to a vision-based motion capture system that allows real-time finding, tracking and classification of facial features for input into a graphics engine that animates an avatar.
BACKGROUND OF THE INVENTION
Virtual spaces filled with avatars are an attractive way to allow for the experience of a shared environment. However, existing shared environments generally lack facial feature sensing of sufficient quality to allow for the incarnation of a user, i.e., the endowment of an avatar with the likeness, expressions or gestures of the user. Quality facial feature sensing is a significant advantage because facial gestures are a primordial means of communications. Thus, the incarnation of a user augments the attractiveness of virtual spaces.
Existing methods of facial feature sensing typically use markers that are glued to a person's face. The use of markers for facial motion capture is cumbersome and has generally restricted the use of facial motion capture to high-cost applications such as movie production. Accordingly, there exists a significant need for a vision based motion capture systems that implements convenient and efficient facial feature sensing. The present invention satisfies this need.
SUMMARY OF THE INVENTION
The present invention is embodied in an apparatus, and related method, for sensing a person's facial movements, features or characteristic. The results of the facial sensing may be used to animate an avatar image. The avatar apparatus uses an image processing technique based on model graphs and bunch graphs that efficiently represent image features as jets composed of wavelet transforms at landmarks on a facial image corresponding to readily identifiable features. The sensing system allows tracking of a person's natural characteristics without any unnatural elements to interfere with the person's natural characteristics.
The feature sensing process operates on a sequence of image frames transforming each image frame using a wavelet transformation to generate a transformed image frame. Node locations associated with wavelets jets of a model graph to the transformed image frame are initialized by moving the model graph across the transformed image frame and placing the model graph at a location in the transformed image frame of maximum jet similarity between the wavelet jets at the node locations and the transformed image frame. The location of one or more node locations of the model graph is tracked between image frames. A tracked node is reinitialized if the node's position deviates beyond a predetermined position constraint between image frames.
In one embodiment of the invention, the facial feature finding may be based on elastic bunch graph matching for individualizing a head model. Also, the model graph for facial image analysis may include a plurality of location nodes (e.g., 18) associated with distinguishing features on a human face.
Other features and advantages of the present invention should be apparent from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the invention.


REFERENCES:
patent: 4725824 (1988-02-01), Yoshioka
patent: 4805224 (1989-02-01), Koezuka et al.
patent: 4827413 (1989-05-01), Baldwin et al.
patent: 5159647 (1992-10-01), Burt
patent: 5168529 (1992-12-01), Peregrim et al.
patent: 5187574 (1993-02-01), Kosemura et al.
patent: 5220441 (1993-06-01), Gerstenberger
patent: 5280530 (1994-01-01), Trew et al.
patent: 5333165 (1994-07-01), Sun
patent: 5383013 (1995-01-01), Cox
patent: 5430809 (1995-07-01), Tomitaka
patent: 5432712 (1995-07-01), Chan
patent: 5511153 (1996-04-01), Azarbayejani et al.
patent: 5533177 (1996-07-01), Wirtz et al.
patent: 5550928 (1996-08-01), Lu et al.
patent: 5581625 (1996-12-01), Connell
patent: 5588033 (1996-12-01), Yeung
patent: 5680487 (1997-10-01), Markandey
patent: 5699449 (1997-12-01), Javidi
patent: 5714997 (1998-02-01), Anderson
patent: 5715325 (1998-02-01), Bang et al.
patent: 5719954 (1998-02-01), Onda
patent: 5736982 (1998-04-01), Suzuki et al.
patent: 5764803 (1998-06-01), Jacquin et al.
patent: 5774591 (1998-06-01), Black et al.
patent: 5802220 (1998-09-01), Black et al.
patent: 5809171 (1998-09-01), Neff et al.
patent: 5828769 (1998-10-01), Burns
patent: 5917937 (1999-06-01), Szeliski et al.
patent: 5982853 (1999-11-01), Libermann
patent: 5995119 (1999-11-01), Cosatto et al.
patent: 6011562 (2000-01-01), Gagné
patent: 6044168 (2000-03-01), Tuceryan et al.
patent: 6052123 (2000-04-01), Lection et al.
patent: 6301370 (2001-10-01), Steffens et al.
patent: 44 06 020 (1995-06-01), None
patent: 0807902 (1997-11-01), None
patent: WO99/53443 (1999-10-01), None

Affiliated with

Also associated with

Rating

Wavelet-based facial motion capture for avatar animation does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Wavelet-based facial motion capture for avatar animation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Wavelet-based facial motion capture for avatar animation will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3133186