Computer graphics processing and selective visual display system – Display driving control circuitry – Controlling the condition of display elements
Reexamination Certificate
1994-11-30
2001-05-15
Hong, Stephen S. (Department: 2176)
Computer graphics processing and selective visual display system
Display driving control circuitry
Controlling the condition of display elements
C345S473000
Reexamination Certificate
active
06232965
ABSTRACT:
FIELD OF THE INVENTION
The present invention defines techniques allowing a computer to simulate an animated image of a human speaking. More specifically, the present invention uses special techniques to simulate human facial expressions associated with various speaking patterns.
BACKGROUND AND SUMMARY OF THE INVENTION
Computer animation has been used to produce computer-generated pictures associated with various characteristics. Usually a computer animation is used to produce a moving animational system. As the users speak, their mouths move, but the movement of the mouths of the speakers and their speech has not been synchronized. This does not bother the viewer, however, since it appears to be a cartoon; and is not intended to be accurate.
The inventors of the present invention recognized that usual computer animation does not provide a sufficiently accurate picture of a user speaking to allow it to be used as a facsimile of that user speaking. That is, under the current state of the art, the inventors of the present invention recognized that a viewer of the computer animation would never be fooled into believing that the computer animation was real. They set about trying to find a way to solve this problem.
The inventors recognized, for the first time, that morphing technology could be used to simulate moving facial characteristics. Morphing technology is well known in the art: it is used to simulate a continuous change from a first image of a first object into an image of a second object. For example, it is easy to morph as apple into an orange. While one is looking at the apple, one sees its characteristics gradually change. It gradually assumes the shape of the orange, and also gradually assumes the color of the orange.
Morphing is well known, but a brief explanation of its operation will be given here anyway. Morphing involves transforming a first object, an “original object” into a second object, a “destination object”. The computer takes the original object and the destination object, and maps various points thereof. These points define the shape and contour of both objects as well as the colors at the various points. Morphing can be carried out using a number of different techniques. For simplicity, we can assume that a small number of points, e.g., 16 points are used.
The morphing process is then calculated in advance: an interim point between the two objects is calculated, and then interim points between those objects are calculated. These interim points can be any points between the two objects. This provides a plurality of images, each image differing from the previous image by only a small amount, and each image incrementally closer to the destination image. By providing a number of images, over an amount of time, the difference between each two adjacent images is very small. The viewer sees the illusion of transformation from one image to another, and thus the user sees a continuously-varying image that changes gradually from the original image to the destination image. It appears as though the apple changes into the orange.
The inventors of the present invention were the first to realize that such morphing technology could be used to simulate an image of a human body part moving between a first position and a second position for computer animation purposes.
One specific aspect of the present invention is the use of morphing technology to change facial image characteristics in a way to simulate the characteristics of speech. To do this, the inventors of the present invention developed a plurality of tools which changed human facial expression in accordance with speech to be spoken. The detailed aspects will be described herein.
It is therefore an object of the present invention to provide a system and method which changes facial expressions of a user's body part, preferably a user's face, in a way that associates those facial expressions with speech.
REFERENCES:
patent: 4260229 (1981-04-01), Bloomstein
patent: 4600281 (1986-07-01), Bloomstein
patent: 4827532 (1989-05-01), Bloomstein
patent: 4884972 (1989-12-01), Gasper
patent: 5111409 (1992-05-01), Gasper et al.
patent: 5278943 (1994-01-01), Gasper et al.
Essa et al, “Tracking Facial Motion”, Motion of Non-Rigid and Articulated Objects, May 1994, pp. 36-42.*
Brooke et al, “Computer Graphics Animation of Talking Faces Based on Stochastic Models”, 1994 INternations Symposium on Speech, Image Processing and Neural Networks, Apr. 13-16, 1994, Hong-Kong, pp. 73-76.*
Bothe et al, “Visual Speech and Coarticulation Effects”, ICASSP '93: Acoustic Speech & Signal Prcessing, Apr. 1993, pp. v634-v637.
Kagels David S.
Scott Kenneth C.
Watson Stephen Hilary
Yeates Matthew C.
California Institute of Technology
Fish & Richardson P.C.
Hong Stephen S.
LandOfFree
Method and apparatus for synthesizing realistic animations... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for synthesizing realistic animations..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for synthesizing realistic animations... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2508112