Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
2000-10-12
2003-12-16
Vo, Cliff N. (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C345S473000
Reexamination Certificate
active
06664956
ABSTRACT:
FIELD OF THE INVENTION
The present invention is related to the field of computer generated modeling, and more specifically, to a technique for generating a personalized three-dimensional (3-D) face model from a sequence of two-dimensional (2-D) images of a person's face.
BACKGROUND OF THE INVENTION
Generation of a 3-D face model of a person involves mapping a real image of the face of the person onto a 3-D triangular mesh that has been shaped to have the same or similar geometry as the face of the person. 3-D triangular mesh refers to a connected set of triangular patches in 3-D whose corners form the nodes of the mesh. Each triangular patch in the 3-D mesh acquires its image data from an associated triangular region in the image of the face.
The methods disclosed in the prior art for generating a 3-D face model can be generally classified as those that involve (i) a fully manual process, (ii) a semi-automatic process and (iii) a fully-automatic process. In a fully manual process, every triangular patch of the 3-D mesh has to be manually mapped onto the image of the face according to the facial features of the face. Fully manual processes are labor intensive and time consuming because the number of triangular patches in the 3-D mesh could range from a hundred to several thousands.
There are known techniques for using markers to track selected facial features such as the eyebrows, ears, mouth and corners of the eyes.
Semi-automatic processes rely on automatically detecting or manually marking certain features on the face, such as eyes, nose and mouth, and initialize the 3-D mesh by an affine warping of a standard 3-D mesh based on the location of the detected facial features. However, a global affine transformation generally does not match the many local facial dimensions. Thus, the locations of the nodes are fine-tuned in a manual process for each person.
Fully automatic processes drastically reduce the time required to map an image onto a 3-D mesh. However, while hardware based fully automatic processes are very costly, software based fully automatic processes are very sensitive to the image data and thus may not consistently produce accurate 3-D face models.
In addition to a 3-D mesh that matches the geometry of the face, a composite image that contains facial image data from various viewing directions also needs to be constructed. In the prior art, the composite image is a mosaic (or sprite) image of the face that is produced either by a 3-D rotating camera or by stitching a number of 2-D images of the face. While the former process is very costly, the latter one is generally very inaccurate.
Hence, there is a need for a fast, inexpensive, and accurate method for generating the 3-D mesh and texture image for a face model. The present invention proposes a semi-automatic method for generating the 3-D mesh with minimal manual fine tuning. The present invention also proposes a simpler and more general technique for generating the texture image, that involves only concatenating and color blending the 2-D images of the face.
SUMMARY OF THE INVENTION
The present invention provides an improvement designed to satisfy the aferomentioned needs. Particularly, the present invention is directed to a computer program product for creating a 3-D face model from a plurality of 2-D images of a person's face, by performing the steps of: (a) receiving the plurality of images of a person; (b) obtaining the geometry mesh by deforming a predefined standard 3-D triangular mesh based on the dimensions and relative positions of the person's facial features; and (c) obtaining the texture image by compositing a plurality of 2-D images of the person taken from particular directions and modifying them in boundary regions to achieve seamless stitching of color for the 3-D face model.
REFERENCES:
patent: 4975960 (1990-12-01), Petajan
patent: 5280530 (1994-01-01), Trew et al.
patent: 5744953 (1998-04-01), Hansen
patent: 5774591 (1998-06-01), Black et al.
patent: 5802220 (1998-09-01), Black et al.
patent: 5805745 (1998-09-01), Graf
patent: 5807284 (1998-09-01), Foxlin
patent: 5907626 (1999-05-01), Toklu et al.
patent: 5923337 (1999-07-01), Yamamoto
patent: 5982909 (1999-11-01), Erdem et al.
patent: 6009210 (1999-12-01), Kang
patent: 6016148 (2000-01-01), Kang et al.
patent: 6020892 (2000-02-01), Dillon
patent: 6028960 (2000-02-01), Graf et al.
patent: 6031539 (2000-02-01), Kang et al.
patent: 6037949 (2000-03-01), DeRose et al.
patent: 6047078 (2000-04-01), Kang
patent: 6052132 (2000-04-01), Christian et al.
patent: 6064390 (2000-05-01), Sagar et al.
patent: 0 926 628 (1999-06-01), None
patent: PCT/IB01/02735 (2002-06-01), None
patent: PCT/IB01/02736 (2002-07-01), None
patent: PCT/IB01/02363 (2002-08-01), None
patent: 8293026 (1996-11-01), None
patent: WO 98 01830 (1998-01-01), None
patent: WO 9906962 (1999-02-01), None
Lavagetto, Fabio, et al., “The Facial Animation Engine: Toward a High-Level Interface for the Design of MPEG-4 Comliant Animated Faces”,IEEE Transactions on circuits and Systems for Video Technology, vol. 9, No. 2, pp. 277-289, (Mar. 1999).
Antunes, Gabriel, et al., “MPEG-4 Facial Animation Technology: Survey, Implementation, and Results”,IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, No. 2, pp. 290-305, (Mar. 1999).
Tomasi, Carlo, et al., “Shape and Motion From Image Streams Under Orthography: A Factorization Method”,International Journal of Computer Vision, vol. 9, No. 2, pp. 137-154, (1992).
Szeliski, Richard, et al., “Recovering 3D Shape and Motion From Image Streams Using Nonlinear Least Squares”,Journal of Visual Communication and Image Representation, vol. 5, No. 1., pp. 10-28, (Mar. 1994).
Lavagetto, Fabio, et al., “MPEG-4 Compliant Calibration of 3D Head Models”,Picture Coding Symposium, pp. 217-220, (1999).
Poelman, Conrad J., et al., “A Paraperspective Factorization Method for Shape and Motion Recovery”,Carnegio Mellon University Technical Report, CMU-CS-93-219, pp. 1-36, (Dec. 11, 1993).
Dorai, Chitra, et al., “Registration and Integration of Multiple Object Views for 3D Model Construction”,IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, No. 1, pp. 83-89, (Jan. 1998).
Akimoto, Takaaki, et al., “Automatic Creation of 3D Facial Models”,IEEE Computer Graphics&Applications, pp. 16-22, (Sep. 1993).
Seferidis, Vassills, et al., “General Approach to Block-Matching Motion Estimation”,Optical Engineering, vol. 32, No. 7, pp. 1464-1474.
Escher, Marc, et al., “Automatic 3D Cloning and Real-Time Animation of a Human Face”,Proceedings of Computer Animation, Geneva, Switzerland, pp. 1-13, (1997).
Li-An Tang et al: “Analysis-based facial expression synthesis” Proceedings of the International Conference of Image Processing (ICIP) Austin, Nov. 13-16, 1994 Los Alamitos, IEEE Comp. Soc. Press, US, vol. 3 Conf. 1, Nov. 13, 1994 , pp. 98-102 XP010146456 ISBN: 0-8186-6952-7 the whole document.
Cosi P et al: “Phonetic recognition by recurrent neural networds working on audio and visual information” Speech Communication, Elsevier Science Publishers, Amsterdam, NL, vol. 19, No. 3, Sep. 1, 1996, pp. 245-252, XP004013654 ; ISSN: 0167-6393 p. 246, right hand column, line 13-page 248, right-hand column, line 7.
Guenther, Brian; Grimm, Cindy; Wood, Daniel; Malvar, Henrique; Pighin Fredrick: “Making faces” Computer Graphics, Proceedings. Siggraph 98 Conference Proceedings, Proceedings of Siggraph 98: 25th International Conference on Computer Graphics and Interactive Techniques, Orlando, FL.
USA, Jul. 19-24, 1998, pp. 55-66, XP002205956 1998, New York, NY, USA, ACM, USA ISBN: 0-89791-999-8 p. 56, paragraph 2 -p. 59, paragraph 3.3.3.
Ebihara K et al: Real-Time 3-D Facial Image Reconstruction for Virtual Space Teleconferencing Electronics & Communications in Japan, Part III-Fundamental Electronic Science, Scripta Technica, NY, US, vol. 82, No. 5, May 1999, pp. 80-90, XP000875659 ISSN:1042-0967 p. 80, left-hand column, line 1-p. 81, rt-hand col, ln 5.
Chen T: “Technologies for building networked collaborative environments” Image Processing, 1999. I
Fitzgerald, Esq. Thomas R.
Momentum Bilgisayar, Yazilim, Danismanlik, Ticaret A. S.
Vo Cliff N.
LandOfFree
Method for generating a personalized 3-D face model does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for generating a personalized 3-D face model, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for generating a personalized 3-D face model will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3116882