Image analysis – Applications – Personnel identification
Reexamination Certificate
2001-01-04
2004-10-19
Boudreau, Leo (Department: 2621)
Image analysis
Applications
Personnel identification
C382S154000
Reexamination Certificate
active
06807290
ABSTRACT:
TECHNICAL FIELD
The disclosure below relates to generating realistic three-dimensional human face models and facial animations from still images of faces.
BACKGROUND
One of the most interesting and difficult problems in computer graphics is the effortless generation of realistic looking, animated human face models. Animated face models are essential to computer games, film making, online chat, virtual presence, video conferencing, etc. So far, the most popular commercially available tools have utilized laser scanners. Not only are these scanners expensive, the data are usually quite noisy, requiring hand touchup and manual registration prior to animating the model. Because inexpensive computers and cameras are widely available, there is a great interest in producing face models directly from images. In spite of progress toward this goal, the available techniques are either manually intensive or computationally expensive.
Facial modeling and animation has been a computer graphics research topic for over 25 years [6, 16, 17, 18, 19, 20, 21, 22, 23, 27, 30, 31, 33]. The reader is referred to Parke and Waters' book [23] for a complete overview.
Lee et al. [17, 18] developed techniques to clean up and register data generated from laser scanners. The obtained model is then animated using a physically based approach.
DeCarlo et al. [5] proposed a method to generate face models based on face measurements randomly generated according to anthropometric statistics. They showed that they were able to generate a variety of face geometries using these face measurements as constraints.
A number of researchers have proposed to create face models from two views [1, 13, 4]. They all require two cameras which must be carefully set up so that their directions are orthogonal. Zheng [37] developed a system to construct geometrical object models from image contours, but it requires a turn-table setup.
Pighin et al. [26] developed a system to allow a user to manually specify correspondences across multiple images, and use vision techniques to computer 3D reconstructions. A 3D mesh model is then fit to the reconstructed 3D points. They were able to generate highly realistic face models, but with a manually intensive procedure.
Blanz and Vetter [3] demonstrated that linear classes of face geometries and images are very powerful in generating convincing 3D human face models from images. Blanz and Vetter used a large image database to cover every skin type.
Kang et al. [14] also use linear spaces of geometrical models to construct 3D face models from multiple images. But their approach requires manually aligning the generic mesh to one of the images, which is in general a tedious task for an average user.
Fua et al. [8] deform a generic face model to fit dense stereo data, but their face model contains a lot more parameters to estimate because basically all of the vertexes are independent parameters, plus reliable dense stereo data are in general difficult to obtain with a single camera. Their method usually takes 30 minutes to an hour, while ours takes 2-3 minutes.
Guenter et al. [9] developed a facial animation capturing system to capture both the 3D geometry and texture image of each frame and reproduce high quality facial animations. The problem they solved is different from what is addressed here in that they assumed the person's 3D model was available and the goal was to track the subsequent facial deformations.
SUMMARY
The system described below allows an untrained user with a PC and an ordinary camera to create and instantly animate his/her face model in no more than a few minutes. The user interface for the process comprises three simple steps. First, the user is instructed to pose for two still images. The user is then instructed to turn his/her head horizontally, first in one direction and then the other. Third, the user is instructed to identify a few key points in the images. Then the system computes the 3D face geometry from the two images, and tracks the video sequences, with reference to the computed 3D face geometry, to create a complete facial texture map by blending frames of the sequence.
To overcome the difficulty of extracting 3D facial geometry from two images, the system matches a sparse set of corners and uses them to compute head motion and the 3D locations of these corner points. The system then fits a linear class of human face geometries to this sparse set of reconstructed corners to generate the complete face geometry. Linear classes of face geometry and image prototypes have previously been demonstrated for constructing 3D face models from images in a morphable model framework. Below, we show that linear classes of face geometries can be used to effectively fit/interpolate a sparse set of 3D reconstructed points. This novel technique allows the system to quickly generate photorealistic 3D face models with minimal user intervention.
REFERENCES:
patent: 5469512 (1995-11-01), Fujita et al.
patent: 5774591 (1998-06-01), Black et al.
patent: 5990901 (1999-11-01), Lawton et al.
patent: 6072496 (2000-06-01), Guenter et al.
patent: 6301370 (2001-10-01), Steffens et al.
patent: 6496594 (2002-12-01), Prokoski
patent: 6664956 (2003-12-01), Erdem
Soon-Cang Pei et al., Global Motion Estimation in Model-Based Image Coding by Tracking Three-Dimensional Contour Featu Points, IEEE 1051-8215/98, 181-190.*
Pighin et al., Resynthesizing facial animation through 3D model-tracking, IEEE Inspec Accession No. 6371194, 1-8.*
Shakunaga et al., Integration of eigentemplate and structure matching for automatic facial feature detection, IEEE Inspec Accession No. 5920393, 1-6.*
Perlin, Reatme responsive animation with personality, IEEE 1071-2626/95, 5-15.*
Skaria et al., Modeling subdivision control meshes for creating coorton faces, IEEE 0-7695-0853-7/01, 216-225.*
Parke, Computer graphics models for the human face, IEEE CH515-6/79/0000-0724, 724-727.*
Terzopoulos et al., Analysis of facial images using physical and anatomical models, IEEE CH2934-8/90/0000/072, 727-732.*
DiPaola, FaceSpace: a facial spatial domain toolkit, IEEE 1093-9547, 1-5.*
DeCarlo et al. An anthropometric face model using variational techniques, University of Pennsylvania, 1-8.*
Blanz et al., Face recohgnition based on fitting a 3d morphable mode, IEEE 0162-8828/03, 1063-1074.*
Zheng et al., 3D models from contours, IEEE 0-8186, 349-353.*
Vetter et al., Linear object classes and image synthesis, IEEE 0162-8828/97, 733-741.*
Dariush et al., Spatiotemporal analysis of face profiles, IEEE Inspec Accession No. 5920418, 248-253.*
Akimoto et al., Automatic creation of 3D facial models, IEEE 0272-16/93, 16-22.*
Faugeras “Three Dimensional Computer Vision”; 1999; pp. 1-663.
Zhang; “A New Multistage Approach to Motion and Structure Estimation: From Essential Parameters to Euclidean Motion Via Fundamental Matrix”; Jun. 1996; pp. 1-38.
More; “The Levenberg-Marquardt Algorithm: Implentation and Theory”; 1978; pp. 105-116.
Jebara et al; “Parametrized Structure from Motion for 3D Adaptive Feedback Tracking of Faces”; 1997, pp. 144-150.
Fua; “Using Model-Driven Bundle-Adjustment to Model Heads from Raw Video Sequence”; pp. 1-8.
Zhang; “On sthe Optimization Criteria Used in Two-View Motion Analysis”; Jul. 1998; pp. 717-729.
Pighin et al; “Synthesizing Realistic Facial Expressions from Photographs”; pp. 1-20.
Fua et al; “From Regular Images to Animated Heads: A Least Squares Approach”; pp. 1-15.
Kang et al; “Appearance-based Structure from Motion Using Linear Classes of 3-D Models”; pp. i-ii, 1-32.
Cohen Michael F.
Jacobs Charles E.
Liu Zicheng
Zhang Zhengyou
Boudreau Leo
Lee & Hayes PLLC
Sherali Ishrat
LandOfFree
Rapid computer modeling of faces for animation does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Rapid computer modeling of faces for animation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Rapid computer modeling of faces for animation will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3264859