Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
2001-08-01
2002-12-10
Zimmerman, Mark (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C345S474000
Reexamination Certificate
active
06492986
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to human face shape and motion estimation. More particularly, the present invention relates to such estimation based on integrating optical flow and deformable models.
BACKGROUND OF THE INVENTION
A wide variety of face models have been used in the extraction and recognition of facial expressions in image sequences. Several 2-D face models based on splines or deformable templates have been developed which track the contours of a face in an image sequence. Terzopoulos and Waters (“Analysis and synthesis of facial image sequences using physical and anatomical models,”
IEEE Pattern Analysis and Machine Intelligence,
15(6):569-579, 1993) and Essa and Pentland (“Facial expression recognition using a dynamic model and motion energy,” in
Proceedings ICCV '
95, pages 360-367, 1995) use a physics-based 3-D mesh with many degrees of freedom, where face motion is measured in terms of muscle activations. Edge forces from snakes are used in the former, while in the latter, the face model is used to ‘clean up’ an optical flow field that is used in expression recognition.
Another approach is to directly use the optical flow field from face images. Yacoob and Davis (“Computing spatio-temporal representations of human faces,”
Proceedings CVPR '
94, pages 70-75, 1994) use statistical properties of the flow for expression recognition. Black and Yacoob (“Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion,”
Proceedings ICCV '
95, pages 374-381, 1995) parameterize the flow field based on the structure of the face under projection. Addressing the problem of image coding, Li, et al. (“3-D motion estimation in model-based facial image coding,”
PAMI,
15(6):545-555, Jun. 1993) estimate face motion using a simple 3-D model by a combination of prediction and a model-based least-squares solution to the optical flow constraint equation. A render-feedback loop is used to combat error accumulation in tracking.
However, none of these approaches permits large head rotations due to the use of a 2-D model or the inability to handle self-occlusion. Also, none of the previous work makes a serious attempt in extracting the 3-D shape of the face from an image sequence. At best, the boundary of face parts are located to align the model with an image. Finally, none of the previous face tracking work integrates multiple cues in the tracking of the face.
Accordingly, a system is desired which uses a 3-D model and allows the tracking of large rotations by using self-occlusion information from the model. A system is also desired which extracts the shape of the face using a combination of edge forces and anthropometry information. Moreover, a system is desired which can easily augment the optical flow solution with additional information to improve such solution and which permits the use of a small number of image points to sample the optical flow field, as well as the computation of edge forces to prevent error accumulation in the motion. The present invention has been developed to meet these needs in the art.
SUMMARY OF THE INVENTION
The present invention satisfies the aforementioned needs by providing a method and apparatus for human face shape and motion estimation based on integrating optical flow and deformable models. The optical flow, constraint equation provides a non-holonomic constraint on the motion of the deformable model. Forces computed from edges and optical flow are used simultaneously. When this dynamic system is solved, a model-based least-squares solution for the optical flow is obtained and improved estimation results are achieved. The use of a 3-D model reduces or eliminates problems associated with optical flow computation. This approach instantiates a general methodology for treating visual cues as constraints on deformable models. The model, which applied to human face shape and motion estimation, uses a small number of parameters to describe a rich variety of face shapes and facial expressions.
REFERENCES:
patent: 5754181 (1998-05-01), Amdursky et al.
Terzopoulos, et al., “Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models”,IEEE Transactions on Pattern Analysis And Machine Intelligence, 1993, 15(6), 569-579.
Esa, et al., “Facial Expression Recognition Using A Dynamic Model and Motion Energy”,Proceedings —Fifth International Conference on Computer Vision, 1995, 360-367.
Yacoob, et al., “Computing Spatio -Temporal Representations of Human Faces”,IEEE Computer Society Society Conference on Computer Vision and Pattern Recognition, 1994, 70-75.
Black, et al., “Tracking and Recognizing Rigid and Non-Rigid Facial Motions Using Local Parametric Models of Image Motion”,Proceedings —Fifth International Conference on Computer Vision, 1995, 374-381.
Li, et al., “3-D Motion Estimation in Model-Based Facial Image Coding”,IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(6), 545-555.
Metaxas, et al., Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis,IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(6), 580-591.
Simoncelli, et al., “Probability Distributions of Optical Flow”,IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1991, 310-315.
Yuille, et al., “Feature Extraction From Faces Using Deformable Templates”,International Journal of Computer Vision, 1992, 8(2), 99-111.
DeCarlo et al., “Blended Deformable Models”,Proceedings —IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994, 566-572.
60/048,298 —U.S. Provisional Pat. application, filed Jun. 2, 1997. (DeCarlo, et al., “The Integration of Optical Flow and Deformable Models with Applications to Human Face Shape and Motion Estimation”,Proceedings —IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1996, 231-238).
Agrawala et al., “Model-based motion estimation of synthetic animations,”ACM Multimedia 95 —Electronic Proceedings, Nov. 1995.
Beauchemin et al., “The computation of optical flow,”ACM computing Surveys, Sep. 1995, 27(3).
Giachetti et al., “Optical flow and deformable Objects,” 0-8186-7042-8/95, Nov. 1995.
Giachetti et al., “Dynamic seqmentation of traffic scences,”Proceedings of the Intelligent Vehicles '95 Symposium, ISBN 0-7803-2983-X, Sep. 1995.
Pan, et al., “Adaptive estimation of optical flow from general object motion,” ACM-0-89791-502-x/92/0002/0489, 1992.
Reinders, et al., “Facial feature localization and adaptation of a generic face model for model-based coding,”Signal Proceeding: Image Communication, 1995, 7, 57-74.
DeCarlo Douglas
Metaxas Dimitris
Padmanabhan Mano
The Trustees of the University of Pennsylvania
Woodcock & Washburn LLP
Zimmerman Mark
LandOfFree
Method for human face shape and motion estimation based on... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for human face shape and motion estimation based on..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for human face shape and motion estimation based on... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2987269