Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
2001-11-06
2004-02-24
Ahmed, Samir (Department: 2623)
Image analysis
Applications
3-d or stereo imaging analysis
C345S419000
Reexamination Certificate
active
06697518
ABSTRACT:
TECHNICAL FIELD
This invention relates to image processing.
BACKGROUND
To construct a realistic two dimensional (2-D) image of an object under arbitrary viewing angles and lighting conditions, a three dimensional (3-D) model of the object is first constructed.
To construct the 3-D model, the surface geometry and reflectance functions of the object need to be determined. The surface geometry of the object can be determined by finding the surface normal vectors of the object. The reflectance functions of the object can be modeled in several ways. One approach is to assume the surface to be Lambertian, and to find the albedo of the surface based on this assumption. Albedo is the fraction of incident light that is reflected by the surface of the object. A Lambertian surface is defined such that when a light ray shines on a point on the surface, the reflected luminous intensity in any direction from that point is proportional to the cosine of the angle between the direction of the light ray and the surface normal vector at that point. To obtain the 2-D image under arbitrary lighting conditions, the brightness of each point on the 3-D surface is first obtained by multiplying vectors representing the incident light rays with the surface normal vectors and the albedo. The 3-D surface with the correct brightness is then projected onto a 2-D plane perpendicular to the viewing direction to show a realistic 2-D image of the object.
To determine the surface normal vectors and albedo of an object, information of the object can be gathered from images taken from one or more cameras. Because a 2-D image of an object lacks depth information, multiple images are required to resolve the ambiguity as to the depth of each portion on the surface. The images may be taken by a camera from different viewing angles, or by a camera situated at a single location but with different lighting conditions.
The images may also be taken by a camera at a single location with the same lighting condition but with the object turned at different angles.
SUMMARY
In general, in one aspect, the invention is directed towards a method for constructing a three dimensional model of an object by illuminating different regions of a display in sequence, obtaining a set of images of the object being illuminated by the different regions, and constructing the three dimensional model of the object based on the set of images.
In general, in another aspect, the invention features a system for constructing a three dimensional model of an object. The system includes a display for showing an illuminated region at different positions on the display in sequence, a camera for obtaining a set of images of an object being illuminated by the illuminated region shown at different positions, and a processor for constructing a three dimensional model of the object based on the set of images.
In general, in another aspect, the invention features a computer program stored on a computer-readable media for causing a computer system to perform the functions of illuminating different regions of a display in sequence, obtaining a set of images of an object being illuminated by the different regions of the display, and constructing a three-dimensional model of the object from the set of images.
In general, in another aspect, the invention features a system for playing a video game. The system includes a display for showing illuminated regions in sequence at different positions on the display, a camera for obtaining a set of images of a user being illuminated by the illuminated regions at different positions on the display, and a processor for constructing a three dimensional model of the user based on the set of images and composing representations of the user within the video game based on the three dimensional model.
In general, in another aspect, the invention features a method of playing a video game by illuminating different regions of a display in sequence, obtaining a set of images of a user being illuminated by the different regions of the display, constructing a three dimensional model of the user based on the set of images; and composing representations of the user within the video game based on the three dimensional model.
Implementations of the invention may include one or more of the following features. The different regions include at least three different regions on the display. The step of constructing the three dimensional model further includes finding an estimate of a set of light source vectors representing the different illuminated regions, and finding an estimate of the shape and albedo of the object based on the estimated set of light source vectors. An ambiguity in the estimate of the shape is resolved by transforming an estimated shape to a transformed shape that is consistent with a pre-calibrated exemplar constructed from a set of objects similar to the object being modeled. The ambiguity in the estimate of the shape is further resolved by enforcing integrability on the data representing the shape of the object. The set of images is obtained by a camera that is facing a direction relatively the same as the display, the camera is at a relatively fixed position in relation to the object, and the object is relatively still. The object can be a human face.
The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.
REFERENCES:
patent: 5028802 (1991-07-01), Webb et al.
patent: 5479279 (1995-12-01), Barbier et al.
Georghiades, et al., “Illumination-based synthesis: Creating Novel Images of Human Faces Under Differing Pose and Lighting”,IEEE Workshop, pp. 47-54, (Jun. 26, 1999).
Yuille, A. L. et al., “Determining Generative Models of Objects under Varying Illumination: Shape and Albedo from Multiple Images using SVD and Integrability,” Jul. 5, 1999.
Zongker, Douglas E. et al., “Environment Matting and Compositing,” 1999.
Chuang, Yung-Yu et al., “Environment Matting Extensions: Towards Higher Accuracy and Real-Time Capture.”
Epstein, R., “Learning Object Representations from Lighting Variations.”
Georghiades, A.S. et al., “Illumination-Based Image Synthesis: Creating Novel Images of Human Faces Under Differing Pose and Lighting,” 1999, IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes.
Georghiades, A.S. et al., “Illumination Cones for Recognition Under Variable Lighting: Faces,” 1998, IEEE Conf. On Computer Vision and Pattern Recognition.
Georghiades, Athinodoros S. et al., “From Few to Many: Generative Models for Recognition Under Variable Pose and Illumination.”
Kriegman, David J. et al., “Representations for Recognition Under Variable Illumination.”
Kriegman, David et al., “Shape and Enlightenment: Reconstruction and Recognition under Variable Illumination,” 1999, ISRR.
Belhumeur et al., “The Bas-Relief Ambiguity”, International Journal of Computer Vision, (Earlier version appeared in CVPR, 1997, pp. 1060-1066).
Belhumeur Peter
Georghiades Athinodoros
Kriegman David
Ahmed Samir
Bali Vikkram
Fish & Richardson P.C.
Yale University
LandOfFree
Illumination based image synthesis does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Illumination based image synthesis, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Illumination based image synthesis will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3341906