Image analysis – Applications – Target tracking or detecting
Reexamination Certificate
2000-02-03
2004-01-06
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Applications
Target tracking or detecting
C382S154000, C382S107000, C382S118000, C382S173000, C348S169000
Reexamination Certificate
active
06674877
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates in general to object detection and tracking, and in particular, to a system and method for visually tracking three-dimensional (3-D) articulated self occluded objects or multiple occluded objects, in real-time using dense disparity maps.
2. Related Art
Efficient and accurate tracking of non-rigid motion of three-dimensional (3-D) objects in image sequences is very desirable in the computer vision field. Non-rigid tracking is generally divided into studies of two categories of motion: deformable object motion and the motion of an articulated object. The latter is of great interest to the HCl (human computer interaction) community because the human body is an articulated object. Current commercially available motion capture systems are based on magnetic or optical trackers. These trackers usually require a human subject (the articulated object to be tracked) to wear a special suit with markers on it or require the human subject to be directly attached to the system by cumbersome cables. Therefore, what is needed is a passive sensing object tracking system that is convenient, less constraining, and suitable for a variety of object tracking uses.
Although there have been several vision-based approaches to human body tracking, ranging from detailed model-based approaches to simplified, but fast statistical algorithms and cardboard models, there is a need for an accurate and efficient system for real-time tracking of articulate objects. For example, in one previous system, “Pfinder: real-time tracking of the human body” by C. Wren, A. Azarbayejani, T. Darrell, A. Pentland, IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.19, no.7, pp.780-5, July 1997, two-dimensional (2-D) tracking was achieved with Gaussian modeling of a set of pixels in the image sharing some kind of similarity. However, this system was limited due to the 2-D based Gaussian modeling. As such, what is also needed is an object tracking system that uses 3D Gaussian modeling. It should be noted that in this prior system and in most object tracking systems each set of pixel is commonly referred to as a “blob”.
In “Real-time self-calibrating stereo person tracking using 3-D shape estimation from blob features” by A. Azarbayejani, A. Pentland, Proceedings of the 13th International Conference on Pattern Recognition, vol. 3, pp.627-32, 1996, in order to track human motion in full 3-D, the above approach was extended by using input from two cameras for upper body tracking. However, in this system, only the hands and head were tracked while the position and orientation of the torso and lower and upper arms were ambiguous, and the two cameras used in this system were not used to calculate a dense disparity map, but rather to estimate 2-D blob parameters in each image. Thus, what is additionally needed is an articulated object tracking system that calculates disparity maps and 3-D object maps.
In “Dynamic models of human motion” by C. Wren, A. Pentland, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition, pp.22-7, 1998, an Extended Kalman Filter was used to impose articulation constraints on portions of the body to provide a guess about the full posture. However, since only three points are measured on a human body, there was not enough information for unambiguous posture tracking. Typically, knowledge of the dynamics of human motion is helpful for tracking, as discussed in the “Dynamic models of human motion” reference. Therefore what is further needed is an object tracking system that can provide enough information for unambiguous posture tracking.
In “Model-based tracking of self-occluding articulated objects” by J. Rehg and T. Kanade, Proceedings 1995 International Conference on Computer Vision, pp. 35-46, 1995, a model-based approach to tracking self-occluding articulated structures was proposed. However, the algorithm was based on template matching and was sensitive to lighting changes. Hence, what is also needed is a tracking system that uses stereo cues so that the disparity computed based on correlation is less sensitive to the intensity changes.
Whatever the merits of the above mentioned systems and methods, they do not achieve the benefits of the present invention.
SUMMARY OF THE INVENTION
To overcome the limitations in the prior art described above, and to overcome other limitations that will become apparent upon reading and understanding the present specification, the present invention is embodied in a system and method for digitally tracking objects in real time. The present invention visually tracks three-dimensional (3-D) objects in real-time using dense disparity maps.
The present invention digitally tracks articulated objects, such as a human body, by digitally segmenting and modeling different body parts using statistical models defined by multiple size parameters, position and orientation. In addition, the present invention is embodied in a system and method for recognizing mutual occlusions of body parts and filling in data for the occluded parts while tracking a human body. The body parts are preferably tracked from frame to frame in image sequences as an articulated structure in which the body parts are connected at the joints instead of as individual objects moving and changing shape and orientation freely.
Specifically, the present invention uses input from multiple cameras suitably spaced apart. A disparity map is computed at a predefined frame rate with well known and readily available techniques, such as the technique described in “Background modeling for segmentation of video-rate stereo sequences” by C. Eveland, K. Konolige, R. Bolles, Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.266-71, 1998, which is herein incorporated by reference. The present invention uses this information based on a generative statistical model of image formation specifically tuned to fast tracking in the presence of self-occlusions among the articulated 3-D Gaussian models.
This model is a graphical model, such as a Bayesian network, that can be used for diverse applications at formalizing generative processes in ways that allow probabilistic inference. Preferably, a maximum likelihood estimate of the posture of an articulated structure is achieved by a simplified, but very fast inference technique that consists of two stages. In the first stage, the disparity map is segmented into different parts of the articulated structure based on the estimated state of the Gaussian mixture using the maximum likelihood principle with an additional mechanism for filling in the missing data due to occlusions. The statistical properties of the individual parts are then re-estimated. In the second stage of the inference technique of the present invention, an extended Kalman Filter (EKF) enforces the articulation constraints and can also improve the tracking performance by modeling the dynamics of the tracked object. Also, it should be noted that the present invention can be simplified and used to track independent self occluding objects without articulation constraints.
REFERENCES:
patent: 5179441 (1993-01-01), Anderson et al.
patent: 5359362 (1994-10-01), Lewis et al.
patent: 5710875 (1998-01-01), Harashima et al.
patent: 5727080 (1998-03-01), Cox et al.
patent: 5917936 (1999-06-01), Katto
patent: 5917937 (1999-06-01), Szeliski et al.
patent: 6047078 (2000-04-01), Kang
patent: 6115052 (2000-09-01), Freeman et al.
patent: 6301370 (2001-10-01), Steffens et al.
patent: 6314211 (2001-11-01), Kim et al.
patent: 6326972 (2001-12-01), Buhler et al.
patent: 6381302 (2002-04-01), Berestov
patent: 6434260 (2002-08-01), Soferman et al.
patent: 6499025 (2002-12-01), Horvitz et al.
patent: 6526161 (2003-02-01), Yan
patent: 2002/0110273 (2002-08-01), Dufour
patent: 2003/0038875 (2003-02-01), Ivanov et al.
Azarbayejani a., Wren C., and Pentland A. Real-Time 3-D Tracking of the Human Body. MIT Media Laboratory Perceptual Computing Section Technical Report No. 374.
Jojic Nebojsa
Turk Matthew A.
Choobin Barry
Fischer Craig S.
Lyon & Harr L.L.P.
Mehta Bhavesh M.
LandOfFree
System and method for visually tracking occluded objects in... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for visually tracking occluded objects in..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for visually tracking occluded objects in... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3267601