Image analysis – Applications – Target tracking or detecting
Reexamination Certificate
1999-12-06
2003-12-02
Werner, Brian (Department: 2621)
Image analysis
Applications
Target tracking or detecting
C382S154000
Reexamination Certificate
active
06658136
ABSTRACT:
BACKGROUND
1. Technical Field
The invention is related to system and process for locating and tracking people and non-stationary objects of interest in a scene, and more particularly, to such a system and process that employs a series of range images of the scene taken over time.
2. Background Art
Most current systems for determining the presence of persons or objects of interest in an image of a scene have involved the use of a sequence of pixel intensity-based images or intensity images for short. For example, a temporal sequence of color images of a scene is often employed for this purpose [1].
Persons or objects are typically recognized and tracked in these systems based on motion detected by one of three methods—namely by background subtraction [2], by adaptive template correlation, or by tracking color contour models [3, 4].
While the aforementioned locating methods are useful, they do have limitations. For example, the use of intensity images results in the presence of background “clutter” that significantly affects the reliability and robustness of these techniques. In addition, the adaptive templates employed in the adaptive template correlation techniques tend to drift as they pick up strong edges or other features from the background, and color contour tracking techniques are susceptible to degradation by intensity gradients in the background near the contour. Further, the image differencing methods typically used in the foregoing techniques are sensitive to shadows, change in lighting conditions or camera gain, and micro-motions between images. As a result, discrimination of foreground from background is difficult.
More recently, the use of sequential range images of the scene has been introduced into systems for locating persons and objects, and for tracking their movements on a real time basis [5, 6, 7]. In general, the advantage of using range images over intensity images is that the range information can be used to discriminate the three-dimensional shape of objects, which can be useful in both locating and tracking. For example, occluding surfaces can be found and dealt with as the tracked object moves behind them. Recognizing objects is also easier, since the actual size of the object, rather than its image size, can be used for matching. Further, tracking using range information presents fewer problems for segmentation, since range information is relatively unaffected by lighting conditions or extraneous motion.
While the locating and tracking systems employing range information can provide superior performance in comparison to systems employing only intensity images, there is still considerable room for improvement. For example, the aforementioned systems use range information typically for background subtraction purposes, but rely mostly on intensity image information to locate individual people or objects in the scene being analyzed. This can result in poor discriminatory ability when two people or objects are close together in the scene.
The system and process according to the present invention resolves the deficiencies of current locating and tracking systems employing range information.
It is noted that in the preceding paragraphs, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. Multiple references are identified by a pair of brackets containing more than one designator, for example, [5, 6, 7]. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.
SUMMARY
The present invention involves a technique for locating and tracking people and non-stationary objects of interest in a scene using a series of range images of the scene taken over time. In regards to locating people and objects, the technique generally entail first generating the series of range images. Preferably, the series of range images is a continuous temporal sequence of depth maps of the scene, such as might be captured using a video-rate stereo imaging system or a laser range finder system. A background model is computed from a block of these range images. In general, this entails identifying pixel locations in the block of range images that have reliable depth values.
Once the background model has been computed, a range image generated subsequent to the aforementioned block of range images is selected for processing. Preferably, this entails selecting the very next range image generated following the last image of the block used to compute the background model. The background is subtracted from this currently selected range image based on the background model to produce a foreground image. Generally, this involves identifying those pixels representing non-static portions of the scene depicted in the selected range image based on the background model. These “non-static” pixels are collectively designated as the foreground image.
At this point, an optional procedure can be employed to connect regions associated with the same person or object that may have become separated by gaps in the preceding background subtraction. To accomplish this, a standard morphologically growing and shrinking technique can be implemented. Essentially, this involves using the technique to first grow the foreground image, and then shrink it, in such a way that pixels in the gaps between related regions are added to the foreground image when pixels in the vicinity of the gap exhibit similar depth values. This connects the regions. If, however, the pixels in the vicinity of the gap do not exhibit similar depth values, this is an indication they belong to a different person or object. In that case, the pixels in the gap are not added to the foreground image and the regions remain separated.
The foreground image is next segmented into regions, each of which represents a different person or object of interest in the scene captured by the currently selected range image. This is essentially accomplished by identifying regions in the foreground image made up of pixels exhibiting smoothly varying depth values. In addition, any region having an actual area too small to represent a person or object of interest is eliminated from further consideration as foreground pixels.
If it is not only desired to locate a person or object in the scene, but to determine their identity as well, the following optional procedure can be adopted. This optional procedure determines the identity of the person or object associated with each segmented region in the foreground image by capturing an intensity image of the scene simultaneously with the generation of the aforementioned currently selected range image. Each region of the intensity image that corresponds to a segmented region in the foreground image can then be identified and used to determine the identity of the person or object represented by that region. It is noted that while the optional identification process can be performed immediately after the foreground image segmentation procedure, it can be even more advantageous to wait until after an optional ground plane segmentation procedure that will be described shortly. In either case, the identification process generally entails first characterizing the identified region in a way similar to a series of previously stored intensity images of known persons and objects. For example, the identified region and stored images might be characterized via a color histogram technique. The characterized region is compared to each of the stored characterizations, and the degree of similarity between each of them is assessed. If the degree of similarity between the identified region and one of the stored characterizations exceeds a prescribed level, the person or object represented by the identified region is designated to be the person or object associated with that stored characterization.
Regardless of whether the segmented foreground
Fischer Craig S.
Lyon Richard T.
Lyon & Harr L.L.P.
Microsoft Corporation
Miller Ryan J.
LandOfFree
System and process for locating and tracking a person or... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and process for locating and tracking a person or..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and process for locating and tracking a person or... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3131228