Image analysis – Pattern recognition – Classification
Reexamination Certificate
2000-04-04
2004-08-03
Mariam, Daniel (Department: 2721)
Image analysis
Pattern recognition
Classification
C382S154000, C382S218000, C382S209000, C382S305000
Reexamination Certificate
active
06771818
ABSTRACT:
BACKGROUND
1. Technical Field
The invention is related to a system and process for locating people and objects of interest in a scene, and more particularly, to a system and process that locates and clusters three-dimensional regions within a depth image, and identifies the content and position of clustered regions by comparing the clusters to a model.
2. Related Art
Most current systems for determining the presence of persons or objects of interest in an image of a scene have involved the use of a sequence of pixel intensity-based images or intensity images for short. For example, a temporal sequence of color images of a scene is often employed for this purpose [1]. Persons or objects are typically recognized and tracked in these systems based on motion detected by one of three methods—namely by background subtraction [2], by adaptive template correlation, or by tracking color contour models [3, 4].
While the aforementioned locating methods are useful, they do have limitations. For example, the use of intensity images results in the presence of background “clutter” that significantly affects the reliability and robustness of these techniques. In addition, the adaptive templates employed in the adaptive template correlation techniques tend to drift as they pick up strong edges or other features from the background, and color contour tracking techniques are susceptible to degradation by intensity gradients in the background near the contour. Further, the image differencing methods typically used in the foregoing techniques are sensitive to shadows, change in lighting conditions or camera gain, and micro-motions between images. As a result, discrimination of foreground from background is difficult.
More recently, the use of sequential range images of the scene has been introduced into systems for locating persons and objects, and for tracking their movements on a real time basis [5, 6, 7]. In general, the advantage of using range images over intensity images is that the range information can be used to discriminate the three-dimensional shape of objects, which can be useful in both locating and tracking. For example, occluding surfaces can be found and dealt with as the tracked object moves behind them. Recognizing objects is also easier, since the actual size of the object, rather than its image size, can be used for matching. Further, tracking using range information presents fewer problems for segmentation, since range information is relatively unaffected by lighting conditions.
While the locating and tracking systems employing range information can provide superior performance in comparison to systems employing only intensity images, there is still considerable room for improvement. For example, the aforementioned systems use range information typically for background subtraction purposes, but rely mostly on intensity image information to locate individual people or objects in the scene being analyzed. Further, when using a background subtraction process, objects in the scene being analyzed tend to separate into a plurality of distinct three-dimensional regions. For these and other reasons, systems using such methods tend to exhibit poor discriminatory ability when two people or objects are close together in the scene. The system and process according to the present invention resolves the deficiencies of current locating and tracking systems employing range information.
It is noted that in the preceding paragraphs, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. Multiple references are identified by a pair of brackets containing more than one designator, for example, [5, 6, 7]. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.
SUMMARY
The present invention involves a new system and process for use in an object recognition scheme for comparing three-dimensional regions (referred to as “blobs”) in images to one or more models in order to identify the location of people or objects within a scene. This object recognition scheme allows for real-time location and tracking of people or objects of interest within the scene. The technique generally entails first generating an initial three-dimensional depth image, often referred to as a background or baseline depth image, of the scene or area of interest. The baseline depth image is generated using conventional methods such as a stereo camera mechanism. Conventional processing of the baseline depth image is used to identify the spatial coordinates of three-dimensional image pixels within the three-dimensional volume represented by the image. During identification and location operations, an image acquisition process, such as, for example, a stereo camera mechanism, is used to capture live depth images at any desired scan rate. The identification and location of people and or objects may then be determined by processing a working image obtained from a background subtraction process using the baseline depth image and a live depth image. In other words, the baseline depth image is subtracted from the live depth image. Any pixel in the live depth image that differs significantly from the background image becomes part of the working image that is then processed to identify and locate people or objects.
The aforementioned background subtraction process typically results in a depth image containing a number of distinct three-dimensional regions or “blobs.” Each resultant blob in the working image is formed of a plurality of image pixels having x, y, and z coordinates defining the spatial location of each pixel within the three-dimensional space representing the scene. The subtraction process typically results in a number of distinct blobs for several reasons. First, featureless or textureless regions within the area of interest do not typically provide good depth data when using stereo cameras. These regions are typically broken up or eliminated in the subtraction process. Consequently, a uniformly lit person wearing relatively smooth solid color clothing such as a jacket or shirt would tend to be represented in the working image as a number of separated blobs. Further, noise in either the baseline or live depth images may cause people or objects to partially blend into the background. As a result, people or objects again tend to break up into a number of separated blobs in the working image. In addition, image noise or distortion, or extraneous objects not of interest, may create spurious blobs that also become part of the working image.
Processing of the working image involves identifying which of the blobs belong to the same person or object of interest so as to accurately identify and locate that person or object within the area of interest. A “clustering” process is used to roughly identify each set of blobs in the working image that may belong to a particular person or object of interest. An analysis of the blob clusters produced by the clustering process is used to identify clusters of blobs that most accurately represents the people or objects of interest by determining the closest match or matches to a model representing the people or objects of interest. The model is a shape such as an ellipsoid having the approximate dimensions of the person or object of interest. In addition, blob clusters may be compared to any number of different models representing people or objects of different shapes and sizes.
One method for determining the closest match between a cluster of blobs and a model is to compare every possible cluster of blobs to the model. However, as the number of blobs increases, a corresponding exponential increase in the number of candidate blob combinations reduces the performance of this method. Further, with this method, some candidate blob clusters are either too small or too large to compare favorably to
Harris Stephen C.
Krumm John C.
Lyon & Harr LLP
Mariam Daniel
Microsoft Corporation
Watson Mark A.
LandOfFree
System and process for identifying and locating people or... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and process for identifying and locating people or..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and process for identifying and locating people or... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3332539