Data processing: generic control systems or specific application – Generic control system – apparatus or process – Optimization or adaptive control
Reexamination Certificate
2000-06-13
2004-06-29
Patel, Ramesh (Department: 2121)
Data processing: generic control systems or specific application
Generic control system, apparatus or process
Optimization or adaptive control
C700S028000, C700S029000, C700S030000, C700S031000, C382S103000, C382S104000, C382S105000, C706S015000, C706S016000, C706S020000
Reexamination Certificate
active
06757571
ABSTRACT:
BACKGROUND
1. Technical Field
The invention is related to a system and process for automatically generating a reliable vision-based tracking system, and more particularly, to a system and process for using information gathered from an initial object tracking system to automatically learn an object model tailored to at least one specific target object to create a tracking system more reliable than the initial object tracking system.
2. Related Art
Most current systems for determining the presence of objects of interest in an image of a scene have involved processing a temporal sequence of color or grayscale images of a scene using a tracking system. Objects are typically recognized, located and/or tracked in these systems using, for example, color-based, edge-based, shape-based, or motion-based tracking schemes to process the images.
While the aforementioned tracking systems are useful, they do have limitations. For example, such object tracking systems typically use a generic object model having parameters that roughly represent an object for which tracking is desired in combination with a tracking function such as, for example, a color-based, edge-based, shape-based; or motion-based tracking function. In general, such object tracking systems use the generic object model and tracking function to probabilistically locate and track at least one object in one or more sequential images.
As the fidelity of the generic object model increases, the accuracy of the tracking function also typically increases. However, it is not generally possible to create a single high fidelity object model that ideally represents each of the many potential derivatives or views of a single object type, such as the faces of different individuals having different skin coloration, facial structure, hair type and style, etc., under any of a number of lighting conditions. Consequently, such tracking systems are prone to error, especially where the actual parameters defining the target object deviate in one or more ways from the parameters defining the generic object model.
However, in an attempt to address this issue, some work has been done to improve existing object models. For example, in some facial pose tracking work, 3D points on the face are adaptively estimated or learned using Extended Kalman Filters (EKF) [1,6]. In such systems, care must be taken to manually structure the EKF correctly [3], but doing so ensures that as the geometry of the target face is better learned, tracking improves as well.
Other work has focused on learning the textural qualities of target objects for use in tracking those objects. In the domain of facial imagery, there is work in which skin color has been modeled as a parametrized mixture of n Gaussians in some color space [7, 8]. Such work has covered both batch [7] and adaptive [8] learning with much success. These systems typically use an expectation-maximization learning algorithm for learning the parameters, such as skin color, associated with specific target objects.
Although color distributions are a gross quality of object texture, learning localized textures of target objects is also of interest. Consequently, other work has focused on intricate facial geometry and texture, using an array of algorithms to recover fine detail [4] of the textures of a target object. These textures are then used in subsequent tracking of the target object.
Finally, work has been done in learning the dynamic geometry, i.e. the changing configuration (pose or articulation), of a target. The most elementary of such systems use one of the many variations of the Kalman Filter, which “learns” a target's geometric state [2]. In these cases, the value of the learned model is fleeting since few targets ever maintain constant dynamic geometries. Other related systems focus on models of motion. Such systems include learning of multi-state motion models of targets that exhibit a few discrete patterns of motion [5, 9].
However, the aforementioned systems typically require manual intervention in learning or fine-tuning those tracking systems. Consequently, it is difficult or impossible for such systems to quickly respond to the dynamic environment often associated with tracking moving target objects under possibly changing lighting conditions. Therefore, in contrast to the aforementioned systems, what is needed is a system and process for automatically learning a reliable tracking system during tracking without the need for manual intervention and training of the automatically learned tracking system. The system and process according to the present invention resolves the deficiencies of current locating and tracking systems by automatically learning, during tracking, a more reliable tracking system tailored to specific target objects under automatically observed conditions.
It is noted that in the preceding paragraphs, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. Multiple references are identified by a pair of brackets containing more than one designator, for example, [5, 6, 7]. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.
SUMMARY
The present invention involves a new system and process for automatically learning an object model for use in a vision-based tracking system. To address the issue of model fidelity with respect to specific target objects, the learned object model is automatically tailored to represent one or more specific target objects, such as, for example, specific aircraft, cars, people, animals, faces, or any other object in a temporal sequence of at least one image. Learning of the object model is accomplished by automatically determining probabilistic relationships between target state estimates produced by an initial generic tracking system and observations gathered from each image. The learned object model is then employed with a final tracking function to produce an improved tracking system more accurate than the initial generic tracking system.
In general, the system and method of the present invention automatically generates a reliable tracking system by using an initial object model in combination with an initial tracking function to process a temporal sequence of images, and a data acquisition function for gathering observations about each image. Further, in one embodiment, these observations are associated with a measure of confidence that represents the belief that the observation is valid. This measure of confidence may be used to weight the observations. Observations gathered by the data acquisition function are relevant to parameters or variables desired for a learned or final object model. These relevant observations may include information such as the color, shape, texture, size, or any other visual or geometric characteristics of a tracked object, and depend on the parameters necessary to support a known final tracking function. These relevant observations are used by a learning function in combination with the output of the initial tracking function for automatically learning an object model automatically tailored to a specific target object.
The final tracking function may be the same as the initial tracking function, or may be entirely different. For example, both the initial and final tracking function may use edge detection methods to locate target objects in an image. Alternately, the initial tracking function may use color-based detection methods while the final tracking function may use shape-based detection methods to locate target objects in an image. Thus, any type or combination of tracking methods may be used for the initial and final tracking functions.
Data output from the initial tracking function, in combination with the observations generated by the data acquisition function, are f
Lyon & Harr LLP
Microsoft Corporation
Patel Ramesh
Watson Mark A.
LandOfFree
System and process for bootstrap initialization of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and process for bootstrap initialization of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and process for bootstrap initialization of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3360193