Image analysis – Applications – Motion or velocity measuring
Reexamination Certificate
2000-09-01
2004-02-17
Johnson, Timothy M. (Department: 2621)
Image analysis
Applications
Motion or velocity measuring
C342S095000, C382S224000
Reexamination Certificate
active
06694044
ABSTRACT:
BACKGROUND OF THE INVENTION
The task of classifying sequences of data is a key technology in a broad range of applications. In econometrics and forecasting applications, classification of data sequences which can represent the inventory of product in a distribution channel over time, or the price of a stock or other financial instrument over time can be crucial. Different classes may, for instance, signify various important stock market states. In human motion applications, data sequences which represent the pose of the human body or one of its parts over time can be classified into categories. For example, a running motion can be distinguished from walking. Movements of hands can be automatically interpreted as individual signs in American Sign Language and then used to interface to a computer.
In classification applications, the goal is to assign class labels to observed data sequences or trajectories. ,Dynamic models can be useful in classification because they provide an efficient coding of the set of all possible trajectories. For example, suppose that two distinct gestures can be described with separate linear dynamical models. For each model, it is straightforward to compute an error signal, called the innovation, which measures the extent to which the model predicts the observations. This innovation can be used directly for classification. The parameters of the linear dynamic models therefore provide a very compact representation of the class of trajectories that make up a particular gesture.
Sets of dynamic models can be used to model qualitatively different regimes of a trajectory associated with one temporal event. For instance, a hand gesture can be segmented into three motion regimes or phases: preparation, stroke and retraction. Each regime can then be associated with a different linear model. The sequence of regimes can be governed by a model switching process.
Switching linear dynamic system (SLDS) models consist of a set of linear dynamic models and a switching variable that determines which model is in effect at any given point in time. In addition, fully connected SLDS models assume that there are temporal dependencies between the switching variables as well as the states of different linear dynamic models. SLDS models are attractive because they can describe complex dynamics using simple linear models as building blocks. Given an SLDS model, the inference problem is to estimate the sequence of model states that best explains an input data sequence. Unfortunately, exact inference in SLDS models is computationally intractable, due to the large number of possible combinations of linear models over time.
One prior method for inference using fully connected SLDS models is described in “Estimation and Tracking: Principles, Techniques, and Software” by Bar-Shalom et al., Artech House, Inc. 1993. In this method, approximate inference is achieved by truncating or collapsing the number of discrete components in the evolving model. The models were used to detect different motion regimes while tracking a maneuvering target.
In another prior method the switching variable determines which linear model is coupled to the measurement at each time instant. See Ghahramani et al., “Variational Leaming for Switching State-Space Models” which will appear in the journal Neural Computation. This method can produce decoupled linear models which reach steady-state before the data series is adequately modeled. It stands in contrast to the prior methods with fully coupled SLDS in which all models are coupled through a single state space. The method was used to segment regimes of no breathing and gasping breathing in data collected from patients with sleep apnea.
A different prior method in “Time-series Classification Using Mixed-State Dynamic Bayesian Networks,” by Pavlovic et al., Proc. of Computer Vision and Pattern Recognition, pages 609-615, June, 1999, considers a single linear dynamical model whose input is modeled as a discrete Markov process. This model explains all measurement variability as a consequence of the changes in input, which may not be true in general. The model was applied to classification of computer mouse-drawn symbols.
Another prior method proposes particle filters as an alternative to using linear models as the building blocks in a switching framework. See Blake et al., “Learning Multi-Class Dynamics,” Advances in Neural Information Processing Systems (NIPS '98), pages 389-395, 1998. The use of a nonparametric, particle-based model can be inefficient in domains where linear models are a powerful building block. Nonparametric methods are particularly expensive when applied to large state spaces, since they are exponential in the state space dimension.
In another prior method, a Hidden Markov Model with an entropic prior is proposed for dynamics learning from sparse input data. See Brand, “Pattern discovery via entropy minimization,” Technical Report TR98-21, Mitsubishi Electric Research Lab, 1998. The method is applied to the synthesis of facial animation and, to a certain extent, the segmentation of facial expressions from voice data, e.g., Brand, “Voice puppetry,” Proceeding of SIGGRAPH99, 1999. The dynamic models produced by this method are time invariant. Each state space neighborhood has a unique distribution over state transitions. In addition, the use of entropic priors results in fairly deterministic models learned from a moderate corpus of training data. In many applications, time-invariant models are unlikely to succeed, since different state space trajectories can originate from the same starting point, depending upon the class of motion being performed.
Modeling Human Dynamics
Technologies for analyzing the motion of the human figure play a key role in a broad range of applications, including computer graphics, user-interfaces, surveillance, and video editing. A motion of the figure can be represented as a trajectory in a state space which is defined by the kinematic degrees of freedom of the figure. Each point in state space represents a single configuration or pose of the figure. A motion such as a plié in ballet is described by a trajectory along which the joint angles of the legs and arms change continuously.
A key issue in human motion analysis and synthesis is modeling the dynamics of the figure. While the kinematics of the figure define the state space, the dynamics define which state trajectories are possible (or probable) in that state space. Prior methods for representing human dynamics have been based on analytic dynamic models. Analytic models are specified by a human designer. They are typically second order differential equations relating joint torque, mass, and acceleration.
The field of biomechanics is a source of complex and realistic analytic models of human dynamics. From the biomechanics point of view, the dynamics of the figure are the result of its mass distribution, joint torques produced by the motor control system, and reaction forces resulting from contact with the environment, e.g., the floor. Research efforts in biomechanics, rehabilitation, and sports medicine have resulted in complex, specialized models of human motion. For example, detailed walking models are described in Inman et al., “Human Walking,” Williams and Wilkins, 1981.
The biomechanical approach has two drawbacks. First, the dynamics of the figure are quite complex, involving a large number of masses and applied torques, along with reaction forces which are difficult to measure. In principle, all of these factors must be modeled or estimated in order to produce physically valid dynamics. Second, in some applications, we may only be interested in a small set of motions, such as a vocabulary of gestures. In the biomechanical approach it may be difficult to reduce the complexity of the model to exploit this restricted focus. Nonetheless, biomechanical models have been applied to human motion analysis.
A prior method for visual tracking uses a biomechanically-derived dynamic model of the upper body. See Wren et al., “Dynamic models of human motion,” Proce
Pavlović Vladimir
Rehg James Matthew
Azarian Seyed
Hewlett--Packard Development Company, L.P.
Johnson Timothy M.
LandOfFree
Method for motion classification using switching linear... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for motion classification using switching linear..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for motion classification using switching linear... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3349434