Image analysis – Applications – Target tracking or detecting
Reexamination Certificate
1998-03-24
2001-05-22
Mancuso, Joseph (Department: 2621)
Image analysis
Applications
Target tracking or detecting
C382S153000, C382S113000
Reexamination Certificate
active
06236737
ABSTRACT:
BRIEF INTRODUCTION
The present invention relates to a system for dynamically addressing targets located on a moving object so that their motion may be viewed by a plurality of cameras and stored in a computer.
According to the present invention, the computer is connected to a device which controls the activation of active targets based on an algorithm stored in the computer that is sensitive to the motion of the targets so that selected groups of targets are activated more often that of other selected groups. This allows the computer to track the motion of the entire moving object in an efficient manner so as to minimize the amount of computing time and power necessary. By employing the present invention multiple targets can be tracked through complex degrees of motion to provide “real time” tracking of the subject using a relatively moderately powerful computer.
The tracking of deformable mobile objects is rapidly becoming a field of interest in a variety of applications in the areas of industry and entertainment.
The problem of interpreting a dynamically changing visual scene has made motion tracking a cumbersome and difficult task, requiring computers to interpret and assess motion in a complicated real-world scene. In order to simplify the task of tracking objects by machines, targets can be placed on an object, avoiding the need to recognize and track arbitrary outlines, of, for example, the arms, hands, and legs of a person in motion.
The targets can be passive targets, such as a white disc fastened onto critical locations of the clothing of a person being observed. The clothing of the person and the background must then be made to contrast with the targets. The cameras can then distinguish between a target and the moving parts of the subject and the background. The more contrast between the target and its surroundings, the better the signal to noise ratio of the system.
Active targets create a better signal to noise ratio and eliminate the need to provide contrasting colors between the subject and background and the targets. One such active target can be a light emitting diode LED. One preferred type of LED emits in the near infrared NIR part of the spectrum. By placing a filter over the lens of the cameras tracking the targets, the NIR sources can be clearly distinguished from any passive background, and the signal to noise ratio of the system is improved.
A number of systems using active targets are currently being developed in a variety of applications. A few of these systems use visible light sources as targets, others use infrared and near infrared sources. There are also systems available based on magnetic and radio active targets. The limitations on prior art efforts have included available computer power, exorbitant cost, active target control, and real-time implementations. Limitations have been due in part to the approaches that have been taken.
To realistically track a complicated, deformable object with many degrees of freedom, a large number of targets must be placed on the object to assist in the interpretation of rotation, bearing, and azimuth of the various moving parts. An object may require greater than 60 targets to adequately describe its motion. In an active or passive system where all targets are visible for an entire frame, geometric reconstruction from multiple two-dimensional views to a three-dimensional coordinate system must be performed on all 60 targets at once. When the combinations and permutations of 60 targets with many degrees of freedom are considered, the complexity of the solution is very large.
In order to simplify the computational burden of such a task, a number of constraints must be applied. When tracking a human, these constraints include inverse kinematics (IK) and dynamics. The general assumption of IK is that angles are constrained to certain ranges due to physiological limitations. Dynamic constraints reflect the fact that the human body can only achieve limited acceleration resulting in a maximum displacement over a given time interval. Finally, assumptions with respect to target location can be made by assuming limb connectivity. This means that the wrist must be somewhere nearby the elbow by definition.
The complications involved in tracking deformable objects affect the complexity of algorithms that must interpret and identify the scene of many targets. Not only is it difficult to develop a robust algorithm to deal with such a situation, but geometric reconstruction algorithms are generally very computer intensive. The most common algorithm implementation in commercial systems is a derivative of global optimization, which is a relatively “brute force” method of finding the best orientation of targets given a set of constraints. Because of the computational complexity of the algorithms, one must trade off response time of the system for cost. Most systems have significant delays between motion and successful tracking, due to the computational delay of the system. Parallel processing units are often implemented in an attempt to shorten the computational delay. This results in very expensive systems that are either real-time or have a short delay, or inexpensive systems that require large intermittent waits after one or two minutes of motion capture.
Many of these systems will not achieve real-time performance without significant investment in powerful computing architecture. The inventors are not the first to use NIR targets, or to enhance the signal to noise ratio of such targets using CCD cameras and suitable filters. The system of the present invention does use a more sophisticated algorithm enabling the active targets, such as to reduce the computational task of target tracking a real-world object to within the capability of a single Pentium (trademark) processor.
In one particular embodiment of the present invention, CCD cameras have been combined with a Kodak (trademark) 87C Wratten (trademark) filter to block out the visible spectrum. Using this embodiment, exceptional signal to noise ratios have been achieved in natural lighting settings without any need for blue screening of the background or the tracked object.
The main challenges in target motion tracking in general are the occlusion and the correspondence problems. Interpreting and tracking 60 targets is further complicated by the fact that paths of targets may often intersect one another. In the case of the human actor, when two wrists cross over one another one target will temporarily disappear then reappear a few frames latter. This is known as occlusion. Interpreting this phenomenon and distinguishing between a clap and crossing of the wrists is difficult. When a virtual actor is running, ambiguity between left and right sides of the individual can prove to be difficult to identity.
These two problems, occlusion and correspondence are difficult problems to solve in target tracking and are discussed below.
The correspondence problem in geometrical reconstruction is primarily the challenge of associating features in one frame of a video signal with those of the next consecutive frame. With one visible target, the correspondence problem is immediately solvable as there is only one obvious match for a single feature in multiple images. However, with a large number of features being tracked in time, the correspondence of multiple features can be quite difficult to establish. This problem is particularly evident for segmented or limbed objects that have many degrees of freedom, making it difficult to differentiate between different motions. Most manufactures of motion tracking apparatus solve the correspondence problem through a brute force optimization method, considering the possible orientations of the tracked object and attempting to force the skeletal structure to the array of recorded features. This method can be improved upon by considering trajectories obtained from previous frames. The disadvantage of this method is the huge computational complexity involved in global optimization of a large number of features.
A target is occluded when it is not obser
Franklin Steve M.
Gregson Peter H.
Dalhousie University
Mancuso Joseph
Tabatabai A
LandOfFree
Dynamic target addressing system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Dynamic target addressing system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dynamic target addressing system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2485978