Image analysis – Applications – Motion or velocity measuring
Reexamination Certificate
1999-08-17
2003-07-01
Patel, Jayanti K. (Department: 2625)
Image analysis
Applications
Motion or velocity measuring
C345S949000
Reexamination Certificate
active
06587574
ABSTRACT:
TECHNICAL FIELD OF THE INVENTION
The present invention is directed, in general, to image retrieval systems and, more specifically, to a system for representing the trajectory of a moving object or region in animated visual data for subsequent content-based indexing, retrieval, editing, analysis and enhanced visualization.
BACKGROUND OF THE INVENTION
The advent of digital television (DTV), the increasing popularity of the Internet, and the introduction of consumer multimedia electronics, such as compact disc (CD) and digital video disc (DVD) players, have made tremendous amounts of multimedia information available to consumers. As video and animated graphics content becomes readily available and products for accessing it reach the consumer market, searching, indexing and identifying large volumes of multimedia data becomes even more challenging and important.
The term “visual animated data” in this disclosure refers to natural video, as well as to synthetic 2D or 3D worlds (e.g., VRML), or to a mixture of both video and graphics (e.g., MPEG-4). Different criteria are used to search and index the content of visual animated data, such as a video clip. Video processing systems have been developed for searching frames of visual animated data to detect, identify and label objects of a particular shape or color, or to detect text in the frames, such as subtitles, advertisement text, or background image text, such as a street sign or a “HOTEL” sign.
However, multimedia content-based indexing and retrieval systems rarely take into account the trajectory of objects in the frames of visual animated data. Many of these systems were developed only for still image retrieval. Some systems were later extended to animated data by first summarizing them as consecutive sequences of shots, then representing each shot using key-frames, and finally applying on the key-frames the techniques that were developed for the still images. In a few systems, consideration was given to camera motion in a shot, but still not to object trajectory.
VideoQ, developed by the ADVENT Project of the Image and Advanced TV Lab at Columbia university, is a multimedia content-based indexing and retrieval system that deals with object motion. VideoQ allows queries based on an object's motion trail(s). The motion trail of an object is described by an ordered sequence of the object's center of mass (i.e., centroid) trajectory vectors, for each time instant in the sequence.
In different application contexts dealing with visual animated data, other representations are used to deal with motion in video frames. In coding standards such as MPEG-1, MPEG-2, MPEG-4, H.261 and H.263, motion is represented as fields of two-dimensional vectors corresponding to the “motion” of blocks of pixels between each image. Motion vectors can be skipped at any time instant on any block(s) of the image. However, this block is then considered as non-moving at that time instant. Since the pixel blocks are typically only 8×8 to 16×16 in size, this representation leads to a large number of vectors in adjacent blocks and/or consecutive images that are very similar to each other.
Moreover, although this information is called “motion” in the above standards, it was not designed to match the actual real “motion” within the animated visual material. Instead, the information is used to find similarities in surrounding images that may reduce the coding cost of the current image. Therefore, such motion vectors are unsuitable for use in multimedia data indexing and retrieval.
Presently under development is a new MPEG standard, MPEG-7, which is intended to establish a standard set of “descriptive elements” that can be used to describe different aspects of multimedia data including the motion of objects. These descriptive elements, called Descriptors and Description Schemes, directly describe the content of the visual animated data, such as a video clip, thereby providing a fast and efficient way to search through an archive of video files and animated graphics files. Besides these Descriptors (D) and Description Schemes (DS), MPEG-7 will also standardize a language to express the descriptions (DDL). Descriptions are coded so that they can be transmitted and stored efficiently. The MPEG-7 standard, however, is nowhere near completion and many of its intended objectives may never be realized. There is no guarantee that the trajectory of objects will be adequately addressed.
There is therefore a need in the art for improved systems and methods for describing the trajectory of objects in a series of visual animated data frames. In particular, there is a need in the art for systems that are capable of determining the trajectory of an object in visual animated data frames and representing the detected trajectory of the objects in a Descriptor or Description Scheme that is suitable for use in a content-based indexing and retrieval system.
SUMMARY OF THE INVENTION
To address the above-discussed deficiencies of the prior art, it is a primary object of the present invention to provide a flexible and generic representation for the trajectory of objects in order to make searching and indexing easier. The disclosure does not address the coding of the description nor its expression within the description definition language (DDL). The proposed descriptive data structure, when using MPEG-7 terminology, can be considered either as a composited Descriptor or as a simple primary Description Scheme.
The present invention is not constrained to the needs of one or more particular applications or to any particular data source format. Advantageously, the present invention links descriptors to human perceptual criteria and to the actual semantic content that the data describe. Humans perceive motion at a high level. Accordingly, the present invention uses a high level description for the trajectory of an object by representing it in the scene as the trajectory of one point of the object, such as its center of mass (or centroid). In order to further describe the motion of a scene, the object-based descriptions can be complemented by a camera (or viewpoint) motion description. Finer details could also be added by complementing it with a description for the object deformation, if any.
In an advantageous embodiment of the present invention, there is provided, for use in a system capable of detecting a movement of a selected object in a sequence of visual animated data frames, a video processing device capable of generating a descriptor data structure representative of a trajectory of the selected object. The video processing device comprises an image processor capable of identifying the selected object in a first visual animated data frame and at least a second visual animated data frame and determining therefrom a trajectory of the selected object in a coordinate space having at least a first dimension and a second dimension. The image processor generates the descriptor data structure from the trajectory by generating at least two of: a) first trajectory data representing a position of the object in the coordinate space; b) second trajectory data from which a speed of the object in the coordinate space may be determined; and c) third trajectory data from which an acceleration of the object in the coordinate space may be determined.
The present invention therefore represents the trajectory of objects in generic, high-level terms that are readily understandable to a user. Thus, a user can search for an object in a sequence of visual animated data frames, such as a video tape, simply by giving an exemplary sequence or by giving a specific speed, acceleration, or location in the frames, or a combination thereof. The video processing device can then rapidly search the trajectory descriptor table for each object in the video tape in order to find object(s) that match the user-specified search criteria.
In one embodiment of the present invention, the coordinate space comprises a first dimension, a second dimension orthogonal to the first dimension, and a third dimension o
Koninklijke Philips Electronics , N.V.
Patel Jayanti K.
Tabatabai Abolfazl
LandOfFree
System and method for representing trajectories of moving... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for representing trajectories of moving..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for representing trajectories of moving... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3019832