Television – Object tracking
Utility Patent
1997-07-03
2001-01-02
Britton, Howard (Department: 2713)
Television
Object tracking
Utility Patent
active
06169573
ABSTRACT:
BACKGROUND OF THE INVENTION
The invention relates to a system and method for hypervideo browsing and linking among moving objects represented in a compressed digital video context, and more particularly to a system and method capable of following objects in changed positions from frame to frame by using motion vectors present in an MPEG-compressed digital video stream or other video stream encoded by a method utilizing motion compensation techniques.
Video has traditionally been recorded on analog media, in which an electrical signal (such as that used by a television receiver) is directly encoded onto a storage medium, such as magnetic tape or a laserdisk. From a technological point of view, analog video encoding and transmission are relatively simple to accomplish; color television uses standards that are nearly fifty years old.
However, analog video has several significant disadvantages. First of all, the signal quality of analog video degrades when sent over long distances, stored and retrieved, or otherwise manipulated in the analog domain. Although precision electronics and strictly controlled environments can minimize the degradation, it generally cannot be completely eliminated. Second, analog video is typically not directly understood by computers. With the increasing convergence between computers and multimedia, this is an important consideration.
Digital encoding, on the other hand, uses the language of computers for the storage of video. There is an increasing trend toward storing and transmitting video that has been digitally stored. For example, the new DVD (Digital Versatile Disk) standard, as well as certain direct-broadcast satellite television systems (such as Hughes Electronics' “DirecTV” system), and several proposed standards for HDTV (high-definition television) all employ digital video encoding. Such digital video, because it is based on a sequence of numerical codes that can be, to some extent, reconstructed if a signal is damaged, does not suffer the degradation problems inherent in analog video. Digital video can be transmitted, received, stored, and retrieved without any necessary loss in signal quality.
Tracking objects in video may be of interest for a number of reasons. One application that has been identified is known as hypervideo. In hypervideo, video clips can be used to link to and from different topics of interest, much as hypertext is commonly used for linking among topics on the World Wide Web. In hypertext, various words in a passage of text may be highlighted, thereby indicating the user that they contain a link to another area. If the user selects (e.g., clicks on) a highlighted word, the text corresponding to the selected word will then be displayed, which may in turn contain additional links.
Hypervideo can work in much the same way. A video clip may contain certain objects which are linked to further information. If the user selects an object, for example a red automobile, he may then be presented with additional information about the automobile, or may be shown a different video clip or a different segment of the same clip, which may in turn contain additional links.
However, in hypervideo applications, it has been found to be necessary to track the individual objects within the video clips used. If the exemplary automobile is moving, it is useful for the hypervideo system to be able to determine whether the user has selected the automobile, regardless of where it is on the screen when the selection is made.
Where the video is artificially prepared, as by computer graphics, this is relatively easy. In order to construct the video, the positions of objects therein are specified on a coordinate system. That information can be then used to track the objects, if desired.
However, when the video is prepared from other sources, such as a recording of a real-world scene, or transferred from film, object tracking becomes far more difficult. The video becomes simply a sequence of frames, wherein each frame represents a still picture or “snapshot” of a particular moment in time. The snapshots contain visual information sufficient to permit the human brain to distinguish individual objects, but a computer usually cannot easily do so.
In digital video applications, each frame contains a pattern of colored picture elements, or “pixels,” which the human eye and mind are easily able to interpret as a scene. For example, a digital video may represent a red automobile traveling down an asphalt highway. Digitally, the auto would be represented as an irregularly-shaped region of red and near-red pixels. A human observer would easily be able to discriminate this pattern from its surroundings.
However, computer applications typically do not have this ability. In a typical computer display memory architecture, the red and near-red pixels representing the auto may be scattered throughout the frame, one line or “raster” at a time, interleaved with various other objects. Moreover, the auto may pass behind certain objects (for example, a tree on the side of the road). Even if a sophisticated computer system is able to recognize an automobile in a video frame by its shape, this ability may be reduced or confused when the auto is partially obscured.
Further complicating this problem is the issue of video compression. Uncompressed, a single frame of relatively low-resolution video (e.g. 352×240 pixels), in full color, can require a storage capacity in excess of 250,000 bytes (250K). As a single second of video includes approximately 30 frames, a five-minute video would then require more than 2,000 megabytes. This amount of storage is impractical, especially when it is considered that a typical feature-length movie is longer than 90 minutes (which would require nearly 40 gigabytes, or 40,000,000,000 bytes, of storage).
Digital video compression can reduce these requirements by a factor of 25 or more. One standard frequently used for the compression of digital video is known as MPEG, for the Moving Picture Experts Group which established the standard. Using MPEG, approximately 60 minutes of relatively low-resolution video (as defined above) can be stored in approximately 650 megabytes of storage, or on a single CDROM disk. MPEG is a “lossy” compression technique, which means that the high degree of compression accomplished has a downside. There is a loss of detail in the video. This loss of detail may or may not be easily visible to the casual observer, but intricate details within individual video frames may be washed out somewhat. This loss of information makes the task of tracking objects represented in the video even more difficult.
A commercial hypervideo and object tracking tool called V-ACTIVE is available from Ephyx Technologies. However, the current hypervideo implementation of V-ACTIVE has at least four notable characteristics. First, objects are tracked by way of hot regions having regular shapes and fixed sizes. These shapes must be specified by the hypervideo author. For example, the hot region for a moving automobile might be represented by a rectangle. Second, the concept of “hypervideo” is limited in its ability to link among scenes in a single video clip: only one sequence through a video document is available. Third, V-ACTIVE has difficulty tracking objects that become partially or wholly obscured for even a few frames. Finally, V-ACTIVE does not appear to be able to track motion in compressed video without first fully decompressing the video.
Accordingly, there is a need for a hypervideo object tracking system and method that does not suffer from the disadvantages of prior art hypervideo authoring tools. Such a tracking system would work on digitally stored video, preferably already in the compressed domain. Trackable objects should be able to have any arbitrary shape, and the tracked “hot regions” should be able to adapt to a changing shape. The system should be able to track an object even when it is partially obscured. The hypervideo system should track objects through multiple paths between scenes or segments in a single video.
Goldman Kenneth J.
Rangan P. Venkat
Sampath-Kumar Srihari
Talreja Kamlesh
Boys Donald R.
Britton Howard
Central Coast Patent Agency
HOTV, Inc.
LandOfFree
Hypervideo system and method with object tracking in a... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Hypervideo system and method with object tracking in a..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Hypervideo system and method with object tracking in a... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2494875