Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
1999-06-17
2002-12-03
Lee, Richard (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
Reexamination Certificate
active
06490323
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present inventions relate to methods and apparatus for processing data in a compressed format, including the processing of a plurality of frames of input data while in a transform domain and, for example, a method and apparatus for processing input vectors associated with the input data frames to determine contributions of cross-products of the input vectors to an output vector associated with a frame of output data in consideration of an orthogonal characteristic of a convolution operation employed to generate the output data.
2. Description of the Related Art
As with many of today's technologies, the current trend in image sequence developing and editing is to use digital formats. Even with motion picture film, editing of image sequences (including image splicing, color processing, and special effects) can be much more precisely accomplished by first converting images to a digital format, and performing desired edits upon the digital format. If desired, images can then be converted back to the original format.
Unfortunately, digital formats usually use enormous amounts of memory and transmission bandwidth. A single image with a resolution of 200×300 pixels can occupy megabytes of memory. When it is considered that many applications (for example, motion picture film processing) use far greater resolution, and that image sequences can include hundreds or thousands of images, it becomes very apparent that many applications are called upon to handle gigabytes of information, creating a bandwidth problem, in terms of computational and transmission resources.
To solve the bandwidth problem, standards have been proposed for image compression. These standards generally rely upon spatial or temporal redundancies which exist in one or more images.
A single image, for example, may have spatial redundancies in the form of regions having the same color (intensity and hue); a single, all blue image could potentially be represented simply by its intensity and hue, and information indicating that the entire frame has the same characteristics.
Temporal redundancies typically exist in sequences of images, and compression usually exploits these redundancies as well. For example, adjacent images in a sequence can be very much alike; exploiting redundancies, a compressed image sequence may include data on how to reconstruct current image frames based upon previously decoded frames. This data can be expressed as a series of vectors and difference information. To obtain this information, pixels in the second frame are grouped into image squares of 8×8 or 16×16 pixels (“blocks” of pixels), and a search is made in a similar location in a prior frame for the closest match. The vectors and difference information direct a decoder to reconstruct each image block of the second frame by going back to the first frame, taking a close match of the data (identified by the vector) and making some adjustments (identified by the difference information), to completely reconstruct the second frame.
One group of standards currently popular for compression of image sequences has been defined by the Moving Pictures Experts' Group, and these standards are generally referred to as “MPEG.” The MPEG standards generally call for compression of individual images into three different types of compressed image frames: compressed independent (“I”) frames exploit only spatial redundancies, and contain all the information used to reconstruct a single frame; compressed prediction (“P”) frames exploit temporal redundancies from a prior frame (either a P or I frame) and typically only use about ⅓ as much data as an I frame for complete frame reconstruction; and compressed bi-directional (“B”) frames can use data from either or both of prior and future frames (P or I frames) to provide frame reconstruction, and may only use ¼ as much data as a P frame. Other compression standards also rely upon exploitation of temporal image redundancies, for example, H.261 and H.263.
Chroma-keying or blue screen matting is used widely in digital video editing to create the illusion of motion or presence at some specific place. In such applications, an object is filmed against a blue background which, in the editing process, is replaced by a static or a moving shot at some specific place to create the desired illusion. Unlike simple overlapping, in which the background of the overlapping image/video is black (a zero in digital image representation) and can be done via masking, chroma-keying uses the transparency of the chromakey pixels thus making a pixel-wise operation necessary. The degree of transparency of each pixel is called the alpha channel or alpha image. Accordingly, chroma-keying is also referred to as “alpha blending”. The alpha channel, which represents the transparency of each pixel, can be derived from the chromakey specified by the video editor, and then a pixel multiplication operation is performed to accomplish the chroma-keying effect.
In digital TV broadcasting, regular TV programs (live or pre-recorded) are typically stored and transmitted in a compressed form. MPEG-2 is a compressed form used in many digital TV consortia such as HDTV or ATSC. Conventional processing of compressed image or video data involves first decompressing the data, and then applying the desired processing function. The processed data is then recompressed for transmission or storage.
Compressed domain processing may yield several advantages vis-a-vis spatial domain processing such as (a) smaller data volume, (b) lower computation complexity since the processes of complete decompression and compression can be avoided, and (c) preservation of image fidelity since decompression-compression processes can often be eliminated. Thus, it would be helpful to replace the spatial domain processing scheme with an equivalent processing of the compressed domain representation.
A conventional way of performing chroma-keying on MPEG sequence is to decompress the sequence, apply the chroma-keying operation and recompress it back. Within this loop, costly DCT and motion estimation operations may make it effectively impossible for real time applications. Therefore, it would be helpful to have a chroma-keying technique applicable in the compressed domain to avoid the DCT and motion estimation bottlenecks.
For example, a logo keying operation is used frequently in the digital video broadcasting environment. Conventionally, the compressed stream is fully decompressed, and then compressed again after the logo-keying operation.
FIG. 2
is a functional block diagram of a conventional logo keying operation
200
. After entropy decoding of the compressed stream at block
202
, a forward discrete cosine transform (FDCT) is employed at block
204
. Logo image data is represented by block
206
. The output of the FDCT and the logo image data are both provided to block
208
which represents keying in the spatial domain. An inverse discrete cosine transform (IDCT) is applied at block
210
to the output of block
208
. And finally, block
212
represents entropy encoding of the output of block
210
to provide the recompressed stream.
In the case of chroma-keying or blue screen matting, it would be helpful if both the object video and the background video could be processed in compressed form, especially in distributed systems, such as where the object video, the background video and/or the composite video are generated in different locations.
SUMMARY OF THE INVENTION
The present inventions provide methods and apparatus for combining separate data segments while still in a compressed form, such as mixing two compressed video segments into a single combined video segment, such as a composite, while keeping the two compressed video segments substantially in their compressed formats during processing. In one aspect of the present inventions, methods and apparatus are provided for pixel multiplication in a compressed domain such as the DCT domain, which leads to an efficient scheme for chro
Bhaskaran Vasudev
Shen Bo
Hewlett--Packard Company
Lee Richard
LandOfFree
Fast compressed domain processing using orthogonality does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Fast compressed domain processing using orthogonality, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Fast compressed domain processing using orthogonality will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2948459