Image analysis – Image compression or coding
Reexamination Certificate
1999-06-17
2003-10-14
Do, Anh Hong (Department: 2721)
Image analysis
Image compression or coding
C382S235000, C382S236000, C382S238000
Reexamination Certificate
active
06633673
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a method and apparatus for processing compressed data and, for example, a method and apparatus for directly modifying transform coefficients associated with blocks of the compressed data in a transform-domain in consideration of temporal dependencies of the compressed data.
2. Description of the Related Art
As with many of today's technologies, the current trend in image sequence developing and editing is to use digital formats. Even with motion picture film, editing of image sequences (including image splicing, color processing, and special effects) can be much more precisely accomplished by first converting images to a digital format, and performing desired edits upon the digital format. If desired, images can then be converted back to the original format.
Unfortunately, digital formats usually use enormous amounts of memory and transmission bandwidth. A single image with a resolution of 200×300 pixels can occupy megabytes of memory. When it is considered that many applications (for example, motion picture film processing) require far greater resolution, and that image sequences can include hundreds or thousands of images, it becomes very apparent that many applications are called upon to handle gigabytes of information, creating a bandwidth problem, in terms of computational and transmission resources.
To solve the bandwidth problem, standards have been proposed for image compression. These standards generally rely upon spatial or temporal redundancies which exist in one or more images.
A single image, for example, may have spatial redundancies in the form of regions having the same color (intensity and hue); a single, all blue image could potentially be represented simply by its intensity and hue, and information indicating that the entire frame has the same characteristics.
Temporal redundancies typically exist in sequences of images, and compression usually exploits these redundancies as well. For example, adjacent images in a sequence can be very much alike; exploiting redundancies, a compressed image sequence may include data on how to reconstruct current image frames based upon previously decoded frames. This data can be expressed as a series of motion vectors and difference information. To obtain this information, pixels in the second frame are grouped into image squares of 8×8 or 16×16 pixels (“blocks” of pixels), and a search is made in a similar location in a prior frame for the closest match. The motion vectors and difference information direct a decoder to reconstruct each image block of the second frame by going back to the first frame, taking a close match of the data (identified by the motion vector) and making some adjustments (identified by the difference information), to completely reconstruct the second frame.
One group of standards currently popular for compression of image sequences has been defined by the Moving Pictures Experts' Group, and these standards are generally referred to as “MPEG.” The MPEG standards generally call for compression of individual images into three different types of compressed image frames: compressed independent (“I”) frames exploit only spatial redundancies, and contain all the information necessary to reconstruct a single frame; compressed prediction (“P”) frames exploit temporal redundancies from a prior frame (either a P or I frame) and typically only require about ⅓ as much data as an I frame for complete frame reconstruction; and compressed bi-directional interpolated (“B”) frames can use data from either or both of prior and future frames (P or I frames) to provide frame reconstruction, and may only require ¼ as much data as a P frame. Other compression standards also rely upon exploitation of temporal image redundancies, for example, H.261 and H.263.
Compressed data such as video signals are often difficult to manipulate without having to decompress, perform an operation and recompress the data. For example, fade operations have typically been carried out on decompressed video. The fade operation is often used in video broadcasting. One typical example is that television (TV) stations splice in commercial clips during regular TV program broadcasting. An abrupt beginning of a commercial could annoy viewers; a gradual fade to the black of the immediately preceding video is much more preferred. The operation of gradually fading to black is called a “fade-out” operation. On the other hand, the operation of gradually fading from black to full or partial picture information is called a “fade-in” operation.
In digital TV broadcasting, regular TV programs (live or pre-recorded) are typically stored and transmitted in a compressed form. MPEG-2 is a compressed form used in many digital TV consortia such as HDTV or ATSC. A conventional way of performing fade on MPEG sequence is to decompress the sequence, apply the fading operation and recompress it back. Within this loop, costly DCT and motion estimation operations make it effectively impossible for real time applications. Therefore, a need exists for a fade technique applicable in the compressed domain to avoid these two bottlenecks.
Although it is known to implement operations directly on compressed JPEG data, see Brian Smith and Larry Rowe, “Algorithms For Manipulating Compressed Images,”
IEEE Computer Graphics and Applications
, pp. 34-42, September 1993, there are problems, particularly with respect to MPEG. For example, since MPEG utilizes interframe coding, frames of a picture may be coded depending on one or two other frames. Also, within these pictures, different coding methods may apply on different types of macroblocks. Thus, a universal scheme for different types of macroblocks in different types of pictures is needed. Additionally, operation on DC coefficients can only change the brightness of the whole DCT block uniformly which may lead to a problem in a fade-out operation when the fade-out approaches black. It would be helpful to have an approximated method based on the consideration that the pictures are almost black when the macroblocks or other data in the fade-out operation approach black. Furthermore, considering the whole process within the MPEG context, the variable quantization used in MPEG may introduce error-accumulation problems. It would be helpful to improve the visual quality of the fade results, such as by a correction process.
SUMMARY OF THE INVENTIONS
The present inventions are directed to methods and apparatus for operating or modifying data, and especially compressed data, without having to decompress the data. They can do so even if there are temporal or spatial dependencies within the data, and even when the data is arranged in a format different than the format in which the data will ultimately be used, such as MPEG video data. In MPEG video, the data is stored in a different order than that in which it will be displayed. As a result, the data can be processed in the order in which it exists, such as in its storage form, rather than in its useful form, e.g. the display order. One particularly advantageous form of the invention is used to produce fade-in and fade-out operations on MPEG video, which is a relatively complicated compression data form. The data is stored as frames of data in one order and displayed as frames of data in another form and in another order. Moreover, the frames of data in the stored format are not complete in and of themselves and depend for their completeness for display purposes on data contained in other frames of stored data. Furthermore, the dependencies on data in other frames apply not only to data in previous frames but also to data in frames displayed subsequently. Several aspects of the present inventions account for these dependencies. However, it can also operate on more simple forms of compressed data.
In an exemplary preferred embodiment, methods and apparatus are provided for a fade operation on MPEG video or other compressed data wherein either one or both of the following
Do Anh Hong
Hewlett--Packard Development Company, L.P.
LandOfFree
Fast fade operation on MPEG video or other compressed data does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Fast fade operation on MPEG video or other compressed data, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Fast fade operation on MPEG video or other compressed data will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3132463