Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
1998-12-04
2001-02-20
Kelley, Chris S. (Department: 2713)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C348S404100
Reexamination Certificate
active
06192080
ABSTRACT:
BACKGROUND OF THE INVENTION
For an overall understanding of general techniques involved in encoding and decoding video image information, reference should be made to the “MPEG-4 Video Verification Model Version 5.0”, prepared for the International Organization for Standardization by the Ad hoc group on MPEG-4 video VM editing, paper Number MPEG 96/N1469, November 1996, the contents of which are herein incorporated by reference.
This invention relates to encoding and decoding of complex video image information including motion components which may be encountered, for example, in multimedia applications, such as video-conferencing, video-phone, and video games. In order to be able to transfer complex video information from one machine to another, it is often desirable or even necessary to employ video compression techniques. One significant approach to achieving a high compression ratio is to remove the temporal and spatial redundancy which is present in a video sequence. To remove spatial redundancy, an image can be divided into disjoint blocks of equal size. These blocks are then subjected to a transformation (e.g., Discrete Cosine Transformation or DCT), which decorrelates the data so that it is represented as discrete frequency components. With this representation, the block energy is more compact, hence the coding of each block can be more efficient. Furthermore, to achieve the actual compression, two-dimensional block elements are quantized. At this point, known run-length and Huffman coding schemes can be applied to convert the quantized data into a bit-stream. If the above process is applied to one block independent of any other block, the block is said to be intra-coded. On the other hand, if the block uses information from another block at a different time, then the block is said to be inter-coded. Inter-coding techniques are used to remove temporal redundancy. The basic approach is that a residual block (or error block) is determined based on the difference between the current block and a block in a reference picture. A vector between these two blocks is then determined and is designated as a motion vector. To keep the energy in the residual block as small as possible, block-matching algorithms (BMAs) are used to determine the block in the reference picture with the greatest correlation to the current block. With the reference block locally available, the current block is reconstructed using the motion vector and the residual block.
For the most part, video coding schemes encode each motion vector differentially with respect to its neighbors. The present inventors have observed that a piecewise continuous motion field can reduce the bit rate in this case. Hence, a rate-optimized motion estimation algorithm has been developed. The unique features of this proposal come from two elements: (1) the number of bits used for encoding motion vectors is incorporated into the minimization criterion, and (2) rather than counting the actual number of bits for motion vectors, the number of motion vector bits is estimated using the residues of the neighboring blocks. With these techniques, the bit-rate is lower than in prior encoders using full-search motion-estimation algorithms. In addition, the computational complexity is much lower than in a method in which rate-distortion is optimized. The resulting motion field is a true motion field, hence the subjective image quality is improved as well.
If we disregard for a moment the advantages that are achieved in terms of coding quality and bit rate savings, and only concentrate on the improvements in subjective image quality, it can be demonstrated that the resulting true motion field can be used at the decoder, as well, in a variety of other ways. More specifically, it has been found that the true motion field can be used to reconstruct missing data, where the data may be a missing frame and/or a missing field. In terms of applications, this translates into frame-rate up-conversion, error concealment and interlaced-to-progressive scan rate conversion capabilities, making use of the true motion information at the decoder end of the system.
Frame-Rate Up-Conversion. The use of frame-rate up-conversion has drawn considerable attention in recent years. To accomplish acceptable coding results at very low bit-rates, most encoders reduce the temporal resolution, i.e., instead of targeting the full frame rate of 30 frames/sec (fps), the frame rate may be reduced to 10 fps, which would mean that 2 out of every 3 frames are never even considered by the encoder. However, to display the full frame rate at the decoder, a recovery mechanism is needed. The simplest mechanism is to repeat each frame until a new one is received. The problem with that interpolation scheme is that the image sequence will appear very discontinuous or jerky, especially in areas where large or complex motion occurs. Another simple mechanism is linear-interpolation between coded frames. The problem with this mechanism is that the image sequence will appear blurry in areas of motion, resulting in what is referred to as ghost artifacts.
From the above, it appears that motion is the major obstacle to image recovery in this manner. This fact has been observed by a number of prior researchers and it has been shown that motion-compensated interpolation can provide better results. In one approach, up-sampling results are presented using decoded frames at low bit-rates. However, the receiver must perform a separate motion estimation just for the interpolation. In a second approach, an algorithm that considers multiple motion is proposed. However, this method assumes that a uniform translational motion exists between two successive frames. In still a third approach, a motion-compensated interpolation scheme is performed, based on an object-based interpretation of the video. The main advantage of the latter scheme is that the decoded motion and segmentation information is used without refinement. This may be attributed to the fact that the object-based representation is true in the “real” world. However, a proprietary codec used in that approach is not readily available to all users.
The method proposed in the present case is applicable to most video coding standards in that it does not require any proprietary information to be transmitted and it does not require an extra motion estimation computation. The present motion-compensated interpolation scheme utilizes the decoded motion information which is used for inter-coding. Since the current true motion estimation process provides a more accurate representation of the motion within a scene, it becomes possible to more readily reconstruct information at the decoder which needs to be recovered before display. Besides quality, the major advantage of this method over other motion compensated interpolation methods is that significantly less computation is required on the decoder side.
Error Concealment. True motion vector information can also be employed to provide improved error concealment. In particular, post-processing operations at the decoder can be employed to recover damaged or lost video areas based on characteristics of images and video signals.
Interlaced-to-Progressive Scan Conversion. In addition to the above motion-compensated interpolation method, a related method for performing interlaced-to-progressive scan conversion also becomes available. In this scenario, rather than recovering an entire frame, an entire field is recovered. This type of conversion is necessary for cases in which a progressive display is intended to display compressed inter-frame, and motion-compensated inter-frame.
Intraframe methods. Simple intraframe techniques interpolate a missing line on the basis of two scanned lines which occur immediately before and after the missing line. One simple example is the “line averaging” which replaces a missing line by averaging the two lines adjacent to it. Some other improved intraframe methods which use more complicated filters or edge information have been proposed by M. H. Lee et al. (See “A New Algorithm for
Chen Yen-Kuang
Kung Sun-Yuan
Sun Huifang
Vetro Anthony
Brinkman Dirk
Kelley Chris S.
Mitsubishi Electric Research Laboratories Inc.
Vo Tung
LandOfFree
Motion compensated digital video signal processing does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Motion compensated digital video signal processing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Motion compensated digital video signal processing will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2571685