Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
1997-12-12
2001-01-23
Britton, Howard (Department: 2713)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
Reexamination Certificate
active
06178205
ABSTRACT:
REFERENCE TO MICROFICHE APPENDIX
The present specification comprises a microfiche appendix. The total number of microfiche sheets in the microfiche appendix is one. The total number of frames in the microfiche appendix is 49.
COPYRIGHT NOTICE
A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever.
BACKGROUND
1. Field of the Invention
This invention relates to systems for decoding video images and particularly to methods for improving decoded video image quality by removing coding artifacts and noise.
2. Description of Related Art
“Coding artifacts” are visible degradations in image quality that may appear as a result of encoding and then decoding a video image using a video compression method such as employed for the MPEG-1, MPEG-2, H.261, or H.263 standard. For example, video encoding for each of the MPEG-1, MPEG-2, H.261, and H.263 standards employs some combination of: partitioning frames of a video image into blocks; determining motion vectors for motion compensation of the blocks;
performing a frequency transform (e.g., a discrete cosine transform) on each block or motion-difference block; and quantizing the resultant transform coefficients. Upon decoding, common coding artifacts in a video image include blockiness that results from discontinuity of block-based motion compensation and inverse frequency transforms at block boundaries and “mosquito” noise surrounding objects in the video image as a result of quantization errors changing transform coefficients. Sources other than encoding and decoding can also introduce noise that degrades image quality. For example, transmission errors or noise in the system recording a video image can create random noise in the video image.
Postfiltering of a video image processes the video image to improve image quality by removing coding artifacts and noise. For example, spatial postfiltering can smooth the discontinuity at block boundaries and reduce the prominence of noise. Such spatial filtering operates on an array of pixel values representing a frame in the video image and modifies at least some pixel values based on neighboring pixel values. Spatial filtering can be applied uniformly or selectively to specific regions in a frame. For example, selective spatial filtering at a block edge (known locations within a frame) smoothes image contrast to reduce blockiness. However, spatial filtering can undesirably make edges and textures of objects in the image look fuzzy or indistinct and selective spatial filtering can cause “flashing” where the clarity of the edges of an object change as the object moves through areas filtered differently.
Temporal filtering operates on a current array of pixel values representing a current frame and combines pixel values from the current array with pixel values from one of more arrays representing prior or subsequent frames. Typically, temporal filtering combines a pixel value in the current array with pixel values in the same relative position in an array representing a prior frame under the assumption that the area remains visually similar. If noise or a coding artifact affects a pixel value in the current array but not the related pixel values in the prior frames, temporal encoding reduces the prominence of the noise or coding artifacts. A problem with temporal encoding arises from motion in the video image where the content of the image in one frame shifts in the next frame so that temporal filtering combines pixels in the current frame with visually dissimilar pixels in prior frames. When this occurs, the contribution of the dissimilar pixels creates a ghost of a prior frame in the current frame. Accordingly, temporal filtering can introduce undesired artifacts in a video image.
Postfiltering processes are sought that better remove coding artifacts and noise while preserving image features and not introducing further degradations.
SUMMARY
In accordance with the invention, a video postfiltering process includes motion compensated temporal filtering and/or spatial adaptive filtering. The motion compensated temporal filtering operates on each target pixel value in an array representing a current frame of a video image and combines each target pixel value with one or more pixel values from an array representing a prior frame. The pixel values from the prior frame alone or in combinations are sometimes referred to herein as reference values. The reference values for a target pixel in the current array are selected according to and depending on the values of a motion vector for a block containing the target pixel value and motion vectors for neighboring blocks. Using the motion vectors of neighboring blocks in the selection of reference values reduces ghosting when compared to temporal filtering without motion vectors or using only the motion vector for the block containing the target pixel.
In one embodiment of the invention, a vector (sometimes referred to herein as a filter vector) for a target pixel is determined from a weighted average of the motion vectors for the block containing the target pixel and the neighboring blocks closest to the target pixel. The weighting factors used in determining the filter vector for the target pixel depend on the position of the target pixel within a block. A pixel value for the target pixel is then filtered or combined with one or more reference values that correspond to an area of the prior frame identified by the filter vector.
An alternative embodiment of temporal filtering combines each target pixel value with pixel values from a prior frame that are in areas identified by the motion vectors for the block containing the target pixel value and neighboring blocks. The pixel values from the prior frame may be combined in a weighted average using weighting factors selected according to the position of the target pixel value within a block.
The adaptive spatial filter selects a filter operation for a target pixel according to the level of coding artifacts and the presence of important features. The level of coding artifacts depends on how well the pixel values are coded as indicated by the quantization factor. The dynamic range of the smallest coding unit, a 8×8 block in most of the standardized encoding processes, is used to estimate the amount of coding noise in the block. A large dynamic range usually indicates more noise. To reduce blurring of image features, a second dynamic range around the target pixel is computed and used in two ways. The second dynamic range indicates the shape of the filter required to avoid mixing pixels from different features together. The second dynamic range also indicates the appropriate strength of the filter. When the second dynamic range is close to the first dynamic range, the target pixel is on or near image features, and a weak filter is used. When the second dynamic range is smaller than a large first dynamic range, the target pixel is likely to be noise around the edges and a strong filter is used. Other combinations of the sizes of the dynamic ranges result in the use of other filters.
Although the temporal filtering and spatial filtering are used in combination to provide the best image quality, either may be used alone in particular embodiments of the invention.
REFERENCES:
patent: 5475434 (1995-12-01), Kim
patent: 5481628 (1996-01-01), Ghaderi
patent: 5502489 (1996-03-01), Kim et al.
patent: 5539469 (1996-07-01), Jung
patent: 5598213 (1997-01-01), Chung et al.
patent: 5610729 (1997-03-01), Nakajima
patent: 5621468 (1997-04-01), Kim
patent: 5654759 (1997-08-01), Augenbraun et al.
patent: 5742344 (1998-04-01), Odaka et al.
patent: 5793435 (1998-08-01), Ward et al.
patent: 5907370 (1999-05-01), Suzuki et al.
Liu, et al., “Adaptive Postprocessing Algorithms for Low Bit Rate Video Signals”,IEEE Transactions on Im
Cheung Sen-ching S.
Drizen David
Haskell Paul E.
Britton Howard
Millers David T.
Philippe Gims
Skjerven Morrill & MacPherson LLP
Vtel Corporation
LandOfFree
Video postfiltering with motion-compensated temporal... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Video postfiltering with motion-compensated temporal..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Video postfiltering with motion-compensated temporal... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2541441