Image analysis – Image enhancement or restoration – Image filter
Reexamination Certificate
2000-07-06
2002-03-19
Lee, Thomas D. (Department: 2724)
Image analysis
Image enhancement or restoration
Image filter
C358S001150
Reexamination Certificate
active
06360024
ABSTRACT:
BACKGROUND TO THE INVENTION
1. Field of the Invention
The present invention relates to a method and apparatus for removing noise in still and moving pictures, and the invention can be used in any still or moving picture compression or coding algorithm where the reconstructed picture suffers from block distortion noise commonly known as blocky noise. This method selectively filters the image such that the picture is restored and the blocky noise as well as ring and mosquito noise are removed.
2. Description of the Related Art
Many algorithms and standards exist for compression of still and moving pictures. These include standards such as the ITU-T H.261 and H.263, and the ISO standards for JPEG, MPEG-1 and MPEG-2. More recently, work has begun on the MPEG-4 video coding standard. All of the algorithms mentioned use the transform coding technique, in particular the Discrete Cosine Transform, DCT. In this technique the picture is first partitioned into small blocks of 8 pixels high and 8 pixels wide. These blocks of pixels are then transformed into the DCT domain where the transform coefficients are subjected to quantization to reduce the amount of information. At high compression ratios, most of these coefficients are quantized to zero. The quantization noise introduced during this process results in visual artifacts when the blocks of coefficients are transformed back into the pixel domain.
This artifact is a result of loss in the high frequency components of the transform coefficients. Since these coefficients are reduced to zero, the inverse quantization process cannot faithfully reproduce the original signal. The result is that the pixels at the edge of two adjacent DCT blocks have very different values. This difference in values makes the picture appear to be made up of blocks. This visual artifact is known as a blocky noise artifact.
FIGS. 1A and 1B
graphically depict how the blocky noise is created by coarse quantization.
FIG. 1A
shows original pixel values in which pixels P
0
, P
1
, P
2
, P
3
, P
4
, P
5
, P
6
and P
7
have gradually increasing values. Note that a line between pixels P
3
and P
4
represent a boundary between the blocks for the DCT processing. Thus, the pixels P
0
, P
1
, P
2
, and P
3
are processed in one DCT block, and pixels P
4
, P
5
, P
6
and P
7
are processed in the next neighboring DCT block. After DCT processing, the DCT coefficients are quantized. During the quantization, some data are lost. Then, after inverse DCT processing, the pixels P
0
, P
1
, P
2
, and P
3
in the one DCT block will take different values from their original values, as shown in FIG.
1
B. Similarly, the pixels P
4
, P
5
, P
6
and P
7
in the neighboring DCT block will take different values from their original values. In
FIG. 1B
, arrows added to the points representing the pixel values show the changes between before and after the DCT processing, quantization processing, and inverse DCT processing. After the inverse DCT processing, as shown in
FIG. 1B
, there will be a step between the pixel values P
3
and P
4
, resulting in a loss of a gradual change of the pixel value at the boundary between the blocks for the DCT processing. Thus, the reproduced image will be a mosaic effect added picture.
FIGS. 2A and 2B
show how the loss of high frequency coefficients in the DCT domain results in ring and mosquito noise artifacts. Note that in
FIG. 2A
, a dotted line between pixels P
2
and P
3
represent a discontinuity of the picture, such as an edge line of a door in the picture. After the DCT processing, quantization processing, and inverse DCT processing, the pixel values P
2
and P
3
will be further separated, as shown in
FIG. 2B
, resulting in a so-called, ring and mosquito noise providing undesirably emphasized edge lines in the picture.
The main problem to be solved is how to devise a general algorithm that can be applied to the picture with the artifact such that the blocky noise as well as ring and mosquito noise are reduced or removed. The goal is to manipulate the picture in such a way that the recovered picture is visually as close to the original picture as possible.
The second problem to be solved is how to prevent the blocky noise as well as ring and mosquito noise from propagating to the next frame in a sequence of moving pictures where motion compensation is used.
SUMMARY OF THE INVENTION
In order to solve the above problem, an integrated filtering technique is used. There are many filtering techniques currently being used for removing noise from pictures. What is novel about this invention is the method of adaptively deciding the pixels that are filtered and those that are not filtered as well as the values that are adaptively reset and used in the non-linear filter depending on the decisions.
To solve the propagation problem, the filter is placed within the motion compensation loop prior to the frame store so that the filtered picture is used in the motion compensation instead of the picture with artifacts. Though it is common to use a filter in the prediction loop to improve the motion compensation performance, the novelty of this invention is the use of this particular deblocking filter in the prediction loop to prevent the propagation of blocky artifacts. In prior art such as the H.261, the loop filter serves no other purpose except for improving the motion compensation prediction. In this invention the filter is not intended for the improvement of the motion compensation performance but rather for the simultaneous task of removing the blocky artifact in the current picture and also for preventing the propagation of blocky artifact into the next picture.
The current invention operates on the reconstructed still picture that is decoded from the compressed bitstream, as shown in FIG.
3
. In the case of moving pictures it can operate outside the prediction loop as a post filter, shown in
FIG. 4
, or within the prediction loop as a loop filter, shown in FIG.
5
. In all these cases the filter used is the same. It operates on the boundary of the blocks where blocky noise manifests.
The blocky noise removal filter is a separable filter and can be applied in the horizontal and vertical directions in tandem. The filter used is a modified version of the bi-linear interpolation filter. Two values of the end points are determined from the pixels on either side of the block boundary. The filtered values on either side of the filter represent the bi-linearly interpolated values of the two endpoints modified by a scaled value determined from the deviation of the actual pixel from the end points.
The ring and mosquito noise filter can be in one of two forms. The first is a block based filter where a window of pixel is conditionally averaged to remove isolated noise. This two dimensional filter can also be implemented in one dimension along the horizontal or vertical direction where the window is only one dimension along the direction of the filtering. This filter also removes the noise by conditionally averaging the pixel around the center of the window.
REFERENCES:
patent: 5229864 (1993-07-01), Moronaga et al.
patent: 5253083 (1993-10-01), Hirota
patent: 5537150 (1996-07-01), Sugiyama et al.
patent: 5563662 (1996-10-01), Kishi
patent: 5610729 (1997-03-01), Nakajima
patent: 5933541 (1999-08-01), Kutka et al.
patent: 5974197 (1999-10-01), Lee et al.
patent: 6088489 (2000-07-01), Miyake
patent: 6188799 (2001-02-01), Tan et al.
Shen Sheng Mei
Tan Thiow Keng
Brimich Stephen
Lee Thomas D.
Wenderoth , Lind & Ponack, L.L.P.
LandOfFree
Method and apparatus for removing noise in still and moving... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for removing noise in still and moving..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for removing noise in still and moving... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2890956