Image analysis – Image compression or coding – Pyramid – hierarchy – or tree structure
Reexamination Certificate
1999-09-07
2002-10-01
Au, Amelia M. (Department: 2721)
Image analysis
Image compression or coding
Pyramid, hierarchy, or tree structure
C382S243000, C375S240110, C375S240210, C348S396100
Reexamination Certificate
active
06459814
ABSTRACT:
The invention relates to an apparatus and a concomitant method for shape or object mask coding. More particularly, the invention relates to a method for increasing the efficiency of scalable shape coding by deriving shape information for chrominance components from luminance component.
BACKGROUND OF THE INVENTION
Shape or object mask encoding (also known as object-oriented image and video coding) has gained acceptance and is presently being promoted to various multimedia standards, e.g., the MPEG-4 (Moving Picture Experts Group) international standard. However, unlike traditional frame encoding methods, each frame or picture is considered as consisting of one or more flexible objects (objects having arbitrary shapes), that may undergo changes such as translations, rotations, scaling, brightness and color variations and the like. Using shape or object mask encoding, functionalities are not only provided at the frame level, but also at the object level.
One functionality is scalability, i.e., providing an image at different spatial resolutions. In general, shape encoding starts by segmenting an image frame into a plurality of objects or video object planes (VOPs), e.g., a speaker in an image is one object and the background is a second object. The resulting “shape information” can be represented as a “binary mask”. The binary shape information provides object mask with only two values: transparent or opaque (where transparent means the corresponding pixel is outside of an object and opaque means the corresponding pixel is within the object). A mask can be broadly defined as the information that defines the shape of an object or pixels associated with an object. More specifically, the shape information or object mask is used to indicate the arbitrary shape of an image or video object and the region in which the texture of this object needs to be coded. Since the object mask is tracked and encoded along with the texture information into the bitstream, it is possible to provide various functionalities based on object. An example of a novel shape or object mask coding is disclosed in the patent application entitled “Method And Apparatus For Generic Scalable Shape Coding” filed on May 17, 1999 with Ser. No. 09/312,797, which is commonly owned by the assignee and is incorporated herein by reference.
Additionally, subband based (e.g., wavelet based coding as discussed below) coding is also supported by the MPEG-4 standard. One of the advantages of wavelet based coding scheme is that it can provide flexible spatial scalability. Thus, the coding standard should support both spatial scalability of texture coding and spatial scalability of shape coding.
Unfortunately, due to the nature of shape-adaptive wavelet transform, the wavelet transforms of texture information within a region are based on the shape information of that region. Thus, in order to perfectly reconstruct the texture information at the decoder end, the shape of the region to be used in the decoder end for inverse wavelet transform must be exactly the same as the shape used in the encoder end for wavelet transform. This is true for each spatial layer for the luminance and chrominance components. Unfortunately, this leads to the requirement that both luminance and chrominance shape information are to be encoded into the bitstream, thereby reducing coding efficiency.
It should be noted that if no spatial scalability of shape coding is required at the decoder end, then one can always start from the full resolution of the shape information for luminance and then derive from that the exact shape for chrominance at whatever spatial level according to the same decomposition order of the encoder. However, when the scalability of shape coding is required, it is desirable to be able to derive the exact shape of the chrominance information from the shape of the luminance at the same spatial level, without having to start from the full resolution of the luminance shape information, which may not be available or may not be computationally practical for a particular application.
Therefore , a need exists in the art for a generic spatially-scalable shape encoding method and apparatus that is capable of deriving the exact shape of the chrominance components from the shape of the luminance component for each spatial layer.
SUMMARY OF THE INVENTION
In the present invention, an embodiment of a generic spatially-scalable shape encoding apparatus and concomitant method for deriving the exact shape of the chrominance components from the shape of the luminance component for each spatial layer, is disclosed. The present generic spatially-scalable shape encoding applies a series of subband (e.g., wavelet) filters, T
1
, T
2
, . . . TN, to obtain N-levels of wavelet decomposition for both texture and shape information of both luminance and chrominance components.
More specifically, a series of subband filters, T
1
, T
2
, . . . TN, are applied to an input image to obtain N-levels of subband decomposition for both texture and shape information of the luminance component. It should be noted that subband filtering comprises two distinct processing steps: spatial filtering and subsampling. In the preferred embodiment, the corresponding subsampling function (T
1
) of said subband filter T
1
is applied to the full resolution luminance shape information or mask to obtain a full resolution chrominance shape information. In turn, the corresponding series of subsampling filters ((T
2
), . . . (TN)) can be applied to the full resolution chrominance shape information to obtain additional levels of chrominance shape information. Next, the corresponding series of subband filters, T
2
, . . . TN, and said plurality of levels of chrominance shape information are applied to the full resolution chrominance texture information to obtain a plurality of levels of chrominance texture information.
Finally, the plurality of levels of resolution of luminance texture and shape information are encoded into the bitstream. However, only the plurality of levels of resolution of chrominance texture information are encoded into the bitstream. Namely, coding efficiency is enhanced by not having to encode the plurality of levels of resolution of chrominance shape information into the bitstream, while still providing the desirable feature of scalability.
Additionally, the present invention also discloses a novel method of interpolating a missing chrominance component corresponding to a luminance pixel within an object. Namely, the missing chrominance component is interpolated by taking the average of its three neighboring values in the chrominance plane.
REFERENCES:
patent: 5623310 (1997-04-01), Kim
patent: 5822460 (1998-10-01), Kim
patent: 5982434 (1999-11-01), Tong et al.
patent: 6351492 (2002-02-01), Kim
patent: 0 261 561 (1988-03-01), None
patent: 0 909 096 (1999-04-01), None
Xiong et al. “Multiresolutional encoding and decoding in embedded image and video coders”. Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing. May 12-15, 1998. vol. 6, p. 3709-3712.*
Martucci et al. “A zerotree wavelet video coder”. IEEE Transactions on Circuits and Systems for Video Technology. Feb. 1997. vol. 7, Iss. 1, p. 109-118.*
Ostermann et al. “Coding of arbitrarily shaped video objects in MPEG-4”. Proceedings, International Conference on Image Processing. Oct. 26-29, 1997 vol. 1, p. 496-499.*
Haridasan et al. “Scalable coding of video objects”. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems. May 31-Jun. 3, 1998. vol. 4, p. 289-292.
Lee Hung-Ju
Li Shipeng
Sodagar Iraj
Au Amelia M.
Burke William J.
LaRose Colin
Sarnoff Corporation
LandOfFree
Method and apparatus for generic scalable shape coding by... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for generic scalable shape coding by..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for generic scalable shape coding by... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2928184