Image analysis – Image compression or coding – Shape – icon – or feature-based compression
Reexamination Certificate
2001-03-22
2003-02-25
Mehta, Bhavesh (Department: 2625)
Image analysis
Image compression or coding
Shape, icon, or feature-based compression
C382S232000, C382S236000, C382S238000, C382S248000, C382S240000, C348S408100, C375S240110
Reexamination Certificate
active
06526177
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a video coding system in which image data is organized into video objects and coded according to a scalable coding scheme. The coding scheme provides spatial scalability, temporal scalability or both.
2. Related Art
Video coding is a field that currently exhibits dynamic change. Video coding generally relates to any method that represents natural and/or synthetic visual information in an efficient manner. A variety of video coding standards currently are established and a number of other coding standards are being drafted. The present invention relates to an invention originally proposed for use in the Motion Pictures Experts Group standard MPEG-4.
One earlier video standard, known as “MPEG-2,” codes video information as video pictures or “frames.” Consider a sequence of video information to be coded, the sequence represented by a series of frames. The MPEG-2 standard coded each frame according to one of three coding methods. A given image could be coded according to:
Intra-coding where the frame was coded without reference to any other frame (known as “I-pictures”),
Predictive-coding where the frame was coded with reference to one previously coded frame (known as “P-pictures”), or
Bi-directionally predictive coding where the frame was coded with reference to as many as two previously coded frames (known as “B-pictures”).
Frames are not necessarily coded in the order in which they appear under MPEG-2. It is possible to code a first frame as an I-picture then code a fourth frame as a P-picture predicted from the I-picture. Second and third frames may be coded as B-pictures, each predicted with reference to the I- and P-pictures previously coded. A time index is provided to permit a decoder to reassemble the correct frame sequence when it decodes coded data.
MPEG-4, currently being drafted, integrated the concept of “video objects” to I-, P- and B-coding. Video object based coders decompose a video sequence into video objects.
An example is provided in FIGS.
1
(
a
)-(
d
). There, a frame includes image data including the head and shoulders of a narrator, a suspended logo and a background. An encoder may determine that the narrator, logo and background are three distinct video objects, each shown separately in FIGS.
1
(
b
)-(
d
). The video coder may code each separately.
Video object-based coding schemes recognize that video objects may remain in a video sequence across many frames. The appearance of a video object on any given frame is a “video object plane” or “VOP”. VOPs may be coded as I-VOPs using intra coding techniques, as P-VOPs using predictive coding techniques or B-VOPs using bi-directionally predictive coding techniques. For each VOP, additional administrative data is transmitted with the coded VOP data that provides information regarding, for example, the video objects location in the displayed image.
Coding video information on a video object-basis may improve coding efficiency in certain applications. For example, if the logo were a static image, an encoder may code it as an initial I-VOP. However, for subsequent frames, coding the logo as a P- or B-VOP would yield almost no image data. The P- or B-coding essentially amounts to an “instruction” that the original image information should be redisplayed for successive frames. Such coding provides improved coding efficiency.
One goal of the MPEG-4 standard is to provide a coding scheme that may be used with decoders of various processing power. Simple decoders should be able to decode coded video data for display. More powerful decoders should be able to decode the coded video data and obtain superior output such as improved image quality or attached functionalities. As of the priority date of this application, no known video object-based coding scheme provides such flexibility.
MPEG-2 provides scalability for its video picture-based coder. However, the scalability protocol defined by MPEG-2 is tremendously complicated. Coding of spatial scalability, where additional data for VOPs is coded into an optional enhancement layer, is coded using a first protocol. Coding of temporal scalability, where data of additional VOPs is coded in the enhancement layer, is coded using a second protocol. Each protocol is separately defined from the other and requires highly context specific analysis and complicated lookup tables in a decoder. The scalability protocol of the MPEG-2 is disadvantageous because its complexity makes it difficult to implement. Accordingly, there is a further need in the art for a generalized scalability protocol.
SUMMARY OF THE INVENTION
The present invention provides a video coding system that codes video objects as video object layers. Data of each video object may be segregated into one or more layers. A base layer contains sufficient information to decode a basic representation of the video object. Enhancement layers contain supplementary data regarding the video object that, if decoded, enhance the basic representation obtained from the base layer. The present invention thus provides a coding scheme suitable for use with decoders of varying processing power. A simple decoder may decode only the base layer to obtain the basic representation. However, more powerful decoders may decode the base layer data and additional enhancement layer data to obtain improved decoded output.
REFERENCES:
patent: 4958226 (1990-09-01), Haskell et al.
patent: 5408328 (1995-04-01), Boliek et al.
patent: 5414469 (1995-05-01), Gonzales et al.
patent: 5465119 (1995-11-01), Demos
patent: 5467136 (1995-11-01), Odaka et al.
patent: 5488418 (1996-01-01), Mishima et al.
patent: 5532747 (1996-07-01), Yoon et al.
patent: 5539468 (1996-07-01), Suzuki et al.
patent: 5612735 (1997-03-01), Haskell et al.
patent: 5619256 (1997-04-01), Haskell et al.
patent: 5633684 (1997-05-01), Teranishi et al.
patent: 5699117 (1997-12-01), Uramoto et al.
patent: 5757971 (1998-05-01), Kim
patent: 5778097 (1998-07-01), Nickerson
patent: 5825421 (1998-10-01), Tan
patent: 5978509 (1999-11-01), Nachtergaele et al.
patent: 5999189 (1999-12-01), Kajiya et al.
patent: 6005623 (1999-12-01), Takahashi et al.
patent: 6005980 (1999-12-01), Eifrig et al.
patent: 6026183 (2000-02-01), Talluri et al.
patent: 6043846 (2000-03-01), Shen et al.
patent: 6057884 (2000-05-01), Chen et al.
patent: 6144701 (2000-11-01), Chiang et al.
patent: 6148026 (2000-11-01), Puri et al.
A Puri et al. (Working Draft 1.0 of ISO/IEC 14496-2), Nov. 1996, version of: Jun. 12, 2001.*
Index of Web Search.
Web site material, “Object-Based Coding,” http://www.ece.concordia.ca/~sorial/seminar 1
ode18.html, (Jul. 14, 1999).
Web site material, “MPEG-4 Video and Its Potential for Future Multimedia Services,” http://wwwam.hhi.de/mpeg-video/papers/sikora/iscas.htm (Jan. 27, 1997).
Web site material, “The Structure of the MPEG-4 Video Coding Algorithm”, http://www.ee.iisc.ernet.in/people/students/gupta/Mpeg4/Coding/fmpeg4vm.htm (Mar. 1, 1998).
Web site material, “MPEG-4: A Flexible Coding Standard for the Emerging Mobile Multimedia Applications”, http://www.tnt.uni-hannover.de/project/eu/momusys/docs/Paper/IST/pimrc/pimrc.html (Oct. 9, 2000).
Haskell, Barry G., et al., “MPEG-4 and the Future”, Digital Video: An Introduction to MPEG-2, pp. 369-371 (1997).
International Organization for Standardisation; Generic Coding of Moving Pictures and Associated Audio Information: Video; Recommendation H.262; ISO/IEC 13818-2 JTC1/SC29/WG11 N0702(revised) Incorporationg N702 Delta of March 24 and Further Editorial Corrections May 10, 1994.
International Organization for Standardisation; MPEG-4 Video Verification Model Vesion 2.1; ISO/IEC JTC1/SC29/WG11 XXXX of May 3, 1996.
International Organization for Standardisation; Working Draft 1.0 of ISO/IEC 14496-2; ISO/IEC JTC1/SC29/WG11 N1470; MPEG/Maceio, Nov. 1996.
International Organization for Standardisation; Working Draft 4.0 of ISO/IEC 14496-2; ISO/IEC JTC1/SC29/WG11 N1797; MPEG97/Stockholm, Jul. 1997.
Organisation Internationale De Normalisation; MPEG-4 Proposal Package Descri
Haskell Barin Geoffry
Puri Atul
Schmidt Robert Lewis
AT&T Corp.
Bayat Ali
Kenyon & Kenyon
Mehta Bhavesh
LandOfFree
Generalized scalability for video coder based on video objects does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Generalized scalability for video coder based on video objects, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Generalized scalability for video coder based on video objects will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3149687