Image analysis – Image compression or coding – Predictive coding
Reexamination Certificate
1999-12-03
2003-09-02
Johnson, Timothy M. (Department: 2623)
Image analysis
Image compression or coding
Predictive coding
C382S240000, C375S240100, C375S240140
Reexamination Certificate
active
06614936
ABSTRACT:
TECHNICAL FIELD
This invention relates to systems and methods for coding video data, and more particularly, to motion-compensation-based video coding schemes that employ fine-granularity layered coding.
BACKGROUND
Efficient and reliable delivery of video data is becoming increasingly important as the Internet continues to grow in popularity. Video is very appealing because it offers a much richer user experience than static images and text. It is more interesting, for example, to watch a video clip of a winning touchdown or a Presidential speech than it is to read about the event in stark print. Unfortunately, video data is significantly larger than other data types commonly delivered over the Internet. As an example, one second of uncompressed video data may consume one or more Megabytes of data. Delivering such large amounts of data over error-prone networks, such as the Internet and wireless networks, presents difficult challenges in terms of both efficiency and reliability.
To promote efficient delivery, video data is typically encoded prior to delivery to reduce the amount of data actually being transferred over the network. Image quality is lost as a result of the compression, but such loss is generally tolerated as necessary to achieve acceptable transfer speeds. In some cases, the loss of quality may not even be detectable to the viewer.
Video compression is well known. One common type of video compression is a motion-compensation-based video coding scheme, which is used in such coding standards as MPEG-1, MPEG-2, MPEG-4, H.261, and H.263.
One particular type of motion-compensation-based video coding scheme is fine-granularity layered coding. Layered coding is a family of signal representation techniques in which the source information is partitioned into a sets called “layers”. The layers are organized so that the lowest, or “base layer”, contains the minimum information for intelligibility. The other layers, called “enhancement layers”, contain additional information that incrementally improves the overall quality of the video. With layered coding, lower layers of video data are often used to predict one or more higher layers of video data.
The quality at which digital video data can be served over a network varies to widely depending upon many factors, including the coding process and transmission bandwidth. “Quality of Service”, or simply “QoS”, is the moniker used to generally describe the various quality levels at which video can be delivered. Layered video coding schemes offer a range of QoSs that enable applications to adopt to different video qualities. For example, applications designed to handle video data sent over the Internet (e.g., multi-party video conferencing) must adapt quickly to continuously changing data rates inherent in routing data over many heterogeneous sub-networks that form the Internet. The QoS of video at each receiver must be dynamically adapted to whatever the current available bandwidth happens to be. Layered video coding is an efficient approach to this problem because it encodes a single representation of the video source to several layers that can be decoded and presented at a range of quality levels.
Apart from coding efficiency, another concern for layered coding techniques is reliability. In layered coding schemes, a hierarchical dependence exists for each of the layers. A higher layer can typically be decoded only when all of the data for lower layers is present. If information at a layer is missing, any data for higher layers is useless. In network applications, this dependency makes the layered encoding schemes very intolerant of packet loss, especially at the lowest layers. If the loss rate is high in layered streams, the video quality at the receiver is very poor.
FIG. 1
depicts a conventional layered coding scheme
20
, known as “fine-granularity scalable” or “FGS”. Three frames are shown, including a first or intraframe
22
followed by two predicted frames
24
and
26
that are predicted from the intraframe
22
. The frames are encoded into four layers: a base layer
28
, a first layer
30
, a second layer
32
, and a third layer
34
. The base layer typically contains the video data that, when played, is minimally acceptable to a viewer. Each additional layer contains incrementally more components of the video data to enhance the base layer. The quality of video thereby improves with each additional layer. This technique is described in more detail in an article by Weiping Li, entitled “Fine Granularity Scalability Using Bit-Plane Coding of DCT Coefficients”, ISO/IEC JTC1/SC 29/WG11, MPEG98/M4204 (December 1998).
With layered coding, the various layers can be sent over the network as separate sub-streams, where the quality level of the video increases as each sub-stream is received and decoded. The base-layer video
28
is transmitted in a well-controlled channel to minimize error or packet-loss. In other words, the base layer is encoded to fit in the minimum channel bandwidth. The goal is to deliver and decode at least the base layer
28
to provide minimal quality video. The enhancement
30
-
34
layers are delivered and decoded as network conditions allow to improve the video quality (e.g., display size, resolution, frame rate, etc.). In addition, a decoder can be configured to choose and decode a particular subset of these layers to get a particular quality according to its preference and capability.
One characteristic of the illustrated FGS coding scheme is that the enhancement layers
30
-
34
are coded from the base layer
28
in the reference frames. As shown in
FIG. 1
, each of the enhancement layers
30
-
34
in the predicted frames
24
and
26
can be predicted from the base layer of the preceding frame. In this example, the enhancement layers of predicted frame
24
can be predicted from the base layer of intraframe
22
. Similarly, the enhancement layers of predicted frame
26
can be predicted from the base layer of preceding predicted frame
24
.
The FGS coding scheme provides good reliability in terms of error recovery from occasional data loss. By predicting all enhancement layers from the base layer, loss or corruption of one or more enhancement layers during transmission can be remedied by reconstructing the enhancement layers from the base layer. For instance, suppose that frame
24
experiences some error during transmission. In this case, the base layer
28
of preceding intraframe
22
can be used to predict the base layer and enhancement layers of frame
24
.
Unfortunately, the FGS coding scheme has a significant drawback in that the scheme is very inefficient from a coding standpoint since the prediction is always based on the lowest quality base layer. Accordingly, there remains a need for a layered coding scheme that is efficient without sacrificing error recovery.
FIG. 2
depicts another conventional layered coding scheme
40
in which three frames are encoded using a technique introduced in an article by James Macnicol, Michael Frater and John Arnold, which is entitled, “Results on Fine Granularity Scalability”, ISO/IEC JTC1/SC29/WG11,MPEG99/m5122 (October 1999). The three frames include a first frame
42
, followed by two predicted frames
44
and
46
that are predicted from the first frame
42
. The frames are encoded into four layers: a base layer
48
, a first layer
50
, a second layer
52
, and a third layer
54
. In this scheme, each layer in a frame is predicted from the same layer of the previous frame. For instance, the enhancement layers of predicted frame
44
can be predicted from the corresponding layer of previous frame
42
. Similarly, the enhancement layers of predicted frame
46
can be predicted from the corresponding layer of previous frame
44
.
The coding scheme illustrated in
FIG. 2
has the advantage of being very efficient from a coding perspective. However, it suffers from a serious drawback in that it cannot easily recover from data loss. Once there is an error or packet loss in the enhancement layers, it propagates to the end of a GOP (group of predicted fra
Li Shipeng
Wu Feng
Zhang Ya-Qin
Johnson Timothy M.
Lee & Hayes PLLC
Microsoft Corporation
LandOfFree
System and method for robust video coding using progressive... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for robust video coding using progressive..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for robust video coding using progressive... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3030275