Television – Camera – system and detail – Combined image signal generator and general image signal...
Reexamination Certificate
2001-02-16
2004-11-09
Rao, Andy (Department: 2613)
Television
Camera, system and detail
Combined image signal generator and general image signal...
C375S240120, C375S240250
Reexamination Certificate
active
06816194
ABSTRACT:
TECHNICAL FIELD
This invention relates to systems and methods for coding video data, and more particularly, to motion-compensation-based video coding schemes that employ error resilience techniques in the enhancement layer bitstream.
BACKGROUND
Efficient and reliable delivery of video data is increasingly important as the Internet and wireless channel networks continue to grow in popularity. Video is very appealing because it offers a much richer user experience than static images and text. It is more interesting, for example, to watch a video clip of a winning touchdown or a Presidential speech than it is to read about the event in stark print. Unfortunately, video data is significantly larger than other data types commonly delivered over the Internet. As an example, one second of uncompressed video data may consume one or more Megabytes of data.
Delivering such large amounts of data over error-prone networks, such as the Internet and wireless networks, presents difficult challenges in terms of both efficiency and reliability. These challenges arise as a result of inherent causes such as bandwidth fluctuations, packet losses, and channel errors. For most Internet applications, packet loss is a key factor that affects the decoded visual quality. For wireless applications, wireless channels are typically noisy and suffer from a number of channel degradations, such as random errors and burst errors, due to fading and multiple path reflections. Although the Internet and wireless channels have different properties of degradations, the harms are the same to the video bitstream. One or multiple video packet losses may cause some consecutive macroblocks and frames to be undecodable.
To promote efficient delivery, video data is typically encoded prior to delivery to reduce the amount of data actually being transferred over the network. Image quality is lost as a result of the compression, but such loss is generally tolerated as necessary to achieve acceptable transfer speeds. In some cases, the loss of quality may not even be detectable to the viewer.
Video compression is well known. One common type of video compression is a motion-compensation-based video coding scheme, which is used in such coding standards as MPEG-1, MPEG-2, MPEG-4, H.261, and H.263.
One particular type of motion-compensation-based video coding scheme is a layer-based coding schemed, such as fine-granularity layered coding. Layered coding is a family of signal representation techniques in which the source information is partitioned into sets called “layers”. The layers are organized so that the lowest, or “base layer”, contains the minimum information for intelligibility. The other layers, called “enhancement layers”, contain additional information that incrementally improves the overall quality of the video. With layered coding, lower layers of video data are often used to predict one or more higher layers of video data.
The quality at which digital video data can be served over a network varies widely depending upon many factors, including the coding process and transmission bandwidth. “Quality of Service”, or simply “QoS”, is the moniker used to generally describe the various quality levels at which video can be delivered. Layered video coding schemes offer a wide range of QoSs that enable applications to adopt to different video qualities. For example, applications designed to handle video data sent over the Internet (e.g., multi-party video conferencing) must adapt quickly to continuously changing data rates inherent in routing data over many heterogeneous sub-networks that form the Internet. The QoS of video at each receiver must be dynamically adapted to whatever the current available bandwidth happens to be. Layered video coding is an efficient approach to this problem because it encodes a single representation of the video source to several layers that can be decoded and presented at a range of quality levels.
Apart from coding efficiency, another concern for layered coding techniques is reliability. In layered coding schemes, a hierarchical dependence exists for each of the layers. A higher layer can typically be decoded only when all of the data for lower layers or the same layer in the previous prediction frame is present. If information at a layer is missing, any data for the same or higher layers is useless. In network applications, this dependency makes the layered encoding schemes very intolerant of packet loss, especially at the lower layers. If the loss rate is high in layered streams, the video quality at the receiver is very poor.
FIG. 1
depicts a conventional layered coding scheme
100
, known as “fine-granularity scalable” or “FGS”. Three frames are shown, including a first or intraframe
102
followed by two predicted frames
104
and
106
that are predicted from the intraframe
102
. The frames are encoded into four layers: a base layer
108
, a first layer
110
, a second layer
112
, and a third layer
114
. The base layer
108
typically contains the video data that, when played, is minimally acceptable to a viewer. Each additional layer
110
-
114
, also known as “enhancement layers”, contains incrementally more components of the video data to enhance the base layer. The quality of video thereby improves with each additional enhancement layer. This technique is described in more detail in an article by Weiping Li, entitled “Fine Granularity Scalability Using Bit-Plane Coding of DCT Coefficients”, ISO/IEC JTC1/SC29/WG11, MPEG98/M4204 (December 1998).
One characteristic of the FGS coding scheme illustrated in
FIG. 1
is that the enhancement layers
110
-
114
in the predicted frames can be predictively coded from the base layer
108
in a preceding reference frame. In this example, the enhancement layers of predicted frame
104
can be predicted from the base layer of intraframe
102
. Similarly, the enhancement layers of predicted frame
106
can be predicted from the base layer of preceding predicted frame
104
.
With layered coding, the various layers can be sent over the network as separate sub-streams, where the quality level of the video increases as each sub-stream is received and decoded. The base layer
108
is sent as one bitstream and one or more enhancement layers
110
-
114
are sent as one or more other bitstreams.
FIG. 2
illustrates the two bitstreams: a base layer bitstream
200
containing the base layer
108
and an enhancement layer bitstream
202
containing the enhancement layers
110
-
114
. Generally, the base layer is very sensitive to any packet losses and errors and hence, any errors in the base bitstream
200
may cause a decoder to lose synchronization and propagate errors. Accordingly, the base layer bitstream
200
is transmitted in a well-controlled channel to minimize error or packet-loss. The base layer is encoded to fit in the minimum channel bandwidth and is typically protected using error protection techniques, such as FEC (Forward Error Correction) techniques. The goal is to deliver and decode at least the base layer
108
to provide minimal quality video.
Research has been done on how to integrate error protection and error recovery capabilities into the base layer syntax. For more information on such research, the reader is directed to R.Talluri, “Error-resilient video coding in the ISO MPEG-4 standard”, IEEE communications Magazine, pp112-119, June, 1998; and Y. Wang, Q. F. Zhu, “Error control and concealment for video communication: A review”, Proceeding of the IEEE, vol. 86, no. 5, pp 974-997, May, 1998.
The enhancement layer bitstream
202
is delivered and decoded, as network conditions allow, to improve the video quality (e.g., display size, resolution, frame rate, etc.). In addition, a decoder can be configured to choose and decode a particular portion or subset of these layers to get a particular quality according to its preference and capability.
The enhancement layer bitstream
202
is normally very robust to packet losses and/or errors. The enhancement layers in the FGS coding scheme provide an example of such robustness. The bitstream
Li Shipeng
Wu Feng
Yan Rong
Zhang Ya-Qin
Lee & Hayes PLLC
Microsoft Corporation
Rao Andy
LandOfFree
Systems and methods with error resilience in enhancement... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Systems and methods with error resilience in enhancement..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Systems and methods with error resilience in enhancement... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3302119