Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
2000-08-10
2002-07-16
Britton, Howard (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C375S240120, C715S252000, C715S252000
Reexamination Certificate
active
06421387
ABSTRACT:
RELATED ART
Transmitting high-quality, real-time interactive video over lossy networks, such as the Internet and wireless networks, is very challenging. Because of limited bandwidth on networks and the bandwidth-intensive nature of video, video transmission requires extremely high compression efficiency. However, state-of-the-art compression standards (MPEG, H.261) are not designed for transmission over a lossy channel. Although they can achieve very impressive compression efficiency, even small data losses can severely degrade video quality. A few bit errors in encoded data can cause the decoder to lose synchronization in the encoded stream and can render useless all the data received until the next synchronization point. Furthermore, motion estimation and compensation in these codecs pose an even more severe problem, namely, error propagation (or error spread). Motion estimation removes temporal redundancy in successive video frames (inter frames) by encoding only pixel value differences (prediction error) between a currently encoded image and a motion-predicted image created from a previously encoded image (reference frame). Image distortion in a reference frame can propagate to its succeeding frames and becomes amplified as more bits are lost.
Conventional work on loss recovery focuses on repairing packet losses before the scheduled display times of those video frames contained in lost packets. However, this approach is ineffective for interactive video because data losses inevitably occur in packet-switched communication, and detecting and repairing losses causes latency. To handle this latency, existing techniques introduce additional delays in frame display times. However, delaying frame playout times greatly impairs the interactiveness of video communication.
Many researchers have proposed using retransmission of lost packets by delaying frame playout times to allow arrival of retransmitted packets before the display times of the video frames associated with the packets. Any packet received after the display time of its associated video frame will be discarded. In these schemes, the display time of a frame is delayed by at least three one-way trip times after its initial transmission (two for frame transmission and one for a retransmission request). This latency can significantly impair the interactiveness of any video applications under the current Internet.
Forward error correction (FEC) is commonly proposed for error recovery of continuous media transmission. However, conventional FEC schemes do not work well for interactive video. The reason that conventional FEC schemes do not work well with interactive video is that unless the playout time of a frame is delayed, both the original packets and their parity or FEC packets must be transmitted within the same frame interval, rendering the schemes very susceptible to burst loss. Moreover, since FEC is applied to a block of packets, before FEC packets are computed and transmitted, a large delay can occur.
Commonly-assigned, co-pending U.S. patent application Ser. No. 09/079,621, filed May 15, 1998 now U.S. Pat. No. 6,104,757, and entitled “SYSTEM AND METHOD OF ERROR CONTROL FOR INTERACTIVE LOW-BIT RATE VIDEO TRANSMISSION” (hereinafter, “the RESCU Patent Application”) discloses a complementary approach to the above-mentioned approaches by focusing on eliminating error propagation when distortion on displayed images occurs. This approach is referred to as recovery from error spread using continuous updates (RESCU). The point of departure from existing approaches described in the RESCU Patent Application is that packets do not have to arrive in time for them to be “useful” for display of that video frame. Of course, if packets can arrive before the display times of their frames, that is optimal. However, due to packet losses and high latency, repair packets inevitably arrive “late,” causing distortion in displayed images, which can propagate to successive frames. These late repair packets can be used to stop error propagation. In motion-compensated codes, the correct display of a frame depends on the successful reception of all of its reference frames. If displayed frames are buffered and late packets are used to restore errors in the buffered frames, error propagation can be stopped. The reason that error propagation can be stopped is that the buffered frames will be used as reference frames for later frames.
RESCU has been shown to be effective for interactive video transmission when retransmission is used to recover lost packets and round trip delays are small. Retransmission tends to prolong error propagation because of the delay involved in detecting and retransmitting lost packets. Moreover, in some networks, such as wireless, cable modems, and direct satellites, feedback channels are highly contentious, and bandwidth-limited. Thus, in these networks, frequent transmission of feedback to the sender is too expensive. Accordingly, there exists a need for improved methods and systems for performing error recovery when transmitting compressed video over a lossy packet based network.
DISCLOSURE OF THE INVENTION
According to one aspect, the present invention includes a new FEC technique for interactive video that combines FEC with RESCU. By incorporating this FEC technique, RESCU can perform very effectively in an environment where little or no feedback is available, or transmission delay is too high for retransmission to be effective. The FEC scheme according to the present invention clearly differs from the conventional schemes in that FEC packets can be transmitted over a longer period than a single frame interval without introducing delay in frame playout times. Since RESCU uses FEC packets to restore buffered reference frames (referred to herein as periodic frames), FEC packets can be transmitted over a relatively longer period, interleaving with the packets of other (non-periodic) frames to help reduce the effect of bursty losses. This interleaving is different from link-level symbol interleaving where symbols from multiple codewords are interleaved. The granularity of interleaving according to the present invention is much larger and thus, more effective than link-level interleaving. Since RESCU makes non-periodic frames temporally depend only on the immediately preceding periodic frame and, unlike retransmission, FEC involves no feedback delay, the proposed technique incurs shorter recovery delays and accordingly shorter error propagation. Thus, the present invention can be effective for high frame rate transmission over lossy, high-latency networks.
Accordingly, it is an object of the present invention to provide improved methods and systems for reducing error spread in video transmission over a packet-based network.
Some of the objects of the invention having been stated above herein, other objects will become evident as the description proceeds when taken in connection with the accompanying drawings as best described herein below.
REFERENCES:
patent: 5680322 (1997-10-01), Shinoda
patent: 5767907 (1998-06-01), Pearlstein
patent: 5794018 (1998-08-01), Vrvilo et al.
patent: 6104757 (2000-08-01), Rhee
Rhee et al., “FEC-Based Loss Recovery for Interactive Video Transmission, ” Proceedings of IEEE International Conference on Multimedia Computing and Systems, vol. 1, pp. 205-256, Florence, Italy, (Jun. 1999).
Rhee, “Error Conrtol Techniques for Interactive Low-Bit Rate Video Transmission Over the Internet,” Proceeding of the ACM SIGCOMM '98, pp. 290-301, (Sep. 1998).
Rhee, “Retransmission-Based Error Control for Interactive Video Applications Over the Internet,” Proceedings of the International Conferecne on Multimedia Computing and Systems, pp. 118-127, (Jun. 1998).
Talluri, “Error-Resilient Video Coding in ISO MPEG-4 Standard,” IEEE Communication Magazine, vol. 36 (No. 6), pp. 112-119, (Jun. 1998).
Podolsky et al., “Stimulation of FEC-Based Error Control for Packet Audio on the Internet,” Proceedings of the ACM SIGMETRIC/PERFORMANCE, pp. 505-515, (Jun. 1998).
Carle et al., “Survey of Error Recovery Techniques
Britton Howard
Jenkins & Wilson, P.A.
North Carolina State University
LandOfFree
Methods and systems for forward error correction based loss... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Methods and systems for forward error correction based loss..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Methods and systems for forward error correction based loss... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2842335