Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
2000-04-27
2001-09-11
Britton, Howard (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C375S240120, C714S018000, C714S048000
Reexamination Certificate
active
06289054
ABSTRACT:
TECHNICAL FIELD
The present invention relates generally to methods and systems for transmitting video over a lossy packet-based network. More particularly, the present invention relates to methods and systems for dynamic hybrid packet loss recovery for video transmission over a lossy packet-based network.
BACKGROUND ART
Packet losses are common in a lossy packet-based network, such as the Internet. For example, during high-traffic time periods, about 5% to 10% packet losses over connections between the east and west coasts of the United States, or over trans-Atlantic or trans-Pacific connections are not unusual. Since packet losses in a lossy packet-based network, such as the current best effort delivery Internet, cannot be avoided, applications such as Internet-based video telephony must be structured to be tolerant of packet loss.
Unfortunately, the quality of compressed video is very susceptible to packet loss because of motion estimation employed in video compression. Motion estimation is the process of estimating the displacement of moving objects in a video sequence. Motion estimation is currently used in popular video compression and decompression algorithms, commonly referred to as codecs, such as H.261, H.263, MPEG-1, MPEG-2, and MPEG-4, to remove temporal redundancy in successive video frames. The temporal redundancy is removed by encoding only pixel value differences between the current image and its motion-predicted image reconstructed from a previously encoded image. The previously encoded image is referred to as a reference frame or R-frame. In these codecs, loss of packets for a particular video frame manifests itself not only in the reduced quality of the frame in which the loss occurs, but also in subsequent frames due to propagation of distortion to the successive frames that reference, either directly or indirectly, the erroneously received frame. This problem is referred to as the error propagation or error spread problem.
Most of the conventional work on loss recovery focuses on recovering packet losses using retransmission and forward error correction before the scheduled displayed times of the video frames contained in the lost packets. However, this approach is ineffective for interactive video because of the delays in detecting and repairing the losses. For example, in order to allow time for loss detection and repair, existing techniques introduce delay in the frame display times. Delaying the frame display times greatly impairs the effectiveness of interactive video communication.
The above-referenced parent U.S. patent application, entitled “RECOVERY FROM ERROR SPREAD USING CONTINUOUS UPDATES (RESCU),” discloses methods and systems for preventing error spread. Unlike conventional techniques, RESCU focuses on eliminating error propagation instead of preventing errors due to packet loss before display of a particular video frame. In a lossy packet-based network, such as today's Internet, where packet losses and high network latency are common, recovering lost packets before the display times of the frames associated with the packets is not always possible. Therefore, some repair packets might arrive after the display times of the associated video frames. Conventional techniques discard these late repair packets. In contrast, RESCU can use the repair packets to stop error propagation. Stopping error propagation is accomplished by buffering displayed frames, restoring the buffered frames when repair packets arrive, and using the buffered frames as reference frames for subsequent frames.
The main benefit of RESCU is that it allows more time for transport-level recovery to succeed. That is, repair packets for a frame are useful until that frame is being used as a reference frame. In order to accommodate recovery delays, RESCU designates every P
th
frame, P being an integer, as a periodic frame. The number of frame intervals between two consecutive periodic frames is referred to as the periodic temporal dependency distance (PTDD). Every frame between periodic frames references only its immediately preceding periodic frame. Such frames are referred to as non-periodic frames.
FIG. 1
illustrates an example of RESCU picture coding with a PTDD value of 2 according to the algorithms described in the above-referenced parent application. In
FIG. 1
, blocks F
0
through F
6
represent frames. Blocks F
0
, F
2
, F
4
and F
6
are periodic frames. Blocks F
1
, F
3
and F
5
are non-periodic frames. As illustrated in
FIG. 1
, the periodic temporal dependency distance is measured from the start of one periodic frame to the next periodic frame. In the illustrated example, the periodic temporal dependency distance is equal to two frames.
When packet losses occur during transmission of a periodic frame, repair packets can be transmitted to repair the losses. The repair packets can be retransmitted packets or forward error correction (FEC) packets that arrive before the decoding time of the next periodic frame. If the FEC or retransmitted packets arrive before the decoding of the next periodic frame, error propagation can be stopped by using the repair packets to repair the reference frame of the next periodic frame. For example, in
FIG. 1
, if frame F
0
is transmitted with errors, and repair packets arrive before display of frame F
2
, the repair packets can be used to repair the buffered F
0
before F
0
is used as a reference for frame F
2
. Accordingly, error propagation due to errors in frame F
0
will not extend beyond frame F
1
Because frames do not reference non-periodic frames, loss in non-periodic frames does not cause error propagation. Since no attempts are made to restore non-periodic frames after display of the non-periodic frames, only periodic frames being recovered need to be buffered for future reference by succeeding periodic frames. This reduces the memory requirements for error recovery.
FIGS. 2 and 3
illustrate the benefits of RESCU. In
FIG. 2
, an error, indicated by distortion in the image of the person, that begins in periodic frame F
0
propagates to subsequent frames F
1
, F
2
, and F
3
. In
FIG. 3
, RESCU is used with a periodic temporal dependency distance of 2. Accordingly, an error that occurs in F
1
is repaired before display of frame F
2
. Accordingly, the error occurring in frame F
1
does not propagate to frame F
2
or successive frames.
Conventional applications of RESCU explore the effectiveness of RESCU using a fixed PTDD. The PTDD determines a deadline within which packets need to be recovered. However, network conditions vary over time. For example, congestion, transmission latency, loss rates and available bandwidth frequently change. As network conditions change, the effectiveness of transport level recovery, i.e., retransmission and FEC, change, and thus the associated recovery delays change. Conventional techniques that utilize a fixed PTDD period are incapable of adapting to changing network conditions. Accordingly, there exists a long-felt need for methods and systems for adjusting the PTDD period to account for changing network conditions.
DISCLOSURE OF THE INVENTION
The methods and systems for performing packet loss recovery for video transmission over a lossy packet-based network according to the present invention include varying the PTDD value in response to network conditions to allow sufficient time for packets to be delivered before recovery of succeeding periodic frames. The PTDD value cannot be set arbitrarily large because this reduces compression efficiency. Thus, finding the minimum PTDD value under given network conditions that maximizes periodic frame recovery is an important aspect of the invention.
Packet loss rate, loss burst length, and transmission delays play an important role in determining PTDD. For example, if the loss rate increases, then additional FEC packets or retransmission attempts are required to maintain high loss recovery probabilities. Thus, the PTDD requires extension to accommodate time required for the increased number of repair attempts. Loss burst characteristics, i.e., th
Britton Howard
Jenkins & Wilson, P.A.
North Carolina University
LandOfFree
Method and systems for dynamic hybrid packet loss recovery... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and systems for dynamic hybrid packet loss recovery..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and systems for dynamic hybrid packet loss recovery... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2527615