Image analysis – Image compression or coding
Reexamination Certificate
1998-10-15
2003-08-26
Chen, Wenpeng (Department: 2724)
Image analysis
Image compression or coding
C375S240260, C370S498000
Reexamination Certificate
active
06611624
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to systems and methods for splicing two compressed bitstreams together to form a single compressed bit stream. In particular, the present invention relates to a system and a method for video bitstream splicing that is frame accurate. Still more particularly, the present invention relates to a system and method for seamless splicing of MPEG-2 video bitstreams.
2. Description of the Related Art
Digital video compression is a process that removes redundancy in the digital video pictures such that the resulting representation of the signal contains much smaller number of bits than that in the original uncompressed pictures. The redundancy in the digital video sequences, consist of a sequence of digital pictures played out in a time continuous manner, is reflected in the form of spatial redundancy in a picture and temporal redundancy between pictures. MPEG-2 compression takes advantage of these redundancies by efficient coding of the spatial digital image content and temporal motion content. Algorithms for MPEG-2 video compression are well known in the art.
Digital stream insertion (also called digital program insertion (DPI), digital spot insertion, etc.) is a process that replaces part of a digital compressed bitstream by another compressed bitstream, which may have been encoded off-line in a different location or at a different time. This process is illustrated via FIG.
1
. In the figure, part of bitstream
1
is replaced by bitstream
2
. In real applications, bitstream
1
may be a real-time feed from the distribution network, and bitstream
2
may be a section of advertisement that is to be inserted into the network feed. As a result of the insertion, the resulting bitstream has the advertisement inserted into the network bitstream feed. Since this is the main application of DPI, we may refer in this application to bitstream
1
as the live stream, and bitstream
2
the stored stream.
The underlying technique for DPI is bitstream splicing (also known as bitstream concatenation), where a transition is made from an old stream to a new stream. The transition is called splice point. The splicing process in its most general form could be between a first and a second stream that are continuously playing and the transition is from the middle of the first stream to the middle of the second stream. However, in the context of DPI, two special cases are of interest. Each insertion involves two splice points: an exit point and an entry point. Here, we define the exit point as the transition from the middle of the first (live) stream to the beginning of a second (stored) stream. We define the entry point as the transition from the end of the second (stored) stream to the middle of the first (live) stream. Both of these points are splice points, and are illustrated in FIG.
1
.
One prior art method for performing splicing is the use of analog splicing equipment. In this case, the two signals to be switched are time continuous pictures. The splicing equipment, at the time of splicing, turns off one signal at the picture frame boundary and turns on the other signal, resulting in a scene change to the viewer. The two signals are assumed to be frame synchronized, the time for the splicing is well defined. However, to splice two digitally compressed bitstreams, the situation is much more complex. This is due to the nature of the motion compensation and variable length encoding of digital video pictures. Specifically, compressed or encoded digital pictures do not necessarily have the same number of bits; in addition, the digital picture content is reconstructed not from a single encoded digital picture, but from several of them via motion compensation. More specifically and as show in
FIGS. 2A and 2B
, the bitstreams are compose of a number of frames or pictures. The MPEG standard defined three types of pictures: intra, predicted and bi-directional. Intra pictures or I-pictures are coded using only information present in the picture itself. Predicted pictures or P-pictures are coded with respect to the nearest previous I- or P-picture as show in FIG.
2
A. Bi-directional pictures or B-pictures are coded with reference to the two most recent sent I/P-pictures as illustrated by FIG.
2
B.
The basic requirement for any digital bitstream splicing system or method is that the splicing must not introduce any visible artifacts into the decoded video of the resulting bitstream. There have been some prior art method for digital bitstream splicing, however, they are not able to provide frame accurate splicing. These prior art method resort to imposing several constraints to the formats of the streams being spliced, thus not providing seamless and frame-precise splicing. For example, some prior art methods allow splicing only immediately before an I-frame. This is problematic because there could be as many as 30 frames between I-frames. Other methods require that the bit rates of the spliced streams be the same and constant. Yet other prior art methods require that the stored stream start with an I-picture and/or the live stream start with an I-picture right after the stored stream has ended. Thus, such prior art methods do not provide for seamless and frame-precise splicing where splicing is allowed to happen at any picture location (this is what frame-precise or accurate splicing means).
A particular problem with the slicing methods of the prior art is that they require both streams to have the same constant bit rate. Splicing of bitstreams may occur in either constant bit rate (CBR) environment or variable bit rate (VBR) environment. In the CBR environment, the live stream has a bit rate, R
1
, that is constant throughout the transmission. In order to splice the stored stream with bit rate, R
2
, the two bit rates must be identical, R
1
=R
2
. For example, assuming the channel transmits at rate R
1
before and after splicing, if R
1
>R
2
, then the decoder buffer will overflow, shortly after the exit point, causing corruption in the decoded pictures, and if R
1
<R
2
, the decoder buffer will under flow, shortly after the entry point, which causes the decoder to freeze displayed pictures. This rate mismatch can be solved by either stuffing of the stored stream if R
1
>R
2
, or by rate reduction if R
1
<R
2
. Thus, the prior art has not provided a splicing system and method able to handle streams with different rates in real time.
Another more generalized look at the same problem described above is the buffer compliance problem (i.e., the problem of matching coded picture bit usage to that of the data delivery rate of the transmission channel). This problem is what is called the rate control problem. The MPEG-2 encoding process must be under a rate control algorithm. Specifically, the encoder rate control must ensure that the decoder buffer cannot under flow or overflow. If no splicing is performed in the bitstream, this responsibility is taken entirely by the encoder. However, when splicing is performed, two bitstreams are involved, each is possibly encoded at a different time by a different encoder. In this case, the decoder's buffer trajectory just before splicing is defined by the rate control algorithm running on the live stream encoder. Starting from the splicing point, the new stream encoding rate control takes over, and this is where the buffer compliance of the decoder may be violated. To see this, consider the following example shown in FIG.
3
. The figure describes the decoder buffer behavior during the splicing. In the example shown, assuming two CBR bitstreams are spliced together without change, and assuming that the first picture of the new stream replaces the B picture right after the last I
1
-picture in the old stream is removed from the decoder buffer. Since the first stream encoder knows that the next picture in the first stream is a B
2
-picture, it will thus assume that the decoder buffer will not under flow given that B
2
has fewer bits. However, unknown to the
Tse Yi Tong
Zhang Ji
Beyer Weaver & Thomas LLP
Chen Wenpeng
Cisco Systems Inc.
LandOfFree
System and method for frame accurate splicing of compressed... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for frame accurate splicing of compressed..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for frame accurate splicing of compressed... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3102418