Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
2000-04-13
2004-07-06
Vo, Tung T. (Department: 2713)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
Reexamination Certificate
active
06760377
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a signal processor, a method of signal processing, and a computer program product for implementing the method. The invention concerns joining digital bit streams which are compressed: that is referred to herein as splicing. Embodiments of the invention described herein are concerned with splicing digital video bitstreams which are compressed according to the MPEG-2 standard.
2. Description of the Prior Art
The invention and its background will be discussed by way of example with reference to MPEG-2 video bitstreams. However the invention is not limited to MPEG-2.
MPEG-2 is well known from see for example ISO/IEC/13818-2, and will not be described in detail herein. Splicing of video is well known. It is used in editing video. Splicing analogue signals is relatively straight forward and can be done at the boundary between adjacent frames, because each analogue frame contains the whole of the video information of that frame independently of other frames. Splicing can be done similarly in the digital domain for both compressed and uncompressed video data if all frames contain the whole video information of the frame.
MPEG-2 compressed video comprises groups of I and P and/or B frames known as GOPs, Groups of Pictures. I, P and B frames are well known. An I or Intra-encoded frame contains all the information of the frame independently of any other frame. A P frame in a GOP ultimately depends on an I frame and may depend on other P frames. A B frame of a GOP ultimately depends on an I-frame and may depend on P frames in the GOP, but not on another B frame.
A GOP typically comprises 12 or 15 frames comprising at least one I frame and several P and B frames. To correctly decode a GOP requires all the frames of the GOP, because a large part of the video information required to decode a B frame in the GOP is in a preceding and/or succeeding frame of the GOP. Likewise a large part of the video information required to decode a P frame is in a preceding frame of the GOP.
Thus if two different bit streams are spliced together in the compressed domain, the information necessary to decode frames each side of the splice point is likely to be lost.
Many papers have been written concerning the splicing of compressed bitstreams, which is a well known problem in MPEG. A paper “Flexible Switching and editing of MPEG-2 Video Bitstreams” by P. J. Brightwell, S. J. Dancer and M. J. Knee was published in “Atlantic Technical Papers 1996/1997” the preface to which is dated September 1997.
The paper discusses the problems of splicing MPEG-2 Video Bitstreams. Two bitstreams A and B to be spliced are decoded in respective decoders. A coder is switched from the decoder of A to the decoder of B at the splicing point. It discloses that near a splicing point where a bitstream A is replaced by a bitstream B, the following modifications are made.
“The picture type may be changed to provide a more suitable refresh strategy around the switch point. In the example below, the first P-frame in bitstream B after the switch is converted to an I-frame to provide a full refresh early in the new scene. Also, bitstream A contains an I-frame just before the switch point—as this is unnecessary, it is recoded as a P-frame to save bits.
Switch point
bitstream
A:
P
B
B
I
|
. . .
bitstream
B:
. . .
|
B
P
B
B
P
B
B
P
modified
P
B
B
P
|
B
I
B
B
P
B
B
P
Prediction modes and motion vectors may require modification to take into account any changes in the picture type on recoding, or to prevent any predictions being made across the switch on recoding. In the example above, macroblocks that originally used forward or bi-directional prediction for the B-frame following the switch point will be recoded using intra mode and backward prediction respectively. In addition, vectors are required for the I-frame that is recoded as a P-frame—these can be estimated from the vectors in surrounding frames, or taken from I-frame concealment vectors that many MPEG-2 bitstreams carry.
The quantisation parameters will be changed as part of the recoder's rate control strategy. As in a conventional coder, this aims to control the buffer trajectory of a downstream decoder to prevent under- or overflow, and to maintain the picture quality as high as possible. In addition, the rate control algorithm for the ATLANTIC switch uses the vbv_delay values in bitstreams A an B (which are carried in the info-bus) to make the buffer trajectory for the switched bitstream identical to that for bitstream B (i.e. the one being switched to) at some future time. Depending on the relative vbv_delay values, this may happen soon after the switch, or a recovery period of a few GOPs may be required. When it has been achieved, the recoder's quantisation parameters are locked to those of bitstream B, and the switch becomes transparent.
The quantisation parameters may also be changed to take advantage of an effect known as temporal masking. This refers to the eye's inability to see moderate or even large amounts of noise around a scene change—typically 5 dB of degradation in the frame after the switch cannot be seen—and allows the number of bits used for the frames very close to the switch point to be reduced, allowing a shorter recovery period.”
“Vbv-delay values” are measures of the number of bits in the buffer of the down stream decoder. The manner in which the “buffer trajectory” for the switch bitstream is made identical to that for bitstream B is not disclosed in the paper. Also the time scale over which that happens may be “soon after the switch” or after a few GOPs.
SUMMARY OF THE INVENTION
It is desirable to make the buffer trajectory for the switched bit stream identical to that for the bitstream B in a more predictable time-scale, preferably one GOP.
According to one aspect of the present invention, there is provided a signal processor for splicing a compressed bitstream B to a compressed bitstream A, comprising
means for decoding the bitstreams A and B, means for switching from decoded bitstream A to decoded bitstream B to splice bitstream B to bitstream A to produce a spliced bitstream C, encoding means for re-encoding the spliced bitstream C for supply to a downstream decoder having a downstream buffer, wherein the encoding is controlled by control means to produce a prediction of the position on bitstream B at which occupancy of the downstream buffer by spliced bitstream C equals what would have been the occupancy by bitstream B alone, and
the value of occupancy for bitstream C is controlled between the splice point and the said predicted position in accordance with the difference between the occupancy value of stream A immediately before the splice and a prediction of the occupancy value of bitstream C at the said predicted position.
Thus by providing predictions of both the position at which occupancy of stream C equals that of B, and of the occupancy value, and controlling the occupancy value in accordance with the predictions, lock tends to be achieved more reliably in a more predictable time scale.
Another aspect of the present invention provides a signal processing method for splicing a compressed bitstream B to a compressed bitstream A, comprising
decoding the bitstreams A and B, switching from decoded bitstream A to decoded bitstream B to splice bitstream B to bitstream A to produce a spliced bitstream C, re-encoding the spliced bitstream C for supply to a downstream decoder having a downstream buffer, wherein the encoding is controlled to produce a prediction of the position on bitstream B at which occupancy of the downstream buffer by spliced bitstream C equals what would have been the occupancy by bitstream B alone, and the value of occupancy for bitstream C is controlled between the splice point and the said predicted position in accordance with the difference between the occupancy value of stream A immediately before the splice and a prediction of the occupancy value of bitstream C at the said predicted position.
Yet another aspect
Burns James Edward
Porter Robert Mark Stefan
Saunders Nicholas Ian
Frommer William S.
Frommer & Lawrence & Haug LLP
Sony United Kingdom Limited
Vo Tung T.
LandOfFree
Signal processing does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Signal processing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Signal processing will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3200522